1
0
mirror of https://github.com/mail-in-a-box/mailinabox.git synced 2025-03-30 23:37:05 +00:00

Merge branch 'master' into quota

This commit is contained in:
downtownallday 2025-02-16 16:29:01 -05:00
commit 271a04333d
40 changed files with 323 additions and 846 deletions

View File

@ -1,5 +1,10 @@
name: commit-tests
on: [push, workflow_dispatch]
on:
workflow_dispatch: { }
push:
branches: [ '**' ]
tags-ignore: [ 'v**' ]
paths-ignore: [ '**.md', 'api/mailinabox.yml' ]
jobs:
# MiaB-LDAP using encryption-at-rest and connected to a remote Nextcloud
remote-nextcloud-docker-ehdd:

View File

@ -1,6 +1,37 @@
CHANGELOG
=========
Version 71 (January 4, 2025)
----------------------------
(Version 71a was posted on January 6, 2025 and fixes a setup regression.)
Upgrades
* Roundcube upgraded to version 1.6.9.
* Z-Push upgraded to version 2.7.5.
Automated Maintenance
* Daily automated tasks are now run at 1am in the box's timezone and full backups are now restricted to running only on Saturdays and Sundays at that time.
* Backups now exclude the owncloud-backup folder so that we're not backing up backups.
* Old TLS certificates are now automatically deleted to improve control panel performance.
Setup
* Fixed broken setup if SSH was configured to listen on multiple ports.
* Ubuntu MOTD advertisements are now disabled.
* Fixed missing Roundcube dependency package if NextCloud isn't installed.
Control Panel
* Improved status checks for secondary nameservers.
* Spamhaus is now queried for the box's IPv6 address also.
* DSA and EC private keys are now accepted for TLS certificates.
* Timeouts for loading slow control panel pages are reduced.
And other minor fixes.
Version 70 (August 15, 2024)
----------------------------
@ -73,7 +104,7 @@ Version 64 (September 2, 2023)
* Fixed backups to work with the latest duplicity package which was not backwards compatible.
* Fixed setting B2 as a backup target with a slash in the application key.
* Turned off OpenDMARC diagnostic reports sent in response to incoming mail.
* Fixed some crashes when using an unrelased version of Mail-in-a-Box.
* Fixed some crashes when using an unreleased version of Mail-in-a-Box.
* Added z-push administration scripts.
Version 63 (July 27, 2023)
@ -1129,7 +1160,7 @@ Control panel:
System:
* The munin system monitoring tool is now installed and accessible at /admin/munin.
* ownCloud updated to version 8.0.4. The ownCloud installation step now is reslient to download problems. The ownCloud configuration file is now stored in STORAGE_ROOT to fix loss of data when moving STORAGE_ROOT to a new machine.
* ownCloud updated to version 8.0.4. The ownCloud installation step now is resilient to download problems. The ownCloud configuration file is now stored in STORAGE_ROOT to fix loss of data when moving STORAGE_ROOT to a new machine.
* The setup scripts now run `apt-get update` prior to installing anything to ensure the apt database is in sync with the packages actually available.
@ -1167,7 +1198,7 @@ DNS:
* Internationalized Domain Names (IDNs) should now work in email. If you had custom DNS or custom web settings for internationalized domains, check that they are still working.
* It is now possible to set multiple TXT and other types of records on the same domain in the control panel.
* The custom DNS API was completely rewritten to support setting multiple records of the same type on a domain. Any existing client code using the DNS API will have to be rewritten. (Existing code will just get 404s back.)
* On some systems the `nsd` service failed to start if network inferfaces were not ready.
* On some systems the `nsd` service failed to start if network interfaces were not ready.
System / Control Panel:

37
Vagrantfile vendored
View File

@ -1,37 +0,0 @@
# -*- mode: ruby -*-
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/jammy64"
# Network config: Since it's a mail server, the machine must be connected
# to the public web. However, we currently don't want to expose SSH since
# the machine's box will let anyone log into it. So instead we'll put the
# machine on a private network.
config.vm.hostname = "mailinabox.lan"
config.vm.network "private_network", ip: "192.168.56.4"
config.vm.provision :shell, :inline => <<-SH
# Set environment variables so that the setup script does
# not ask any questions during provisioning. We'll let the
# machine figure out its own public IP.
export NONINTERACTIVE=1
export PUBLIC_IP=auto
export PUBLIC_IPV6=auto
export PRIMARY_HOSTNAME=auto
#export SKIP_NETWORK_CHECKS=1
# Start the setup script.
cd /vagrant
setup/start.sh
SH
end

42
changelog/v71.md Normal file
View File

@ -0,0 +1,42 @@
## Commits for v71
| COMMIT | DATE | AUTHOR | TITLE |
| ------ | ---- | ------ | ----- |
| [c5e33b51](https://github.com/downtownallday/mailinabox-ldap/commit/c5e33b51e5d112a09420d242c3d8cf1c23aeeafa) | 2025-01-04 | _downtownallday_ | Update license |
| [d58dd0c9](https://github.com/downtownallday/mailinabox-ldap/commit/d58dd0c91dd677acd6940d9b6099e2abb0ede729) | 2025-01-04 | _Joshua Tauberer_ | v71 |
| [f73da3db](https://github.com/downtownallday/mailinabox-ldap/commit/f73da3db60fc221fd2ecae17eac16db426800b2b) | 2025-01-04 | _Joshua Tauberer_ | Fix likely merge mistake in 564ed59bb47da24c9ebc50ae9137e6dcbcae9826 |
| [4c2e4bab](https://github.com/downtownallday/mailinabox-ldap/commit/4c2e4bab29a1030d87f5213b12dcfb38bcec2e83) | 2024-12-22 | _downtownallday_ | fix error when glob matches nothing (variable 'file' will have the glob as a value in the for loop and produce the error "mv: cannot stat '/home/user-data/ssl/*-+([0-9])-+([0-9a-f]).pem': No such file or directory") |
| [18721e42](https://github.com/downtownallday/mailinabox-ldap/commit/18721e42d19e87df5b7ba0182525739928dd39fa) | 2024-12-22 | _yeah_ | Cronjob for cleaning up expired SSL certificates in order to improve page load times with many domains (#2410) |
| [e0b93718](https://github.com/downtownallday/mailinabox-ldap/commit/e0b93718a33338115e953564170d87af6a55e1f9) | 2024-12-22 | _yeah_ | Revert "increase timeout for the nginx proxy that provides access to the Mail…" (#2411) |
| [2e0482e1](https://github.com/downtownallday/mailinabox-ldap/commit/2e0482e1817fd1a167b247a9137b86ee190d2947) | 2024-12-22 | _KiekerJan_ | Exclude the owncloud-backup folder from the nightly backup (#2413) |
| [0d738889](https://github.com/downtownallday/mailinabox-ldap/commit/0d7388899c02a3785714bfe75d711f5929b3ded2) | 2024-12-22 | _Tomasz Stanczak_ | Allow DSA end EllipticCurve private keys to be used additionally to RSA for HTTPS certificates (#2416) |
| [4f094f78](https://github.com/downtownallday/mailinabox-ldap/commit/4f094f7859cab6ee72792b96313c1c7d4407685d) | 2024-12-22 | _zoof_ | Change hour of daily tasks to run at 1am and only run full backups on weekends (#2424) |
| [564ed59b](https://github.com/downtownallday/mailinabox-ldap/commit/564ed59bb47da24c9ebc50ae9137e6dcbcae9826) | 2024-12-22 | _KiekerJan_ | Add check on ipv6 for spamhaus (#2428) |
| [9f87b36b](https://github.com/downtownallday/mailinabox-ldap/commit/9f87b36ba182e5ec6e519a4a6c27e9ead8c08469) | 2024-12-22 | _KiekerJan_ | add check on SOA record to determine up to date synchronization of secondary nameserver (#2429) |
| [e36c17fc](https://github.com/downtownallday/mailinabox-ldap/commit/e36c17fc72249fef1eb6b638c4fa3ad2ad765d32) | 2024-12-22 | _matidau_ | Fixstates only after Z-Push upgrade (#2432) |
| [3d59f2d7](https://github.com/downtownallday/mailinabox-ldap/commit/3d59f2d7e0d0c2794f88fc36d5fca11fc757f9a7) | 2024-12-22 | _KiekerJan_ | Update roundcube to 1.6.9 (#2440) |
| [ee0d750b](https://github.com/downtownallday/mailinabox-ldap/commit/ee0d750b8560b0e2e9a9bf0afe52ed12982cb7f2) | 2024-12-22 | _Harm Berntsen_ | Add missing php-xml package for Roundcube without Nextcloud (#2441) |
| [d8563be3](https://github.com/downtownallday/mailinabox-ldap/commit/d8563be38b2fa047725ee85c7330bdf775101cdd) | 2024-12-22 | _Paul_ | Disable MOTD advertisements (#2457) |
| [81b0e0a6](https://github.com/downtownallday/mailinabox-ldap/commit/81b0e0a64f3ed295205dbc5461bb8f4fc2791e3d) | 2024-12-22 | _Nicholas Wilson_ | Updated CHANGELOG.md, fix typo(s) (#2459) |
| [7ef859ce](https://github.com/downtownallday/mailinabox-ldap/commit/7ef859ce961ea24b70f7e4f8307f069a8f7b42b3) | 2024-12-13 | _matidau_ | Update zpush.sh to version 2.7.5 (#2463) |
| [a8d13b84](https://github.com/downtownallday/mailinabox-ldap/commit/a8d13b84b4e2ac7332ae825177c4f9aa7a01e782) | 2024-11-27 | _Downtown Allday_ | fix: NameError: name 'subprocess' is not defined (#2425) |
| [196f5588](https://github.com/downtownallday/mailinabox-ldap/commit/196f5588cc61e6531cda9491f3eb26f152630528) | 2024-10-07 | _downtownallday_ | eliminate the use of deprecated utcnow() |
| [119b11f0](https://github.com/downtownallday/mailinabox-ldap/commit/119b11f0227b6565148221bbd0bccb6ef4011a15) | 2024-10-04 | _downtownallday_ | remove upstream Vagrantfile |
| [696b597a](https://github.com/downtownallday/mailinabox-ldap/commit/696b597a9c4a7beea1e76ff5ade57f94bbd9770e) | 2024-10-04 | _downtownallday_ | use bash as 'source' is needed in provision scripts |
| [ae056e50](https://github.com/downtownallday/mailinabox-ldap/commit/ae056e507beaf86272dc5cdc20545f5d9c2ae41c) | 2024-10-04 | _downtownallday_ | validate argument |
| [3b6e6177](https://github.com/downtownallday/mailinabox-ldap/commit/3b6e6177d03c4fc10d3c3afe07760e8f4b49f181) | 2024-10-04 | _downtownallday_ | Remove vagrant references - everything has moved to lxd |
| [706c3e7a](https://github.com/downtownallday/mailinabox-ldap/commit/706c3e7af93add6ced094a473ebbc57716cd03f7) | 2024-09-20 | _downtownallday_ | QA: updates for recent nextcloud change |
| [62b691f4](https://github.com/downtownallday/mailinabox-ldap/commit/62b691f44a2f201f55052831e2b1ddad962a9cb9) | 2024-09-20 | _downtownallday_ | QA: updates for recent nextcloud changes |
| [1699ab8c](https://github.com/downtownallday/mailinabox-ldap/commit/1699ab8c02e6813075a65fff9903c85e31d52445) | 2024-09-17 | _matidau_ | Update zpush.sh to version 2.7.4 (#2423) |
| [3e0a6214](https://github.com/downtownallday/mailinabox-ldap/commit/3e0a6214508724496ce2c629b598cedf4be1b22c) | 2024-09-10 | _downtownallday_ | allow supplying a command line to execute to ssh remove debugging echo statements add -q argument to suppress outputting lxc command line |
| [4fedfb37](https://github.com/downtownallday/mailinabox-ldap/commit/4fedfb377da393d7fe22ca8781a964e9759a18ce) | 2024-09-10 | _downtownallday_ | during wait for boot, also wait until vm has an ip address |
| [2e0b37a0](https://github.com/downtownallday/mailinabox-ldap/commit/2e0b37a09a964c0e7499c2bff7d4f2d25361e9d9) | 2024-09-07 | _downtownallday_ | fix syntax error |
| [6d25bc47](https://github.com/downtownallday/mailinabox-ldap/commit/6d25bc47bf20ba48c53bc861962e9356713fcbfa) | 2024-09-05 | _downtownallday_ | add a restart command |
| [54a3bd10](https://github.com/downtownallday/mailinabox-ldap/commit/54a3bd100c43710800ce3208acf1380e071bc0a3) | 2024-09-04 | _downtownallday_ | Add provision defaults to lxc init |
| [0fce66db](https://github.com/downtownallday/mailinabox-ldap/commit/0fce66dbc7e3c46b08d31698deca27badfbb0682) | 2024-09-03 | _downtownallday_ | back out assert_kernel_modules |
| [446aacb9](https://github.com/downtownallday/mailinabox-ldap/commit/446aacb9b6ae4dce8c021c8d1a4b09efefebabdb) | 2024-09-03 | _downtownallday_ | Don't exit on missing kernel module during non-interactive scenario |
| [c027db8b](https://github.com/downtownallday/mailinabox-ldap/commit/c027db8bf49091e6d4cef214722a062635e80d3c) | 2024-09-03 | _downtownallday_ | reword comment |
| [ca123515](https://github.com/downtownallday/mailinabox-ldap/commit/ca123515aad102327701b18a7d65d180f800b815) | 2024-09-02 | _Downtown Allday_ | fix variable (#2439) |
| [a1d6f671](https://github.com/downtownallday/mailinabox-ldap/commit/a1d6f6713578097b1b68bb0cea80f6327a2c3577) | 2024-09-02 | _downtownallday_ | change from vagrant to lxd as the virtualization system |
| [a79a6c00](https://github.com/downtownallday/mailinabox-ldap/commit/a79a6c00eb252de8c2581744894c8173a34b2f92) | 2024-09-02 | _downtownallday_ | encryption-at-rest: Ensure required kernel modules are installed |
| [3b8f4a2f](https://github.com/downtownallday/mailinabox-ldap/commit/3b8f4a2fe8bd686f9d3ff405d9bb380c3c6315a8) | 2024-08-30 | _matidau_ | Z-Push remove config lines no longer supported (#2433) |
| [f453c44d](https://github.com/downtownallday/mailinabox-ldap/commit/f453c44d524b68a3a99f567168dd401f88556633) | 2024-08-30 | _darren_ | Update setup to handle multiple SSH ports (#2437) |

5
changelog/v71a.md Normal file
View File

@ -0,0 +1,5 @@
## Commits for v71a
| COMMIT | DATE | AUTHOR | TITLE |
| ------ | ---- | ------ | ----- |
| [e6c354c3](https://github.com/downtownallday/mailinabox-ldap/commit/e6c354c3125bdfdf32fecabb851288a385705e72) | 2025-01-06 | _Joshua Tauberer_ | v71a |
| [432b470d](https://github.com/downtownallday/mailinabox-ldap/commit/432b470d2931a15a3761a3e35f1c30cac4e83b49) | 2025-01-06 | _Paul_ | New & improved Disable MOTD advertisements (#2470) |

View File

@ -8,7 +8,6 @@
rewrite ^/admin/munin$ /admin/munin/ redirect;
location /admin/ {
proxy_pass http://127.0.0.1:10222/;
proxy_read_timeout 600s;
proxy_set_header X-Forwarded-For $remote_addr;
add_header X-Frame-Options "DENY";
add_header X-Content-Type-Options nosniff;

View File

@ -19,6 +19,7 @@
import os, os.path, re, datetime, sys
import dateutil.parser, dateutil.relativedelta, dateutil.tz
from datetime import date
import rtyaml
from exclusiveprocess import Lock
@ -167,6 +168,8 @@ def should_force_full(config, env):
# since the last full backup is greater than half the size
# of that full backup.
inc_size = 0
# Check if day of week is a weekend day
weekend = date.today().weekday()>=5
for bak in backup_status(env)["backups"]:
if not bak["full"]:
# Scan through the incremental backups cumulating
@ -175,12 +178,14 @@ def should_force_full(config, env):
else:
# ...until we reach the most recent full backup.
# Return if we should to a full backup, which is based
# on the size of the increments relative to the full
# backup, as well as the age of the full backup.
if inc_size > .5*bak["size"]:
return True
if dateutil.parser.parse(bak["date"]) + datetime.timedelta(days=config["min_age_in_days"]*10+1) < datetime.datetime.now(dateutil.tz.tzlocal()):
return True
# on whether it is a weekend day, the size of the
# increments relative to the full backup, as well as
# the age of the full backup.
if weekend:
if inc_size > .5*bak["size"]:
return True
if dateutil.parser.parse(bak["date"]) + datetime.timedelta(days=config["min_age_in_days"]*10+1) < datetime.datetime.now(dateutil.tz.tzlocal()):
return True
return False
else:
# If we got here there are no (full) backups, so make one.
@ -348,6 +353,7 @@ def perform_backup(full_backup):
"--verbosity", "warning", "--no-print-statistics",
"--archive-dir", backup_cache_dir,
"--exclude", backup_root,
"--exclude", os.path.join(env["STORAGE_ROOT"], "owncloud-backup"),
"--volsize", "250",
"--gpg-options", "'--cipher-algo=AES256'",
"--allow-source-mismatch",
@ -429,6 +435,7 @@ def run_duplicity_verification():
"--compare-data",
"--archive-dir", backup_cache_dir,
"--exclude", backup_root,
"--exclude", os.path.join(env["STORAGE_ROOT"], "owncloud-backup"),
*get_duplicity_additional_args(env),
get_duplicity_target_url(config),
env["STORAGE_ROOT"],

View File

@ -23,7 +23,7 @@ def get_ssl_certificates(env):
# that the certificates are good for to the best certificate for
# the domain.
from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey
from cryptography.hazmat.primitives.asymmetric import dsa, rsa, ec
from cryptography.x509 import Certificate
# The certificates are all stored here:
@ -44,6 +44,12 @@ def get_ssl_certificates(env):
# the cert that it should be a
# symlink to.
continue
if fn in ['ca','ca_certificate.pem','ca_private_key.pem']:
# Ignore as these are for generating a temporary
# "self-signed" certificate in a virgin setup (before
# Let's Encrypt gives us a certificate).
#
continue
fn = os.path.join(ssl_root, fn)
if os.path.isfile(fn):
yield fn
@ -68,13 +74,15 @@ def get_ssl_certificates(env):
# Not a valid PEM format for a PEM type we care about.
continue
# Is it a private key?
if isinstance(pem, RSAPrivateKey):
private_keys[pem.public_key().public_numbers()] = { "filename": fn, "key": pem }
# Is it a certificate?
if isinstance(pem, Certificate):
certificates.append({ "filename": fn, "cert": pem })
# It is a private key
elif (isinstance(pem, rsa.RSAPrivateKey)
or isinstance(pem, dsa.DSAPrivateKey)
or isinstance(pem, ec.EllipticCurvePrivateKey)):
private_keys[pem.public_key().public_numbers()] = { "filename": fn, "key": pem }
# Process the certificates.
domains = { }
@ -100,7 +108,7 @@ def get_ssl_certificates(env):
# Sort the certificates to prefer good ones.
import datetime
now = datetime.datetime.utcnow()
now = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None)
ret = { }
for domain, cert_list in domains.items():
#for c in cert_list: print(domain, c["cert"].not_valid_before, c["cert"].not_valid_after, "("+str(now)+")", c["cert"].issuer, c["cert"].subject, c._filename if hasattr(c,"_filename") else "")
@ -518,7 +526,7 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
# Check that the ssl_certificate & ssl_private_key files are good
# for the provided domain.
from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey
from cryptography.hazmat.primitives.asymmetric import rsa, dsa, ec
from cryptography.x509 import Certificate
# The ssl_certificate file may contain a chain of certificates. We'll
@ -552,7 +560,9 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
except ValueError as e:
return (f"The private key file {ssl_private_key} is not a private key file: {e!s}", None)
if not isinstance(priv_key, RSAPrivateKey):
if (not isinstance(priv_key, rsa.RSAPrivateKey)
and not isinstance(priv_key, dsa.DSAPrivateKey)
and not isinstance(priv_key, ec.EllipticCurvePrivateKey)):
return ("The private key file %s is not a private key file." % ssl_private_key, None)
if priv_key.public_key().public_numbers() != cert.public_key().public_numbers():
@ -579,7 +589,7 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
# Check that the certificate hasn't expired. The datetimes returned by the
# certificate are 'naive' and in UTC. We need to get the current time in UTC.
import datetime
now = datetime.datetime.utcnow()
now = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None)
if not(cert.not_valid_before <= now <= cert.not_valid_after):
return (f"The certificate has expired or is not yet valid. It is valid from {cert.not_valid_before} to {cert.not_valid_after}.", None)
@ -655,7 +665,7 @@ def load_pem(pem):
msg = "File is not a valid PEM-formatted file."
raise ValueError(msg)
pem_type = pem_type.group(1)
if pem_type in {b"RSA PRIVATE KEY", b"PRIVATE KEY"}:
if pem_type.endswith(b"PRIVATE KEY"):
return serialization.load_pem_private_key(pem, password=None, backend=default_backend())
if pem_type == b"CERTIFICATE":
return load_pem_x509_certificate(pem, default_backend())

View File

@ -293,26 +293,45 @@ def run_network_checks(env, output):
# The user might have ended up on an IP address that was previously in use
# by a spammer, or the user may be deploying on a residential network. We
# will not be able to reliably send mail in these cases.
# See https://www.spamhaus.org/news/article/807/using-our-public-mirrors-check-your-return-codes-now. for
# information on spamhaus return codes
rev_ip4 = ".".join(reversed(env['PUBLIC_IP'].split('.')))
zen = query_dns(rev_ip4+'.zen.spamhaus.org', 'A', nxdomain=None)
evaluate_spamhaus_lookup(env['PUBLIC_IP'], 'IPv4', rev_ip4, output, zen)
if not env['PUBLIC_IPV6']:
return
from ipaddress import IPv6Address
rev_ip6 = ".".join(reversed(IPv6Address(env['PUBLIC_IPV6']).exploded.split(':')))
zen = query_dns(rev_ip6+'.zen.spamhaus.org', 'A', nxdomain=None)
evaluate_spamhaus_lookup(env['PUBLIC_IPV6'], 'IPv6', rev_ip6, output, zen)
def evaluate_spamhaus_lookup(lookupaddress, lookuptype, lookupdomain, output, zen):
# See https://www.spamhaus.org/news/article/807/using-our-public-mirrors-check-your-return-codes-now. for
# information on spamhaus return codes
if zen is None:
output.print_ok("IP address is not blacklisted by zen.spamhaus.org.")
output.print_ok(f"{lookuptype} address is not blacklisted by zen.spamhaus.org.")
elif zen == "[timeout]":
output.print_warning("Connection to zen.spamhaus.org timed out. Could not determine whether this box's IP address is blacklisted. Please try again later.")
output.print_warning(f"""Connection to zen.spamhaus.org timed out. Could not determine whether this box's
{lookuptype} address is blacklisted. Please try again later.""")
elif zen == "[Not Set]":
output.print_warning("Could not connect to zen.spamhaus.org. Could not determine whether this box's IP address is blacklisted. Please try again later.")
output.print_warning(f"""Could not connect to zen.spamhaus.org. Could not determine whether this box's
{lookuptype} address is blacklisted. Please try again later.""")
elif zen == "127.255.255.252":
output.print_warning("Incorrect spamhaus query: %s. Could not determine whether this box's IP address is blacklisted." % (rev_ip4+'.zen.spamhaus.org'))
output.print_warning(f"""Incorrect spamhaus query: {lookupdomain + '.zen.spamhaus.org'}. Could not determine whether
this box's {lookuptype} address is blacklisted.""")
elif zen == "127.255.255.254":
output.print_warning("Mail-in-a-Box is configured to use a public DNS server. This is not supported by spamhaus. Could not determine whether this box's IP address is blacklisted.")
output.print_warning(f"""Mail-in-a-Box is configured to use a public DNS server. This is not supported by
spamhaus. Could not determine whether this box's {lookuptype} address is blacklisted.""")
elif zen == "127.255.255.255":
output.print_warning("Too many queries have been performed on the spamhaus server. Could not determine whether this box's IP address is blacklisted.")
output.print_warning(f"""Too many queries have been performed on the spamhaus server. Could not determine
whether this box's {lookuptype} address is blacklisted.""")
else:
output.print_error("""The IP address of this machine {} is listed in the Spamhaus Block List (code {}),
which may prevent recipients from receiving your email. See http://www.spamhaus.org/query/ip/{}.""".format(env['PUBLIC_IP'], zen, env['PUBLIC_IP']))
output.print_error(f"""The {lookuptype} address of this machine {lookupaddress} is listed in the Spamhaus Block
List (code {zen}), which may prevent recipients from receiving your email. See
http://www.spamhaus.org/query/ip/{lookupaddress}.""")
def run_domain_checks(rounded_time, env, output, pool, domains_to_check=None):
# Get the list of domains we handle mail for.
@ -532,6 +551,8 @@ def check_dns_zone(domain, env, output, dns_zonefiles):
# Check that each custom secondary nameserver resolves the IP address.
if custom_secondary_ns and not probably_external_dns:
SOARecord = query_dns(domain, "SOA", at=env['PUBLIC_IP'])# Explicitly ask the local dns server.
for ns in custom_secondary_ns:
# We must first resolve the nameserver to an IP address so we can query it.
ns_ips = query_dns(ns, "A")
@ -541,15 +562,36 @@ def check_dns_zone(domain, env, output, dns_zonefiles):
# Choose the first IP if nameserver returns multiple
ns_ip = ns_ips.split('; ')[0]
checkSOA = True
# Now query it to see what it says about this domain.
ip = query_dns(domain, "A", at=ns_ip, nxdomain=None)
if ip == correct_ip:
output.print_ok("Secondary nameserver %s resolved the domain correctly." % ns)
output.print_ok(f"Secondary nameserver {ns} resolved the domain correctly.")
elif ip is None:
output.print_error("Secondary nameserver %s is not configured to resolve this domain." % ns)
output.print_error(f"Secondary nameserver {ns} is not configured to resolve this domain.")
# No need to check SOA record if not configured as nameserver
checkSOA = False
elif ip == '[timeout]':
output.print_error(f"Secondary nameserver {ns} did not resolve this domain, result: {ip}")
checkSOA = False
else:
output.print_error(f"Secondary nameserver {ns} is not configured correctly. (It resolved this domain as {ip}. It should be {correct_ip}.)")
if checkSOA:
# Check that secondary DNS server is synchronized with our primary DNS server. Simplified by checking the SOA record which has a version number
SOASecondary = query_dns(domain, "SOA", at=ns_ip)
if SOARecord == SOASecondary:
output.print_ok(f"Secondary nameserver {ns} has consistent SOA record.")
elif SOARecord == '[Not Set]':
output.print_error(f"Secondary nameserver {ns} has no SOA record configured.")
elif SOARecord == '[timeout]':
output.print_error(f"Secondary nameserver {ns} timed out on checking SOA record.")
else:
output.print_error(f"""Secondary nameserver {ns} has inconsistent SOA record (primary: {SOARecord} versus secondary: {SOASecondary}).
Check that synchronization between secondary and primary DNS servers is properly set-up.""")
def check_dns_zone_suggestions(domain, env, output, dns_zonefiles, domains_with_a_records):
# Warn if a custom DNS record is preventing this or the automatic www redirect from
# being served.

View File

@ -215,6 +215,7 @@ def get_ssh_port():
def get_ssh_config_value(parameter_name):
# Returns ssh configuration value for the provided parameter
import subprocess
try:
output = shell('check_output', ['sshd', '-T'])
except FileNotFoundError:

View File

@ -51,7 +51,7 @@ if [ -z "$TAG" ]; then
if [ "$UBUNTU_VERSION" == "Ubuntu 22.04 LTS" ]; then
# This machine is running Ubuntu 22.04, which is supported by
# Mail-in-a-Box versions 60 and later.
TAG=v70
TAG=v71a
elif [ "$UBUNTU_VERSION" == "Ubuntu 18.04 LTS" ]; then
# This machine is running Ubuntu 18.04, which is supported by
# Mail-in-a-Box versions 0.40 through 5x.

View File

@ -115,14 +115,14 @@ tools/editconf.py /etc/dovecot/conf.d/10-auth.conf \
# Enable SSL, specify the location of the SSL certificate and private key files.
# Use Mozilla's "Intermediate" recommendations at https://ssl-config.mozilla.org/#server=dovecot&server-version=2.2.33&config=intermediate&openssl-version=1.1.1,
# except that the current version of Dovecot does not have a TLSv1.3 setting, so we only use TLSv1.2.
tools/editconf.py /etc/dovecot/conf.d/10-ssl.conf \
tools/editconf.py /etc/dovecot/conf.d/10-ssl.conf -E \
ssl=required \
"ssl_cert=<$STORAGE_ROOT/ssl/ssl_certificate.pem" \
"ssl_key=<$STORAGE_ROOT/ssl/ssl_private_key.pem" \
"ssl_min_protocol=TLSv1.2" \
"ssl_cipher_list=ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" \
"ssl_prefer_server_ciphers=no" \
"ssl_dh_parameters_length=2048" \
"ssl_dh_parameters_length=" \
"ssl_dh=<$STORAGE_ROOT/ssl/dh2048.pem"
# Disable in-the-clear IMAP/POP because there is no reason for a user to transmit

View File

@ -126,7 +126,7 @@ minute=$((RANDOM % 60)) # avoid overloading mailinabox.email
cat > /etc/cron.d/mailinabox-nightly << EOF;
# Mail-in-a-Box --- Do not edit / will be overwritten on update.
# Run nightly tasks: backup, status checks.
$minute 3 * * * root (cd $PWD && management/daily_tasks.sh)
$minute 1 * * * root (cd $PWD && management/daily_tasks.sh)
EOF
# Start the management server.

View File

@ -219,3 +219,12 @@ fi
if [ ! -f "$STORAGE_ROOT/ssl/dh2048.pem" ]; then
openssl dhparam -out "$STORAGE_ROOT/ssl/dh2048.pem" 2048
fi
# Cleanup expired SSL certificates from $STORAGE_ROOT/ssl daily
cat > /etc/cron.daily/mailinabox-ssl-cleanup << EOF;
#!/bin/bash
# Mail-in-a-Box
# Cleanup expired SSL certificates
$(pwd)/tools/ssl_cleanup
EOF
chmod +x /etc/cron.daily/mailinabox-ssl-cleanup

View File

@ -92,6 +92,15 @@ fi
# (See https://discourse.mailinabox.email/t/journalctl-reclaim-space-on-small-mailinabox/6728/11.)
tools/editconf.py /etc/systemd/journald.conf MaxRetentionSec=10day
# ### Improve server privacy
# Disable MOTD adverts to prevent revealing server information in MOTD request headers
# See https://ma.ttias.be/what-exactly-being-sent-ubuntu-motd/
if [ -f /etc/default/motd-news ]; then
tools/editconf.py /etc/default/motd-news ENABLED=0
rm -f /var/cache/motd-news
fi
# ### Add PPAs.
# We install some non-standard Ubuntu packages maintained by other

View File

@ -34,7 +34,7 @@ echo "Installing Roundcube (webmail)..."
apt_install \
dbconfig-common \
php"${PHP_VER}"-cli php"${PHP_VER}"-sqlite3 php"${PHP_VER}"-intl php"${PHP_VER}"-common php"${PHP_VER}"-curl php"${PHP_VER}"-imap \
php"${PHP_VER}"-gd php"${PHP_VER}"-pspell php"${PHP_VER}"-mbstring libjs-jquery libjs-jquery-mousewheel libmagic1 \
php"${PHP_VER}"-gd php"${PHP_VER}"-pspell php"${PHP_VER}"-mbstring php"${PHP_VER}"-xml libjs-jquery libjs-jquery-mousewheel libmagic1 \
sqlite3
apt_install php"${PHP_VER}"-ldap
@ -49,8 +49,8 @@ apt_install php"${PHP_VER}"-ldap
# https://github.com/mstilkerich/rcmcarddav/releases
# The easiest way to get the package hashes is to run this script and get the hash from
# the error message.
VERSION=1.6.8
HASH=00586f5163b3f6c1b0798be745982e3547b1b24a
VERSION=1.6.10
HASH=0cfbb457e230793df8c56c2e6d3655cf3818f168
PERSISTENT_LOGIN_VERSION=version-5.3.0
HTML5_NOTIFIER_VERSION=68d9ca194212e15b3c7225eb6085dbcf02fd13d7 # version 0.6.4+
CARDDAV_VERSION=4.4.3

View File

@ -31,8 +31,8 @@ apt_install \
phpenmod -v "$PHP_VER" imap
# Copy Z-Push into place.
VERSION=2.7.3
TARGETHASH=9d4bec41935e9a4e07880c5ff915bcddbda4443b
VERSION=2.7.5
TARGETHASH=f0b0b06e255f3496173ab9d28a4f2d985184720e
needs_update=0 #NODOC
if [ ! -f /usr/local/lib/z-push/version ]; then
needs_update=1 #NODOC
@ -120,4 +120,6 @@ restart_service php"$PHP_VER"-fpm
# Fix states after upgrade
hide_output php"$PHP_VER" /usr/local/lib/z-push/z-push-admin.php -a fixstates
if [ $needs_update == 1 ]; then
hide_output php"$PHP_VER" /usr/local/lib/z-push/z-push-admin.php -a fixstates
fi

View File

@ -201,7 +201,25 @@ lx_wait_for_boot() {
echo ""
echo -n "Wait for cloud-init "
lxc --project "$project" exec "$inst" -- cloud-init status --wait
local rc=$?
if [ $rc -eq 0 ]; then
echo "Wait for ip address "
local ip=""
local count=0
while [ $count -lt 10 ]; do
let count+=1
ip="$(lxc --project "$project" exec "$inst" -- hostname -I | awk '{print $1}')"
rc=$?
echo " [${count}] got: $ip"
if [ $rc -ne 0 -o "$ip" != "" ];then
break
fi
sleep 5
done
fi
echo ""
return $rc
}
lx_get_ssh_identity() {

View File

@ -67,7 +67,7 @@ provision_shell() {
else
local tmp=$(mktemp)
echo "#!/bin/sh" >"$tmp"
echo "#!/bin/bash" >"$tmp"
cat >>"$tmp"
lxc --project "$project" file push "$tmp" "${inst}${remote_path}" $lxc_flags || return 1
rm -f "$tmp"

View File

@ -37,6 +37,12 @@
D=$(dirname "$BASH_SOURCE")
. "$D/lx_functions.sh" || exit 1
show_cl="yes"
if [ "$1" = "-q" ]; then
show_cl="no"
shift
fi
vlx_guess() {
if [ $# -eq 2 ]; then
LX_PROJECT="$1"
@ -83,7 +89,6 @@ vlx_exec() {
if [ "${args[$idx]}" = "--" ]; then
if [ $idx -eq 3 ]; then
# format 1 with cwd
echo "f1"
wd="$3"
vlx_guess "$1" "$2" || return 1
shift; shift; shift; shift;
@ -92,12 +97,10 @@ vlx_exec() {
if [ "${2#/}" != "$2" ]; then
# wd starts with /, so it's a path
# format 2
echo "f2"
wd="$2"
vlx_guess "" "$1" || return 1
else
# format 1 w/o cwd
echo "f1 w/o cwd"
vlx_guess "$1" "$2" || return 1
fi
shift; shift; shift;
@ -106,12 +109,10 @@ vlx_exec() {
if [ "${1#/}" != "$1" ]; then
# wd starts with /, so it's a path
# format 3
echo "f3"
wd="$1"
vlx_guess || return 1
else
# format 2 w/o cwd
echo "f2 w/o cwd"
vlx_guess "$1" || return 1
fi
shift; shift;
@ -121,7 +122,6 @@ vlx_exec() {
fi
else
# format 4
echo "f4"
vlx_guess || return 1
fi
@ -129,28 +129,43 @@ vlx_exec() {
if [ ! -z "$wd" ]; then
xargs="--cwd $wd"
fi
echo lxc --project "$LX_PROJECT" exec "$LX_INST" $xargs -- "$@"
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" exec "$LX_INST" $xargs -- "$@"
lxc --project "$LX_PROJECT" exec "$LX_INST" $xargs -- "$@"
}
vlx_shell() {
vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" exec "$LX_INST" -- bash
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" exec "$LX_INST" -- bash
lxc --project "$LX_PROJECT" exec "$LX_INST" -- bash
}
vlx_hostname() {
vlx_guess "$@" || return 1
local host
lxc --project "$LX_PROJECT" exec "$LX_INST" -- /usr/bin/hostname --fqdn || return 1
}
vlx_ipaddr() {
vlx_guess "$@" || return 1
local hostip
hostip="$(lxc --project "$LX_PROJECT" exec "$LX_INST" -- /usr/bin/hostname -I)"
[ $? -ne 0 -o -z "$hostip" ] && return 1
awk '{print $1}' <<<"$hostip"
}
vlx_ssh() {
local host="$1"
if [ "$host" = "--" ]; then
host=""
else
shift
fi
if [ -z "$host" ]; then
host="$(vlx_hostname)"
host="$(vlx_ipaddr)"
if [ $? -ne 0 ]; then
echo "Could not determine hostname, please specify"
echo "Could not determine ip address, please specify"
host=""
fi
if [ -z "$host" ]; then
@ -161,20 +176,22 @@ vlx_ssh() {
local id="$(lx_get_ssh_identity)"
local known_hosts="$(lx_get_ssh_known_hosts)"
local vmuser="vmuser"
#echo ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host"
#echo ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host" "$@"
echo "Connecting to $vmuser@$host ..."
ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host"
ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host" "$@"
}
vlx_list() {
vlx_guess "$1" || return 1
echo lxc --project "$LX_PROJECT" list
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" list
lxc --project "$LX_PROJECT" list
}
vlx_images() {
vlx_guess "$1" || return 1
echo lxc --project "$LX_PROJECT" image list
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" image list
lxc --project "$LX_PROJECT" image list
}
@ -191,19 +208,22 @@ vlx_up() {
vlx_start() {
vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" start "$LX_INST"
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" start "$LX_INST"
lxc --project "$LX_PROJECT" start "$LX_INST"
}
vlx_stop() {
vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" stop "$LX_INST"
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" stop "$LX_INST"
lxc --project "$LX_PROJECT" stop "$LX_INST"
}
vlx_delete() {
vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" delete --force --interactive "$LX_INST"
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" delete --force --interactive "$LX_INST"
lxc --project "$LX_PROJECT" delete --force --interactive "$LX_INST"
}
@ -224,7 +244,8 @@ vlx_status() {
vlx_restart() {
vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" restart "$LX_INST"
[ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" restart "$LX_INST"
lxc --project "$LX_PROJECT" restart "$LX_INST"
}

View File

@ -26,10 +26,11 @@ class NcContactsAutomation(object):
els = d.find_els('div.contacts-list div.list-item-content,div.option__details')
d.say_verbose('found %s contacts' % len(els))
for el in els:
# .line-one (nc 25+)
# .list-item-content__name (nc 29+)
# .line-one (nc 25-28)
# .option__lineone (nc <25)
fullname = el.find_el('.line-one,.option__lineone').content().strip()
email = el.find_el('.line-two,.option__linetwo').content().strip()
fullname = el.find_el('.line-one,.option__lineone,.list-item-content__name').content().strip()
email = el.find_el('.line-two,.option__linetwo,.list-item-content__subname').content().strip()
d.say_verbose('contact: "%s" <%s>', fullname, email)
# NC 28: email not present in html
ignore_email = True if email == '' else False
@ -43,7 +44,7 @@ class NcContactsAutomation(object):
d = self.d
d.say("Wait for contact to load")
d.wait_for_el('section.contact-details', secs=secs)
def delete_current_contact(self):
d = self.d
d.say("Delete current contact")

View File

@ -58,7 +58,10 @@ class NextcloudAutomation(object):
d.say("Logout of Nextcloud")
self.click_avatar()
el = d.find_el('[data-id="logout"] a', throws=False) # nc < 26
el = d.find_el('a#logout', throws=False)
if not el:
# nc >= 29
el = d.find_el('[data-id="logout"] a', throws=False) # nc < 26
if not el:
# nc >= 26
el = d.find_el('#logout > a', throws=False)
@ -72,7 +75,10 @@ class NextcloudAutomation(object):
d = self.d
d.say("Open contacts")
# nc 25+
el = d.find_el('header [data-app-id="contacts"]', throws=False)
el = d.find_el('header [title="Contacts"]', throws=False)
if not el:
# nc < 29
el = d.find_el('header [data-app-id="contacts"]', throws=False)
if not el:
# nc < 25
el = d.find_el('header [data-id="contacts"]')

View File

@ -0,0 +1 @@
TODO

View File

@ -0,0 +1 @@
TODO - convert to lxd

View File

@ -17,7 +17,7 @@ D=$(dirname "$BASH_SOURCE")
provision_start "" "/mailinabox" || exit 1
# Setup system
if [ "$1" = "ciab" ]; then
if [ "$TESTS" = "ciab" -o "$1" = "ciab" ]; then
# use a remote cloudinabox (does not have to be running)
provision_shell <<<"
cd /mailinabox
@ -44,7 +44,7 @@ exit \$rc
"
provision_done $?
else
elif [ -z "$1" ]; then
# vanilla (default - no miab integration)
provision_shell <<<"
cd /mailinabox
@ -62,4 +62,7 @@ exit \$rc
"
provision_done $?
else
echo "Invalid argument: $1"
exit 1
fi

View File

@ -7,6 +7,7 @@
##### details.
#####
from browser.automation import (
TestDriver,
TimeoutException,
@ -50,16 +51,20 @@ try:
# open Contacts
#
d.start("Open contacts app")
contacts = nc.open_contacts()
try:
contacts = nc.open_contacts()
except NoSuchElementException:
nc.close_first_run_wizard()
contacts = nc.open_contacts()
nc.wait_for_app_load()
#
# handle selected operation
# handle selected operation
#
if op=='exists':
d.start("Check that contact %s exists", contact['email'])
contacts.click_contact(contact) # raises NoSuchElementException if not found
elif op=='delete':
d.start("Delete contact %s", contact['email'])
contacts.click_contact(contact)
@ -68,7 +73,7 @@ try:
elif op=='nop':
pass
else:
raise ValueError('Invalid operation: %s' % op)

View File

@ -172,7 +172,7 @@ test_web_config() {
record "output=$REST_OUTPUT"
if [ $code -eq 0 ]; then
test_failure "carddav url works, but expecting 401/NotAuthenticated from server"
elif [ $code -eq 1 -o $REST_HTTP_CODE -ne 401 ] || ! grep "NotAuthenticated" <<<"$REST_OUTPUT" >/dev/null; then
elif [ $REST_HTTP_CODE -ne 401 ]; then
test_failure "carddav url doesn't work: $REST_ERROR"
fi
fi
@ -189,7 +189,7 @@ test_web_config() {
record "output=$REST_OUTPUT"
if [ $code -eq 0 ]; then
test_failure "caldav url works, but expecting 401/NotAuthenticated from server"
elif [ $code -eq 1 -o $REST_HTTP_CODE -ne 401 ] || ! grep "NotAuthenticated" <<<"$REST_OUTPUT" >/dev/null; then
elif [ $REST_HTTP_CODE -ne 401 ]; then
test_failure "caldav url doesn't work: $REST_ERROR"
fi
fi

View File

@ -1,3 +0,0 @@
.vagrant
out
*-console.log

View File

@ -1,104 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
load './funcs.rb'
Vagrant.configure("2") do |config|
config.vm.synced_folder "../..", "/mailinabox", id: "mailinabox", automount: false
use_preloaded_box config, "ubuntu/jammy64"
# fresh install with encryption-at-rest
if ENV['tests']=='all'
config.vm.define "remote-nextcloud-docker-ehdd" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
export PRIMARY_HOSTNAME=qa1.abc.com
export FEATURE_MUNIN=false
export EHDD_KEYFILE=$HOME/keyfile
echo -n "boo" >$EHDD_KEYFILE
tests/system-setup/remote-nextcloud-docker.sh || exit 1
tests/runner.sh -no-smtp-remote remote-nextcloud ehdd default || exit 2
SH
end
end
# remote-nextcloud-docker w/basic data
config.vm.define "remote-nextcloud-docker" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
export PRIMARY_HOSTNAME=qa2.abc.com
export FEATURE_MUNIN=false
tests/system-setup/remote-nextcloud-docker.sh upgrade --populate=basic || exit 1
tests/runner.sh -no-smtp-remote remote-nextcloud upgrade-basic default || exit 2
SH
end
# upgrade-from-upstream
config.vm.define "upgrade-from-upstream" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
export PRIMARY_HOSTNAME=qa3.abc.com
# TODO: change UPSTREAM_TAG to 'main' once upstream is installable
export UPSTREAM_TAG=v67
tests/system-setup/upgrade-from-upstream.sh --populate=basic --populate=totpuser || exit 1
tests/runner.sh -no-smtp-remote upgrade-basic upgrade-totpuser default || exit 2
SH
end
# upgrade
# this test is only needed when testing migrations from miabldap
# to a newer miabldap with a migration step
#
# upgrade will handle testing upgrades of
# miabldap with or without a new migration step
config.vm.define "upgrade" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
# TODO: remove DEB_PYTHON_INSTALL_LAYOUT once MIABLDAP_RELEASE_TAG >= v66 (see https://github.com/downtownallday/mailinabox-ldap/commit/371f5bc1b236de40a1ed5d9118140ee13fddf5dc)
export DEB_PYTHON_INSTALL_LAYOUT='deb'
export PRIMARY_HOSTNAME=upgrade.abc.com
tests/system-setup/upgrade.sh --populate=basic --populate=totpuser || exit 1
tests/runner.sh -no-smtp-remote upgrade-basic upgrade-totpuser default || exit 2
SH
end
# unsetvars: because miab sets bash '-e' to fail any setup script
# when a script command returns a non-zero exit code, and more
# importantly '-u' which fails scripts when any unset variable is
# accessed, this definition sets a minimal number of environment
# variables prior to running start.sh. Doing so will test that no
# failures occur during setup in the most common use case because
# other vagrant definitions in this file load
# tests/system-setup/setup-default.sh, which pre-assign a value to
# most variables.
if ENV['tests']=='all' or ENV['tests']=='pre-commit'
config.vm.define "unsetvars" do |m1|
m1.vm.hostname = "mailinabox.lan"
m1.vm.network "private_network", ip: "192.168.56.4"
m1.vm.provision :shell, :inline => <<-SH
export NONINTERACTIVE=1
export PUBLIC_IP=auto
export PUBLIC_IPV6=auto
export PRIMARY_HOSTNAME=auto
export SKIP_NETWORK_CHECKS=1
cd /mailinabox
setup/start.sh
SH
end
end
end

View File

@ -1,25 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
def use_preloaded_box(obj, name, preloaded_dir=".")
obj.vm.box = String.new(name)
_name=name.sub! '/','-' # ubuntu/bionic64 => ubuntu-bionic64
if File.file?("#{preloaded_dir}/preloaded/preloaded-#{_name}.box")
# box name needs to be unique on the system
obj.vm.box = "preloaded-miabldap-#{_name}"
obj.vm.box_url = "file://" + Dir.pwd + "/#{preloaded_dir}/preloaded/preloaded-#{_name}.box"
if Vagrant.has_plugin?('vagrant-vbguest')
# do not update additions when booting this machine
obj.vbguest.auto_update = false
end
end
end
# Grab the name of the default interface
$default_network_interface = `ip route | awk '/^default/ {printf "%s", $5; exit 0}'`

View File

@ -1,86 +0,0 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# Parallel provisioning for virtualbox because "The Vagrant VirtualBox
# provider does not support parallel execution at this time"
# (https://www.vagrantup.com/docs/providers/virtualbox/usage.html)
#
# Credit to:
# https://dzone.com/articles/parallel-provisioning-speeding
#
. "$(dirname "$0")/../lib/color-output.sh"
. "$(dirname "$0")/../lib/misc.sh"
if [ -z "$tests" ]; then
export tests="pre-commit"
fi
OUTPUT_DIR=out
#rm -rf "$OUTPUT_DIR"
mkdir -p "$OUTPUT_DIR"
# set total parallel vms to (#cores minus 1)
MAX_PROCS=$(cat /proc/cpuinfo | grep processor | wc -l)
let MAX_PROCS-=1
parallel_provision() {
while read box; do
outfile="$OUTPUT_DIR/$box.out.txt"
rm -f "$outfile"
echo "Provisioning '$box'. Output will be in: $outfile" 1>&2
echo $box
done | xargs -P $MAX_PROCS -I"BOXNAME" \
sh -c 'vagrant provision BOXNAME >'"$OUTPUT_DIR/"'BOXNAME.out.txt 2>&1 && echo "EXITCODE: 0" >> '"$OUTPUT_DIR/"'BOXNAME.out.txt || echo "EXITCODE: $?" >>'"$OUTPUT_DIR/"'BOXNAME.out.txt'
}
## -- main -- ##
start_time="$(date +%s)"
# start boxes sequentially to avoid vbox explosions
vagrant up --no-provision
# but run provision tasks in parallel
boxes="$(vagrant status | awk '/running \(/ {print $1}')"
echo "$boxes" | parallel_provision
# output overall result - Vagrantfile script must output "EXITCODE: <num>"
H1 "Results"
rc=0
for box in $boxes; do
file="$OUTPUT_DIR"/$box.out.txt
exitcode="$(tail "$file" | grep EXITCODE: | awk '{print $NF}')"
echo -n "$box: "
if [ -z "$exitcode" ]; then
danger "NO EXITCODE!"
[ $rc -eq 0 ] && rc=2
elif [ "$exitcode" == "0" ]; then
success "SUCCESS"
else
danger "FAILURE ($exitcode)"
rc=1
fi
done
# output elapsed time
end_time="$(date +%s)"
echo ""
echo "Elapsed time: $(elapsed_pretty $start_time $end_time)"
# exit
echo ""
echo "Guest VMs are running! Destroy them with 'vagrant destroy -f'"
exit $rc

View File

@ -1,2 +0,0 @@
*.box
src/

View File

@ -1,58 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
def checkout_tag_and_mount(obj, tag)
if "#{tag}" == ""
obj.vm.synced_folder "../../..", "/mailinabox", id: "mailinabox", automount: false
else
_srcdir="src/maibldap-#{tag}"
if not Dir.exist?(_srcdir)
puts "Cloning tag #{tag} to #{_srcdir}"
if tag.size==40 and tag.match?(/\A[0-9a-fA-F]+\Z/)
system("git clone #{ENV['MIABLDAP_GIT']} #{_srcdir}")
system("cd #{_srcdir}; git reset --hard #{tag}")
else
system("git clone -b #{tag} --depth 1 #{ENV['MIABLDAP_GIT']} #{_srcdir}")
end
end
obj.vm.synced_folder _srcdir, "/mailinabox", id: "mailinabox", automount: false
end
end
Vagrant.configure("2") do |config|
checkout_tag_and_mount config, ENV['RELEASE_TAG']
config.vm.define "preloaded-ubuntu-bionic64" do |m1|
m1.vm.box = "ubuntu/bionic64"
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
tests/vagrant/preloaded/prepvm.sh --no-dry-run
rc=$?
echo "$rc" > "/vagrant/prepcode.txt"
[ $rc -gt 0 ] && exit 1
exit 0
SH
end
config.vm.define "preloaded-ubuntu-jammy64" do |m1|
m1.vm.box = "ubuntu/jammy64"
m1.vm.boot_timeout = 30
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
tests/vagrant/preloaded/prepvm.sh --no-dry-run
rc=$?
echo "$rc" > "/vagrant/prepcode.txt"
[ $rc -gt 0 ] && exit 1
exit 0
SH
end
end

View File

@ -1,134 +0,0 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# load defaults for MIABLDAP_GIT and MIABLDAP_FINAL_RELEASE_TAG_BIONIC64 (make available to Vagrantfile)
pushd "../../.." >/dev/null
source tests/lib/color-output.sh
source tests/system-setup/setup-defaults.sh || exit 1
popd >/dev/null
H1 "Destroy any running boxes"
vagrant destroy -f
rm -f prepcode.txt
H1 "Ensure plugins are installed"
for plugin in "vagrant-vbguest" "vagrant-reload"
do
if ! vagrant plugin list | grep -F "$plugin" >/dev/null; then
vagrant plugin install "$plugin" || exit 1
fi
done
H1 "Upgrade base boxes"
vagrant box update
boxes=(
"preloaded-ubuntu-jammy64"
"preloaded-ubuntu-bionic64"
)
# preload packages from source of the following git tags. empty string
# means use the current source tree
tags=(
""
"$MIABLDAP_FINAL_RELEASE_TAG_BIONIC64"
)
try_reboot=(
true
false
)
idx=0
for box in "${boxes[@]}"
do
if [ -z "$1" ]; then
# no cli arguments - only process first box
[ $idx -ge 1 ] && break
else
# cli argument specifies "all" or a named box
if [ "$1" != "all" -a "$1" != "$box" ]; then
let idx+=1
continue
fi
fi
H1 "Provision: $box"
export RELEASE_TAG="${tags[$idx]}"
vagrant up $box | tee /tmp/$box.out
upcode=$?
if [ $upcode -eq 0 -a ! -e "./prepcode.txt" ] && ${try_reboot[$idx]} && grep -F 'Authentication failure' /tmp/$box.out >/dev/null; then
# note: upcode is 0 only if config.vm.boot_timeout is set.
# If this works it may be an indication that ruby's internal
# ssh does not support the algorithm required by the server,
# or the public key does not match (vagrant and vm out of
# sync)
echo ""
echo "VAGRANT AUTHENTICATION FAILURE - TRYING LOOSER ALLOWED SSHD ALGS"
if vagrant ssh $box -c "sudo bash -c 'echo PubkeyAcceptedAlgorithms +ssh-rsa > /etc/ssh/sshd_config.d/miabldap.conf; sudo systemctl restart sshd'"; then
vagrant halt $box
vagrant up $box
upcode=$?
fi
fi
if [ $upcode -ne 0 -a ! -e "./prepcode.txt" ] && ${try_reboot[$idx]}
then
# a reboot may be necessary if guest addtions was newly
# compiled by vagrant plugin "vagrant-vbguest"
echo ""
echo "VAGRANT UP RETURNED $upcode -- RETRYING AFTER REBOOT"
vagrant halt $box
vagrant up $box
upcode=$?
fi
rm -f /tmp/$box.out
let idx+=1
prepcode=$(cat "./prepcode.txt")
rm -f prepcode.txt
echo ""
echo "VAGRANT UP RETURNED $upcode"
echo "PREPVM RETURNED $prepcode"
if [ "$prepcode" != "0" -o $upcode -ne 0 ]; then
echo "FAILED!!!!!!!!"
vagrant destroy -f $box
exit 1
fi
if vagrant ssh $box -- cat /var/run/reboot-required >/dev/null 2>&1; then
echo "REBOOT REQUIRED"
vagrant reload $box
else
echo "REBOOT NOT REQUIRED"
fi
vagrant halt $box
vagrant package $box
rm -f $box.box
mv package.box $box.box
vagrant destroy -f $box
cached_name="$(sed 's/preloaded-/preloaded-miabldap-/' <<<"$box")"
echo "Removing cached box $cached_name"
if [ -e "../funcs.rb" ]; then
pushd .. > /dev/null
vagrant box remove $cached_name
code=$?
popd > /dev/null
else
vagrant box remove $cached_name
code=$?
fi
echo "Remove cache box result: $code - ignoring"
done

View File

@ -1,230 +0,0 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# Run this on a VM to pre-install all the packages, then
# take a snapshot - it will greatly speed up subsequent
# test installs
#
# What won't be installed:
#
# Nextcloud and Roundcube are downloaded with wget by the setup
# scripts, so they are not included
#
# slapd - we want to test installation with setup/ldap.sh
#
if [ ! -d "setup" ]; then
echo "Run from the miab root directory"
exit 1
fi
source tests/lib/misc.sh
source tests/lib/system.sh
source tests/lib/color-output.sh
dry_run=true
start=$(date +%s)
if [ "$1" == "--no-dry-run" ]; then
dry_run=false
fi
if $dry_run; then
echo "WARNING: dry run is TRUE, no changes will be made"
fi
# prevent apt from running needrestart(1)
export NEEDRESTART_SUSPEND=true
# prevent interaction during package install
export DEBIAN_FRONTEND=noninteractive
# what major version of ubuntu are we installing on?
OS_MAJOR=$(. /etc/os-release; echo $VERSION_ID | awk -F. '{print $1}')
remove_line_continuation() {
local file="$1"
awk '
BEGIN { C=0 }
C==1 && /[^\\]$/ { C=0; print $0; next }
C==1 { printf("%s",substr($0,0,length($0)-1)); next }
/\\$/ { C=1; printf("%s",substr($0,0,length($0)-1)); next }
{ print $0 }' \
"$file"
}
install_packages() {
local return_code=0
while read line; do
pkgs=""
case "$line" in
apt_install* )
pkgs="$(cut -c12- <<<"$line")"
;;
"apt-get install"* )
pkgs="$(cut -c16- <<<"$line")"
;;
"apt install"* )
pkgs="$(cut -c12- <<<"$line")"
;;
esac
# don't install slapd
pkgs="$(sed 's/slapd//g' <<< "$pkgs")"
# manually set PHP_VER if necessary
if grep "PHP_VER" <<<"$pkgs" >/dev/null; then
pkgs="$(sed "s/\"\?\${*PHP_VER}*\"\?/$PHP_VER/g" <<< "$pkgs")"
fi
if [ ! -z "$pkgs" ]; then
H2 "install: $pkgs"
if ! $dry_run; then
exec_no_output apt-get install -y $pkgs
let return_code+=$?
fi
fi
done
return $return_code
}
install_ppas() {
H1 "Add apt repositories"
grep 'hide_output add-apt-repository' setup/system.sh |
while read line; do
line=$(sed 's/^hide_output //' <<< "$line")
H2 "$line"
if ! $dry_run; then
exec_no_output $line
fi
done
}
add_swap() {
H1 "Add a swap file to the system"
if ! $dry_run; then
dd if=/dev/zero of=/swapfile bs=1024 count=$[1024*1024] status=none
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo "/swapfile none swap sw 0 0" >> /etc/fstab
fi
}
# install PPAs from sources
install_ppas
# add swap file
add_swap
# obtain PHP_VER variable from sources
PHP_VER=$(source setup/functions.sh; echo $PHP_VER)
if ! $dry_run; then
H1 "Upgrade system"
H2 "apt update"
exec_no_output apt-get update -y || exit 1
H2 "apt upgrade"
exec_no_output apt-get upgrade -y --with-new-pkgs || exit 1
H2 "apt autoremove"
exec_no_output apt-get autoremove -y
fi
# without using the same installation order as setup/start.sh, we end
# up with the system's php getting installed in addition to the
# non-system php that may also installed by setup (don't know why,
# probably one of the packages has a dependency). create an ordered
# list of files to process so we get a similar system setup.
setup_files=( $(ls setup/*.sh) )
desired_order=(
setup/functions.sh
setup/preflight.sh
setup/questions.sh
setup/network-checks.sh
setup/system.sh
setup/ssl.sh
setup/dns.sh
setup/ldap.sh
setup/mail-postfix.sh
setup/mail-dovecot.sh
setup/mail-users.sh
setup/dkim.sh
setup/spamassassin.sh
setup/web.sh
setup/webmail.sh
setup/nextcloud.sh
setup/zpush.sh
setup/management.sh
setup/management-capture.sh
setup/munin.sh
setup/firstuser.sh
)
ordered_files=()
for file in "${desired_order[@]}" "${setup_files[@]}"; do
if [ -e "$file" ] && ! array_contains "$file" "${ordered_files[@]}"; then
ordered_files+=( "$file" )
fi
done
failed=0
for file in ${ordered_files[@]}; do
H1 "$file"
remove_line_continuation "$file" | install_packages
[ $? -ne 0 ] && let failed+=1
done
if ! $dry_run; then
# bonus
H1 "install extras"
H2 "openssh, emacs, ntpdate, net-tools, jq"
exec_no_output apt-get install -y openssh-server emacs-nox ntpdate net-tools jq || let failed+=1
# these are added by system-setup scripts and needed for test runner
H2 "python3-dnspython"
exec_no_output apt-get install -y python3-dnspython || let failed+=1
H2 "pyotp(pip)"
exec_no_output python3 -m pip install pyotp --quiet || let failed+=1
# ...and for browser-based tests
#H2 "x11" # needed for chromium w/head (not --headless)
#exec_no_output apt-get install -y xorg openbox xvfb gtk2-engines-pixbuf dbus-x11 xfonts-base xfonts-100dpi xfonts-75dpi xfonts-cyrillic xfonts-scalable x11-apps imagemagick || let failed+=1
H2 "chromium"
#exec_no_output apt-get install -y chromium-browser || let failed+=1
exec_no_output snap install chromium || let failed+=1
H2 "selenium(pip)"
exec_no_output python3 -m pip install selenium --quiet || let failed+=1
# remove apache, which is what setup will do
H2 "remove apache2"
exec_no_output apt-get -y purge apache2 apache2-\*
fi
end=$(date +%s)
echo ""
echo ""
if [ $failed -gt 0 ]; then
echo "$failed failures! ($(elapsed_pretty $start $end))"
echo ""
exit 1
else
echo "Successfully prepped in $(elapsed_pretty $start $end). Take a snapshot...."
echo ""
exit 0
fi

View File

@ -1,2 +0,0 @@
.vagrant
*-console.log

View File

@ -1,87 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
load '../funcs.rb'
Vagrant.configure("2") do |config|
config.vm.synced_folder "../../..", "/mailinabox", id: "mailinabox", automount: false
config.vm.network "public_network", bridge: "#$default_network_interface"
use_preloaded_box config, "ubuntu/jammy64", ".."
if ENV['tests']=='ciab'
# vanilla connected to ciab (ciab does not need to be up)
config.vm.define "vanilla" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cat >/tmp/provision.sh <<EOF
#!/bin/bash
if [ \\$EUID -ne 0 ]; then
echo "Must be root"
exit 1
fi
cd /mailinabox
export PRIMARY_HOSTNAME=vanilla.local
export NC_PROTO=https
export NC_HOST=vanilla-ciab.local
export NC_PORT=443
export NC_PREFIX=/
export SKIP_SYSTEM_UPDATE=0
tests/system-setup/vanilla.sh --qa-ca --enable-mod=remote-nextcloud
if ! ufw status | grep remote_nextcloud >/dev/null; then
# firewall rules aren't added when ciab is down
echo "For testing, allow ldaps from anywhere"
ufw allow ldaps
fi
echo "Add smart host alias - so \\$NC_HOST can send mail to/via this host"
(
source tests/lib/all.sh
rest_urlencoded POST /admin/mail/aliases/add qa@abc.com Test_1234 "address=@\\$NC_HOST" "description=smart-host" "permitted_senders=qa@abc.com" 2>/dev/null
echo "\\$REST_HTTP_CODE: \\$REST_OUTPUT"
)
EOF
chmod +x /tmp/provision.sh
/tmp/provision.sh
SH
end # vanilla connected to ciab
else
# vanilla (default) install
config.vm.define "vanilla" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cat >/tmp/provision.sh <<EOF
#!/bin/bash
if [ \\$EUID -ne 0 ]; then
echo "Must be root"
exit 1
fi
start=\\$(date +%s)
cd /mailinabox
export PRIMARY_HOSTNAME=vanilla.local
#export FEATURE_MUNIN=false
#export FEATURE_NEXTCLOUD=false
export SKIP_SYSTEM_UPDATE=0
tests/system-setup/vanilla.sh
# --enable-mod=roundcube-master \
# --enable-mod=roundcube-debug \
# --enable-mod=rcmcarddav-composer
end=\\$(date +%s)
echo "Provisioning took \\$(source tests/lib/misc.sh; elapsed_pretty \\$start \\$end)"
EOF
chmod +x /tmp/provision.sh
/tmp/provision.sh
SH
end # vanilla (default)
end
end

27
tools/ssl_cleanup Executable file
View File

@ -0,0 +1,27 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# Cleanup SSL certificates which expired more than 7 days ago from $STORAGE_ROOT/ssl and move them to $STORAGE_ROOT/ssl.expired
source /etc/mailinabox.conf
shopt -s extglob nullglob
retain_after="$(date --date="7 days ago" +%Y%m%d)"
mkdir -p $STORAGE_ROOT/ssl.expired
ls $STORAGE_ROOT/ssl/*-+([0-9])-+([0-9a-f]).pem 2>/dev/null | while read file
do
pem="$(basename "$file")"
not_valid_after="$(cut -d- -f1 <<< "${pem: -21}")"
if [ "$not_valid_after" -lt "$retain_after" ]; then
mv "$file" "$STORAGE_ROOT/ssl.expired/${pem}"
fi
done