external/mailinabox
1
0
Fork 0
mirror of https://github.com/mail-in-a-box/mailinabox.git synced 2025-04-01 23:57:05 +00:00
Code Issues Releases Wiki Activity

Merge branch 'master' into quota

Browse Source
This commit is contained in:
downtownallday 2025-02-16 16:29:01 -05:00
commit 271a04333d
40 changed files with 323 additions and 846 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
.github/workflows/commit-tests.yml vendored
View File

@ -1,5 +1,10 @@
name: commit-tests name: commit-tests
on: [push, workflow_dispatch] on:
workflow_dispatch: { }
push:
branches: [ '**' ]
tags-ignore: [ 'v**' ]
paths-ignore: [ '**.md', 'api/mailinabox.yml' ]
jobs: jobs:
# MiaB-LDAP using encryption-at-rest and connected to a remote Nextcloud # MiaB-LDAP using encryption-at-rest and connected to a remote Nextcloud
remote-nextcloud-docker-ehdd: remote-nextcloud-docker-ehdd:

37
CHANGELOG.md
View File

@ -1,6 +1,37 @@
CHANGELOG CHANGELOG
========= =========
Version 71 (January 4, 2025)
----------------------------
(Version 71a was posted on January 6, 2025 and fixes a setup regression.)
Upgrades
* Roundcube upgraded to version 1.6.9.
* Z-Push upgraded to version 2.7.5.
Automated Maintenance
* Daily automated tasks are now run at 1am in the box's timezone and full backups are now restricted to running only on Saturdays and Sundays at that time.
* Backups now exclude the owncloud-backup folder so that we're not backing up backups.
* Old TLS certificates are now automatically deleted to improve control panel performance.
Setup
* Fixed broken setup if SSH was configured to listen on multiple ports.
* Ubuntu MOTD advertisements are now disabled.
* Fixed missing Roundcube dependency package if NextCloud isn't installed.
Control Panel
* Improved status checks for secondary nameservers.
* Spamhaus is now queried for the box's IPv6 address also.
* DSA and EC private keys are now accepted for TLS certificates.
* Timeouts for loading slow control panel pages are reduced.
And other minor fixes.
Version 70 (August 15, 2024) Version 70 (August 15, 2024)
---------------------------- ----------------------------
@ -73,7 +104,7 @@ Version 64 (September 2, 2023)
* Fixed backups to work with the latest duplicity package which was not backwards compatible. * Fixed backups to work with the latest duplicity package which was not backwards compatible.
* Fixed setting B2 as a backup target with a slash in the application key. * Fixed setting B2 as a backup target with a slash in the application key.
* Turned off OpenDMARC diagnostic reports sent in response to incoming mail. * Turned off OpenDMARC diagnostic reports sent in response to incoming mail.
* Fixed some crashes when using an unrelased version of Mail-in-a-Box. * Fixed some crashes when using an unreleased version of Mail-in-a-Box.
* Added z-push administration scripts. * Added z-push administration scripts.
Version 63 (July 27, 2023) Version 63 (July 27, 2023)
@ -1129,7 +1160,7 @@ Control panel:
System: System:
* The munin system monitoring tool is now installed and accessible at /admin/munin. * The munin system monitoring tool is now installed and accessible at /admin/munin.
* ownCloud updated to version 8.0.4. The ownCloud installation step now is reslient to download problems. The ownCloud configuration file is now stored in STORAGE_ROOT to fix loss of data when moving STORAGE_ROOT to a new machine. * ownCloud updated to version 8.0.4. The ownCloud installation step now is resilient to download problems. The ownCloud configuration file is now stored in STORAGE_ROOT to fix loss of data when moving STORAGE_ROOT to a new machine.
* The setup scripts now run `apt-get update` prior to installing anything to ensure the apt database is in sync with the packages actually available. * The setup scripts now run `apt-get update` prior to installing anything to ensure the apt database is in sync with the packages actually available.
@ -1167,7 +1198,7 @@ DNS:
* Internationalized Domain Names (IDNs) should now work in email. If you had custom DNS or custom web settings for internationalized domains, check that they are still working. * Internationalized Domain Names (IDNs) should now work in email. If you had custom DNS or custom web settings for internationalized domains, check that they are still working.
* It is now possible to set multiple TXT and other types of records on the same domain in the control panel. * It is now possible to set multiple TXT and other types of records on the same domain in the control panel.
* The custom DNS API was completely rewritten to support setting multiple records of the same type on a domain. Any existing client code using the DNS API will have to be rewritten. (Existing code will just get 404s back.) * The custom DNS API was completely rewritten to support setting multiple records of the same type on a domain. Any existing client code using the DNS API will have to be rewritten. (Existing code will just get 404s back.)
* On some systems the `nsd` service failed to start if network inferfaces were not ready. * On some systems the `nsd` service failed to start if network interfaces were not ready.
System / Control Panel: System / Control Panel:

37
Vagrantfile vendored
View File

@ -1,37 +0,0 @@
# -*- mode: ruby -*-
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/jammy64"
# Network config: Since it's a mail server, the machine must be connected
# to the public web. However, we currently don't want to expose SSH since
# the machine's box will let anyone log into it. So instead we'll put the
# machine on a private network.
config.vm.hostname = "mailinabox.lan"
config.vm.network "private_network", ip: "192.168.56.4"
config.vm.provision :shell, :inline => <<-SH
# Set environment variables so that the setup script does
# not ask any questions during provisioning. We'll let the
# machine figure out its own public IP.
export NONINTERACTIVE=1
export PUBLIC_IP=auto
export PUBLIC_IPV6=auto
export PRIMARY_HOSTNAME=auto
#export SKIP_NETWORK_CHECKS=1
# Start the setup script.
cd /vagrant
setup/start.sh
SH
end

42
changelog/v71.md Normal file
View File

@ -0,0 +1,42 @@
## Commits for v71
| COMMIT | DATE | AUTHOR | TITLE |
| ------ | ---- | ------ | ----- |
| [c5e33b51](https://github.com/downtownallday/mailinabox-ldap/commit/c5e33b51e5d112a09420d242c3d8cf1c23aeeafa) | 2025-01-04 | _downtownallday_ | Update license |
| [d58dd0c9](https://github.com/downtownallday/mailinabox-ldap/commit/d58dd0c91dd677acd6940d9b6099e2abb0ede729) | 2025-01-04 | _Joshua Tauberer_ | v71 |
| [f73da3db](https://github.com/downtownallday/mailinabox-ldap/commit/f73da3db60fc221fd2ecae17eac16db426800b2b) | 2025-01-04 | _Joshua Tauberer_ | Fix likely merge mistake in 564ed59bb47da24c9ebc50ae9137e6dcbcae9826 |
| [4c2e4bab](https://github.com/downtownallday/mailinabox-ldap/commit/4c2e4bab29a1030d87f5213b12dcfb38bcec2e83) | 2024-12-22 | _downtownallday_ | fix error when glob matches nothing (variable 'file' will have the glob as a value in the for loop and produce the error "mv: cannot stat '/home/user-data/ssl/*-+([0-9])-+([0-9a-f]).pem': No such file or directory") |
| [18721e42](https://github.com/downtownallday/mailinabox-ldap/commit/18721e42d19e87df5b7ba0182525739928dd39fa) | 2024-12-22 | _yeah_ | Cronjob for cleaning up expired SSL certificates in order to improve page load times with many domains (#2410) |
| [e0b93718](https://github.com/downtownallday/mailinabox-ldap/commit/e0b93718a33338115e953564170d87af6a55e1f9) | 2024-12-22 | _yeah_ | Revert "increase timeout for the nginx proxy that provides access to the Mail…" (#2411) |
| [2e0482e1](https://github.com/downtownallday/mailinabox-ldap/commit/2e0482e1817fd1a167b247a9137b86ee190d2947) | 2024-12-22 | _KiekerJan_ | Exclude the owncloud-backup folder from the nightly backup (#2413) |
| [0d738889](https://github.com/downtownallday/mailinabox-ldap/commit/0d7388899c02a3785714bfe75d711f5929b3ded2) | 2024-12-22 | _Tomasz Stanczak_ | Allow DSA end EllipticCurve private keys to be used additionally to RSA for HTTPS certificates (#2416) |
| [4f094f78](https://github.com/downtownallday/mailinabox-ldap/commit/4f094f7859cab6ee72792b96313c1c7d4407685d) | 2024-12-22 | _zoof_ | Change hour of daily tasks to run at 1am and only run full backups on weekends (#2424) |
| [564ed59b](https://github.com/downtownallday/mailinabox-ldap/commit/564ed59bb47da24c9ebc50ae9137e6dcbcae9826) | 2024-12-22 | _KiekerJan_ | Add check on ipv6 for spamhaus (#2428) |
| [9f87b36b](https://github.com/downtownallday/mailinabox-ldap/commit/9f87b36ba182e5ec6e519a4a6c27e9ead8c08469) | 2024-12-22 | _KiekerJan_ | add check on SOA record to determine up to date synchronization of secondary nameserver (#2429) |
| [e36c17fc](https://github.com/downtownallday/mailinabox-ldap/commit/e36c17fc72249fef1eb6b638c4fa3ad2ad765d32) | 2024-12-22 | _matidau_ | Fixstates only after Z-Push upgrade (#2432) |
| [3d59f2d7](https://github.com/downtownallday/mailinabox-ldap/commit/3d59f2d7e0d0c2794f88fc36d5fca11fc757f9a7) | 2024-12-22 | _KiekerJan_ | Update roundcube to 1.6.9 (#2440) |
| [ee0d750b](https://github.com/downtownallday/mailinabox-ldap/commit/ee0d750b8560b0e2e9a9bf0afe52ed12982cb7f2) | 2024-12-22 | _Harm Berntsen_ | Add missing php-xml package for Roundcube without Nextcloud (#2441) |
| [d8563be3](https://github.com/downtownallday/mailinabox-ldap/commit/d8563be38b2fa047725ee85c7330bdf775101cdd) | 2024-12-22 | _Paul_ | Disable MOTD advertisements (#2457) |
| [81b0e0a6](https://github.com/downtownallday/mailinabox-ldap/commit/81b0e0a64f3ed295205dbc5461bb8f4fc2791e3d) | 2024-12-22 | _Nicholas Wilson_ | Updated CHANGELOG.md, fix typo(s) (#2459) |
| [7ef859ce](https://github.com/downtownallday/mailinabox-ldap/commit/7ef859ce961ea24b70f7e4f8307f069a8f7b42b3) | 2024-12-13 | _matidau_ | Update zpush.sh to version 2.7.5 (#2463) |
| [a8d13b84](https://github.com/downtownallday/mailinabox-ldap/commit/a8d13b84b4e2ac7332ae825177c4f9aa7a01e782) | 2024-11-27 | _Downtown Allday_ | fix: NameError: name 'subprocess' is not defined (#2425) |
| [196f5588](https://github.com/downtownallday/mailinabox-ldap/commit/196f5588cc61e6531cda9491f3eb26f152630528) | 2024-10-07 | _downtownallday_ | eliminate the use of deprecated utcnow() |
| [119b11f0](https://github.com/downtownallday/mailinabox-ldap/commit/119b11f0227b6565148221bbd0bccb6ef4011a15) | 2024-10-04 | _downtownallday_ | remove upstream Vagrantfile |
| [696b597a](https://github.com/downtownallday/mailinabox-ldap/commit/696b597a9c4a7beea1e76ff5ade57f94bbd9770e) | 2024-10-04 | _downtownallday_ | use bash as 'source' is needed in provision scripts |
| [ae056e50](https://github.com/downtownallday/mailinabox-ldap/commit/ae056e507beaf86272dc5cdc20545f5d9c2ae41c) | 2024-10-04 | _downtownallday_ | validate argument |
| [3b6e6177](https://github.com/downtownallday/mailinabox-ldap/commit/3b6e6177d03c4fc10d3c3afe07760e8f4b49f181) | 2024-10-04 | _downtownallday_ | Remove vagrant references - everything has moved to lxd |
| [706c3e7a](https://github.com/downtownallday/mailinabox-ldap/commit/706c3e7af93add6ced094a473ebbc57716cd03f7) | 2024-09-20 | _downtownallday_ | QA: updates for recent nextcloud change |
| [62b691f4](https://github.com/downtownallday/mailinabox-ldap/commit/62b691f44a2f201f55052831e2b1ddad962a9cb9) | 2024-09-20 | _downtownallday_ | QA: updates for recent nextcloud changes |
| [1699ab8c](https://github.com/downtownallday/mailinabox-ldap/commit/1699ab8c02e6813075a65fff9903c85e31d52445) | 2024-09-17 | _matidau_ | Update zpush.sh to version 2.7.4 (#2423) |
| [3e0a6214](https://github.com/downtownallday/mailinabox-ldap/commit/3e0a6214508724496ce2c629b598cedf4be1b22c) | 2024-09-10 | _downtownallday_ | allow supplying a command line to execute to ssh remove debugging echo statements add -q argument to suppress outputting lxc command line |
| [4fedfb37](https://github.com/downtownallday/mailinabox-ldap/commit/4fedfb377da393d7fe22ca8781a964e9759a18ce) | 2024-09-10 | _downtownallday_ | during wait for boot, also wait until vm has an ip address |
| [2e0b37a0](https://github.com/downtownallday/mailinabox-ldap/commit/2e0b37a09a964c0e7499c2bff7d4f2d25361e9d9) | 2024-09-07 | _downtownallday_ | fix syntax error |
| [6d25bc47](https://github.com/downtownallday/mailinabox-ldap/commit/6d25bc47bf20ba48c53bc861962e9356713fcbfa) | 2024-09-05 | _downtownallday_ | add a restart command |
| [54a3bd10](https://github.com/downtownallday/mailinabox-ldap/commit/54a3bd100c43710800ce3208acf1380e071bc0a3) | 2024-09-04 | _downtownallday_ | Add provision defaults to lxc init |
| [0fce66db](https://github.com/downtownallday/mailinabox-ldap/commit/0fce66dbc7e3c46b08d31698deca27badfbb0682) | 2024-09-03 | _downtownallday_ | back out assert_kernel_modules |
| [446aacb9](https://github.com/downtownallday/mailinabox-ldap/commit/446aacb9b6ae4dce8c021c8d1a4b09efefebabdb) | 2024-09-03 | _downtownallday_ | Don't exit on missing kernel module during non-interactive scenario |
| [c027db8b](https://github.com/downtownallday/mailinabox-ldap/commit/c027db8bf49091e6d4cef214722a062635e80d3c) | 2024-09-03 | _downtownallday_ | reword comment |
| [ca123515](https://github.com/downtownallday/mailinabox-ldap/commit/ca123515aad102327701b18a7d65d180f800b815) | 2024-09-02 | _Downtown Allday_ | fix variable (#2439) |
| [a1d6f671](https://github.com/downtownallday/mailinabox-ldap/commit/a1d6f6713578097b1b68bb0cea80f6327a2c3577) | 2024-09-02 | _downtownallday_ | change from vagrant to lxd as the virtualization system |
| [a79a6c00](https://github.com/downtownallday/mailinabox-ldap/commit/a79a6c00eb252de8c2581744894c8173a34b2f92) | 2024-09-02 | _downtownallday_ | encryption-at-rest: Ensure required kernel modules are installed |
| [3b8f4a2f](https://github.com/downtownallday/mailinabox-ldap/commit/3b8f4a2fe8bd686f9d3ff405d9bb380c3c6315a8) | 2024-08-30 | _matidau_ | Z-Push remove config lines no longer supported (#2433) |
| [f453c44d](https://github.com/downtownallday/mailinabox-ldap/commit/f453c44d524b68a3a99f567168dd401f88556633) | 2024-08-30 | _darren_ | Update setup to handle multiple SSH ports (#2437) |

5
changelog/v71a.md Normal file
View File

@ -0,0 +1,5 @@
## Commits for v71a
| COMMIT | DATE | AUTHOR | TITLE |
| ------ | ---- | ------ | ----- |
| [e6c354c3](https://github.com/downtownallday/mailinabox-ldap/commit/e6c354c3125bdfdf32fecabb851288a385705e72) | 2025-01-06 | _Joshua Tauberer_ | v71a |
| [432b470d](https://github.com/downtownallday/mailinabox-ldap/commit/432b470d2931a15a3761a3e35f1c30cac4e83b49) | 2025-01-06 | _Paul_ | New & improved Disable MOTD advertisements (#2470) |

1
conf/nginx-primaryonly.conf
View File

@ -8,7 +8,6 @@
rewrite ^/admin/munin$ /admin/munin/ redirect; rewrite ^/admin/munin$ /admin/munin/ redirect;
location /admin/ { location /admin/ {
proxy_pass http://127.0.0.1:10222/; proxy_pass http://127.0.0.1:10222/;
proxy_read_timeout 600s;
proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-For $remote_addr;
add_header X-Frame-Options "DENY"; add_header X-Frame-Options "DENY";
add_header X-Content-Type-Options nosniff; add_header X-Content-Type-Options nosniff;

19
management/backup.py
View File

@ -19,6 +19,7 @@
import os, os.path, re, datetime, sys import os, os.path, re, datetime, sys
import dateutil.parser, dateutil.relativedelta, dateutil.tz import dateutil.parser, dateutil.relativedelta, dateutil.tz
from datetime import date
import rtyaml import rtyaml
from exclusiveprocess import Lock from exclusiveprocess import Lock
@ -167,6 +168,8 @@ def should_force_full(config, env):
# since the last full backup is greater than half the size # since the last full backup is greater than half the size
# of that full backup. # of that full backup.
inc_size = 0 inc_size = 0
# Check if day of week is a weekend day
weekend = date.today().weekday()>=5
for bak in backup_status(env)["backups"]: for bak in backup_status(env)["backups"]:
if not bak["full"]: if not bak["full"]:
# Scan through the incremental backups cumulating # Scan through the incremental backups cumulating
@ -175,12 +178,14 @@ def should_force_full(config, env):
else: else:
# ...until we reach the most recent full backup. # ...until we reach the most recent full backup.
# Return if we should to a full backup, which is based # Return if we should to a full backup, which is based
# on the size of the increments relative to the full # on whether it is a weekend day, the size of the
# backup, as well as the age of the full backup. # increments relative to the full backup, as well as
if inc_size > .5*bak["size"]: # the age of the full backup.
return True if weekend:
if dateutil.parser.parse(bak["date"]) + datetime.timedelta(days=config["min_age_in_days"]*10+1) < datetime.datetime.now(dateutil.tz.tzlocal()): if inc_size > .5*bak["size"]:
return True return True
if dateutil.parser.parse(bak["date"]) + datetime.timedelta(days=config["min_age_in_days"]*10+1) < datetime.datetime.now(dateutil.tz.tzlocal()):
return True
return False return False
else: else:
# If we got here there are no (full) backups, so make one. # If we got here there are no (full) backups, so make one.
@ -348,6 +353,7 @@ def perform_backup(full_backup):
"--verbosity", "warning", "--no-print-statistics", "--verbosity", "warning", "--no-print-statistics",
"--archive-dir", backup_cache_dir, "--archive-dir", backup_cache_dir,
"--exclude", backup_root, "--exclude", backup_root,
"--exclude", os.path.join(env["STORAGE_ROOT"], "owncloud-backup"),
"--volsize", "250", "--volsize", "250",
"--gpg-options", "'--cipher-algo=AES256'", "--gpg-options", "'--cipher-algo=AES256'",
"--allow-source-mismatch", "--allow-source-mismatch",
@ -429,6 +435,7 @@ def run_duplicity_verification():
"--compare-data", "--compare-data",
"--archive-dir", backup_cache_dir, "--archive-dir", backup_cache_dir,
"--exclude", backup_root, "--exclude", backup_root,
"--exclude", os.path.join(env["STORAGE_ROOT"], "owncloud-backup"),
*get_duplicity_additional_args(env), *get_duplicity_additional_args(env),
get_duplicity_target_url(config), get_duplicity_target_url(config),
env["STORAGE_ROOT"], env["STORAGE_ROOT"],

30
management/ssl_certificates.py
View File

@ -23,7 +23,7 @@ def get_ssl_certificates(env):
# that the certificates are good for to the best certificate for # that the certificates are good for to the best certificate for
# the domain. # the domain.
from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey from cryptography.hazmat.primitives.asymmetric import dsa, rsa, ec
from cryptography.x509 import Certificate from cryptography.x509 import Certificate
# The certificates are all stored here: # The certificates are all stored here:
@ -44,6 +44,12 @@ def get_ssl_certificates(env):
# the cert that it should be a # the cert that it should be a
# symlink to. # symlink to.
continue continue
if fn in ['ca','ca_certificate.pem','ca_private_key.pem']:
# Ignore as these are for generating a temporary
# "self-signed" certificate in a virgin setup (before
# Let's Encrypt gives us a certificate).
#
continue
fn = os.path.join(ssl_root, fn) fn = os.path.join(ssl_root, fn)
if os.path.isfile(fn): if os.path.isfile(fn):
yield fn yield fn
@ -68,13 +74,15 @@ def get_ssl_certificates(env):
# Not a valid PEM format for a PEM type we care about. # Not a valid PEM format for a PEM type we care about.
continue continue
# Is it a private key?
if isinstance(pem, RSAPrivateKey):
private_keys[pem.public_key().public_numbers()] = { "filename": fn, "key": pem }
# Is it a certificate? # Is it a certificate?
if isinstance(pem, Certificate): if isinstance(pem, Certificate):
certificates.append({ "filename": fn, "cert": pem }) certificates.append({ "filename": fn, "cert": pem })
# It is a private key
elif (isinstance(pem, rsa.RSAPrivateKey)
or isinstance(pem, dsa.DSAPrivateKey)
or isinstance(pem, ec.EllipticCurvePrivateKey)):
private_keys[pem.public_key().public_numbers()] = { "filename": fn, "key": pem }
# Process the certificates. # Process the certificates.
domains = { } domains = { }
@ -100,7 +108,7 @@ def get_ssl_certificates(env):
# Sort the certificates to prefer good ones. # Sort the certificates to prefer good ones.
import datetime import datetime
now = datetime.datetime.utcnow() now = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None)
ret = { } ret = { }
for domain, cert_list in domains.items(): for domain, cert_list in domains.items():
#for c in cert_list: print(domain, c["cert"].not_valid_before, c["cert"].not_valid_after, "("+str(now)+")", c["cert"].issuer, c["cert"].subject, c._filename if hasattr(c,"_filename") else "") #for c in cert_list: print(domain, c["cert"].not_valid_before, c["cert"].not_valid_after, "("+str(now)+")", c["cert"].issuer, c["cert"].subject, c._filename if hasattr(c,"_filename") else "")
@ -518,7 +526,7 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
# Check that the ssl_certificate & ssl_private_key files are good # Check that the ssl_certificate & ssl_private_key files are good
# for the provided domain. # for the provided domain.
from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey from cryptography.hazmat.primitives.asymmetric import rsa, dsa, ec
from cryptography.x509 import Certificate from cryptography.x509 import Certificate
# The ssl_certificate file may contain a chain of certificates. We'll # The ssl_certificate file may contain a chain of certificates. We'll
@ -552,7 +560,9 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
except ValueError as e: except ValueError as e:
return (f"The private key file {ssl_private_key} is not a private key file: {e!s}", None) return (f"The private key file {ssl_private_key} is not a private key file: {e!s}", None)
if not isinstance(priv_key, RSAPrivateKey): if (not isinstance(priv_key, rsa.RSAPrivateKey)
and not isinstance(priv_key, dsa.DSAPrivateKey)
and not isinstance(priv_key, ec.EllipticCurvePrivateKey)):
return ("The private key file %s is not a private key file." % ssl_private_key, None) return ("The private key file %s is not a private key file." % ssl_private_key, None)
if priv_key.public_key().public_numbers() != cert.public_key().public_numbers(): if priv_key.public_key().public_numbers() != cert.public_key().public_numbers():
@ -579,7 +589,7 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
# Check that the certificate hasn't expired. The datetimes returned by the # Check that the certificate hasn't expired. The datetimes returned by the
# certificate are 'naive' and in UTC. We need to get the current time in UTC. # certificate are 'naive' and in UTC. We need to get the current time in UTC.
import datetime import datetime
now = datetime.datetime.utcnow() now = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None)
if not(cert.not_valid_before <= now <= cert.not_valid_after): if not(cert.not_valid_before <= now <= cert.not_valid_after):
return (f"The certificate has expired or is not yet valid. It is valid from {cert.not_valid_before} to {cert.not_valid_after}.", None) return (f"The certificate has expired or is not yet valid. It is valid from {cert.not_valid_before} to {cert.not_valid_after}.", None)
@ -655,7 +665,7 @@ def load_pem(pem):
msg = "File is not a valid PEM-formatted file." msg = "File is not a valid PEM-formatted file."
raise ValueError(msg) raise ValueError(msg)
pem_type = pem_type.group(1) pem_type = pem_type.group(1)
if pem_type in {b"RSA PRIVATE KEY", b"PRIVATE KEY"}: if pem_type.endswith(b"PRIVATE KEY"):
return serialization.load_pem_private_key(pem, password=None, backend=default_backend()) return serialization.load_pem_private_key(pem, password=None, backend=default_backend())
if pem_type == b"CERTIFICATE": if pem_type == b"CERTIFICATE":
return load_pem_x509_certificate(pem, default_backend()) return load_pem_x509_certificate(pem, default_backend())

68
management/status_checks.py
View File

@ -293,26 +293,45 @@ def run_network_checks(env, output):
# The user might have ended up on an IP address that was previously in use # The user might have ended up on an IP address that was previously in use
# by a spammer, or the user may be deploying on a residential network. We # by a spammer, or the user may be deploying on a residential network. We
# will not be able to reliably send mail in these cases. # will not be able to reliably send mail in these cases.
# See https://www.spamhaus.org/news/article/807/using-our-public-mirrors-check-your-return-codes-now. for
# information on spamhaus return codes
rev_ip4 = ".".join(reversed(env['PUBLIC_IP'].split('.'))) rev_ip4 = ".".join(reversed(env['PUBLIC_IP'].split('.')))
zen = query_dns(rev_ip4+'.zen.spamhaus.org', 'A', nxdomain=None) zen = query_dns(rev_ip4+'.zen.spamhaus.org', 'A', nxdomain=None)
evaluate_spamhaus_lookup(env['PUBLIC_IP'], 'IPv4', rev_ip4, output, zen)
if not env['PUBLIC_IPV6']:
return
from ipaddress import IPv6Address
rev_ip6 = ".".join(reversed(IPv6Address(env['PUBLIC_IPV6']).exploded.split(':')))
zen = query_dns(rev_ip6+'.zen.spamhaus.org', 'A', nxdomain=None)
evaluate_spamhaus_lookup(env['PUBLIC_IPV6'], 'IPv6', rev_ip6, output, zen)
def evaluate_spamhaus_lookup(lookupaddress, lookuptype, lookupdomain, output, zen):
# See https://www.spamhaus.org/news/article/807/using-our-public-mirrors-check-your-return-codes-now. for
# information on spamhaus return codes
if zen is None: if zen is None:
output.print_ok("IP address is not blacklisted by zen.spamhaus.org.") output.print_ok(f"{lookuptype} address is not blacklisted by zen.spamhaus.org.")
elif zen == "[timeout]": elif zen == "[timeout]":
output.print_warning("Connection to zen.spamhaus.org timed out. Could not determine whether this box's IP address is blacklisted. Please try again later.") output.print_warning(f"""Connection to zen.spamhaus.org timed out. Could not determine whether this box's
{lookuptype} address is blacklisted. Please try again later.""")
elif zen == "[Not Set]": elif zen == "[Not Set]":
output.print_warning("Could not connect to zen.spamhaus.org. Could not determine whether this box's IP address is blacklisted. Please try again later.") output.print_warning(f"""Could not connect to zen.spamhaus.org. Could not determine whether this box's
{lookuptype} address is blacklisted. Please try again later.""")
elif zen == "127.255.255.252": elif zen == "127.255.255.252":
output.print_warning("Incorrect spamhaus query: %s. Could not determine whether this box's IP address is blacklisted." % (rev_ip4+'.zen.spamhaus.org')) output.print_warning(f"""Incorrect spamhaus query: {lookupdomain + '.zen.spamhaus.org'}. Could not determine whether
this box's {lookuptype} address is blacklisted.""")
elif zen == "127.255.255.254": elif zen == "127.255.255.254":
output.print_warning("Mail-in-a-Box is configured to use a public DNS server. This is not supported by spamhaus. Could not determine whether this box's IP address is blacklisted.") output.print_warning(f"""Mail-in-a-Box is configured to use a public DNS server. This is not supported by
spamhaus. Could not determine whether this box's {lookuptype} address is blacklisted.""")
elif zen == "127.255.255.255": elif zen == "127.255.255.255":
output.print_warning("Too many queries have been performed on the spamhaus server. Could not determine whether this box's IP address is blacklisted.") output.print_warning(f"""Too many queries have been performed on the spamhaus server. Could not determine
whether this box's {lookuptype} address is blacklisted.""")
else: else:
output.print_error("""The IP address of this machine {} is listed in the Spamhaus Block List (code {}), output.print_error(f"""The {lookuptype} address of this machine {lookupaddress} is listed in the Spamhaus Block
which may prevent recipients from receiving your email. See http://www.spamhaus.org/query/ip/{}.""".format(env['PUBLIC_IP'], zen, env['PUBLIC_IP'])) List (code {zen}), which may prevent recipients from receiving your email. See
http://www.spamhaus.org/query/ip/{lookupaddress}.""")
def run_domain_checks(rounded_time, env, output, pool, domains_to_check=None): def run_domain_checks(rounded_time, env, output, pool, domains_to_check=None):
# Get the list of domains we handle mail for. # Get the list of domains we handle mail for.
@ -532,6 +551,8 @@ def check_dns_zone(domain, env, output, dns_zonefiles):
# Check that each custom secondary nameserver resolves the IP address. # Check that each custom secondary nameserver resolves the IP address.
if custom_secondary_ns and not probably_external_dns: if custom_secondary_ns and not probably_external_dns:
SOARecord = query_dns(domain, "SOA", at=env['PUBLIC_IP'])# Explicitly ask the local dns server.
for ns in custom_secondary_ns: for ns in custom_secondary_ns:
# We must first resolve the nameserver to an IP address so we can query it. # We must first resolve the nameserver to an IP address so we can query it.
ns_ips = query_dns(ns, "A") ns_ips = query_dns(ns, "A")
@ -541,15 +562,36 @@ def check_dns_zone(domain, env, output, dns_zonefiles):
# Choose the first IP if nameserver returns multiple # Choose the first IP if nameserver returns multiple
ns_ip = ns_ips.split('; ')[0] ns_ip = ns_ips.split('; ')[0]
checkSOA = True
# Now query it to see what it says about this domain. # Now query it to see what it says about this domain.
ip = query_dns(domain, "A", at=ns_ip, nxdomain=None) ip = query_dns(domain, "A", at=ns_ip, nxdomain=None)
if ip == correct_ip: if ip == correct_ip:
output.print_ok("Secondary nameserver %s resolved the domain correctly." % ns) output.print_ok(f"Secondary nameserver {ns} resolved the domain correctly.")
elif ip is None: elif ip is None:
output.print_error("Secondary nameserver %s is not configured to resolve this domain." % ns) output.print_error(f"Secondary nameserver {ns} is not configured to resolve this domain.")
# No need to check SOA record if not configured as nameserver
checkSOA = False
elif ip == '[timeout]':
output.print_error(f"Secondary nameserver {ns} did not resolve this domain, result: {ip}")
checkSOA = False
else: else:
output.print_error(f"Secondary nameserver {ns} is not configured correctly. (It resolved this domain as {ip}. It should be {correct_ip}.)") output.print_error(f"Secondary nameserver {ns} is not configured correctly. (It resolved this domain as {ip}. It should be {correct_ip}.)")
if checkSOA:
# Check that secondary DNS server is synchronized with our primary DNS server. Simplified by checking the SOA record which has a version number
SOASecondary = query_dns(domain, "SOA", at=ns_ip)
if SOARecord == SOASecondary:
output.print_ok(f"Secondary nameserver {ns} has consistent SOA record.")
elif SOARecord == '[Not Set]':
output.print_error(f"Secondary nameserver {ns} has no SOA record configured.")
elif SOARecord == '[timeout]':
output.print_error(f"Secondary nameserver {ns} timed out on checking SOA record.")
else:
output.print_error(f"""Secondary nameserver {ns} has inconsistent SOA record (primary: {SOARecord} versus secondary: {SOASecondary}).
Check that synchronization between secondary and primary DNS servers is properly set-up.""")
def check_dns_zone_suggestions(domain, env, output, dns_zonefiles, domains_with_a_records): def check_dns_zone_suggestions(domain, env, output, dns_zonefiles, domains_with_a_records):
# Warn if a custom DNS record is preventing this or the automatic www redirect from # Warn if a custom DNS record is preventing this or the automatic www redirect from
# being served. # being served.

1
management/utils.py
View File

@ -215,6 +215,7 @@ def get_ssh_port():
def get_ssh_config_value(parameter_name): def get_ssh_config_value(parameter_name):
# Returns ssh configuration value for the provided parameter # Returns ssh configuration value for the provided parameter
import subprocess
try: try:
output = shell('check_output', ['sshd', '-T']) output = shell('check_output', ['sshd', '-T'])
except FileNotFoundError: except FileNotFoundError:

2
setup/bootstrap.sh
View File

@ -51,7 +51,7 @@ if [ -z "$TAG" ]; then
if [ "$UBUNTU_VERSION" == "Ubuntu 22.04 LTS" ]; then if [ "$UBUNTU_VERSION" == "Ubuntu 22.04 LTS" ]; then
# This machine is running Ubuntu 22.04, which is supported by # This machine is running Ubuntu 22.04, which is supported by
# Mail-in-a-Box versions 60 and later. # Mail-in-a-Box versions 60 and later.
TAG=v70 TAG=v71a
elif [ "$UBUNTU_VERSION" == "Ubuntu 18.04 LTS" ]; then elif [ "$UBUNTU_VERSION" == "Ubuntu 18.04 LTS" ]; then
# This machine is running Ubuntu 18.04, which is supported by # This machine is running Ubuntu 18.04, which is supported by
# Mail-in-a-Box versions 0.40 through 5x. # Mail-in-a-Box versions 0.40 through 5x.

4
setup/mail-dovecot.sh
View File

@ -115,14 +115,14 @@ tools/editconf.py /etc/dovecot/conf.d/10-auth.conf \
# Enable SSL, specify the location of the SSL certificate and private key files. # Enable SSL, specify the location of the SSL certificate and private key files.
# Use Mozilla's "Intermediate" recommendations at https://ssl-config.mozilla.org/#server=dovecot&server-version=2.2.33&config=intermediate&openssl-version=1.1.1, # Use Mozilla's "Intermediate" recommendations at https://ssl-config.mozilla.org/#server=dovecot&server-version=2.2.33&config=intermediate&openssl-version=1.1.1,
# except that the current version of Dovecot does not have a TLSv1.3 setting, so we only use TLSv1.2. # except that the current version of Dovecot does not have a TLSv1.3 setting, so we only use TLSv1.2.
tools/editconf.py /etc/dovecot/conf.d/10-ssl.conf \ tools/editconf.py /etc/dovecot/conf.d/10-ssl.conf -E \
ssl=required \ ssl=required \
"ssl_cert=<$STORAGE_ROOT/ssl/ssl_certificate.pem" \ "ssl_cert=<$STORAGE_ROOT/ssl/ssl_certificate.pem" \
"ssl_key=<$STORAGE_ROOT/ssl/ssl_private_key.pem" \ "ssl_key=<$STORAGE_ROOT/ssl/ssl_private_key.pem" \
"ssl_min_protocol=TLSv1.2" \ "ssl_min_protocol=TLSv1.2" \
"ssl_cipher_list=ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" \ "ssl_cipher_list=ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" \
"ssl_prefer_server_ciphers=no" \ "ssl_prefer_server_ciphers=no" \
"ssl_dh_parameters_length=2048" \ "ssl_dh_parameters_length=" \
"ssl_dh=<$STORAGE_ROOT/ssl/dh2048.pem" "ssl_dh=<$STORAGE_ROOT/ssl/dh2048.pem"
# Disable in-the-clear IMAP/POP because there is no reason for a user to transmit # Disable in-the-clear IMAP/POP because there is no reason for a user to transmit

2
setup/management.sh
View File

@ -126,7 +126,7 @@ minute=$((RANDOM % 60)) # avoid overloading mailinabox.email
cat > /etc/cron.d/mailinabox-nightly << EOF; cat > /etc/cron.d/mailinabox-nightly << EOF;
# Mail-in-a-Box --- Do not edit / will be overwritten on update. # Mail-in-a-Box --- Do not edit / will be overwritten on update.
# Run nightly tasks: backup, status checks. # Run nightly tasks: backup, status checks.
$minute 3 * * * root (cd $PWD && management/daily_tasks.sh) $minute 1 * * * root (cd $PWD && management/daily_tasks.sh)
EOF EOF
# Start the management server. # Start the management server.

9
setup/ssl.sh
View File

@ -219,3 +219,12 @@ fi
if [ ! -f "$STORAGE_ROOT/ssl/dh2048.pem" ]; then if [ ! -f "$STORAGE_ROOT/ssl/dh2048.pem" ]; then
openssl dhparam -out "$STORAGE_ROOT/ssl/dh2048.pem" 2048 openssl dhparam -out "$STORAGE_ROOT/ssl/dh2048.pem" 2048
fi fi
# Cleanup expired SSL certificates from $STORAGE_ROOT/ssl daily
cat > /etc/cron.daily/mailinabox-ssl-cleanup << EOF;
#!/bin/bash
# Mail-in-a-Box
# Cleanup expired SSL certificates
$(pwd)/tools/ssl_cleanup
EOF
chmod +x /etc/cron.daily/mailinabox-ssl-cleanup

9
setup/system.sh
View File

@ -92,6 +92,15 @@ fi
# (See https://discourse.mailinabox.email/t/journalctl-reclaim-space-on-small-mailinabox/6728/11.) # (See https://discourse.mailinabox.email/t/journalctl-reclaim-space-on-small-mailinabox/6728/11.)
tools/editconf.py /etc/systemd/journald.conf MaxRetentionSec=10day tools/editconf.py /etc/systemd/journald.conf MaxRetentionSec=10day
# ### Improve server privacy
# Disable MOTD adverts to prevent revealing server information in MOTD request headers
# See https://ma.ttias.be/what-exactly-being-sent-ubuntu-motd/
if [ -f /etc/default/motd-news ]; then
tools/editconf.py /etc/default/motd-news ENABLED=0
rm -f /var/cache/motd-news
fi
# ### Add PPAs. # ### Add PPAs.
# We install some non-standard Ubuntu packages maintained by other # We install some non-standard Ubuntu packages maintained by other

6
setup/webmail.sh
View File

@ -34,7 +34,7 @@ echo "Installing Roundcube (webmail)..."
apt_install \ apt_install \
dbconfig-common \ dbconfig-common \
php"${PHP_VER}"-cli php"${PHP_VER}"-sqlite3 php"${PHP_VER}"-intl php"${PHP_VER}"-common php"${PHP_VER}"-curl php"${PHP_VER}"-imap \ php"${PHP_VER}"-cli php"${PHP_VER}"-sqlite3 php"${PHP_VER}"-intl php"${PHP_VER}"-common php"${PHP_VER}"-curl php"${PHP_VER}"-imap \
php"${PHP_VER}"-gd php"${PHP_VER}"-pspell php"${PHP_VER}"-mbstring libjs-jquery libjs-jquery-mousewheel libmagic1 \ php"${PHP_VER}"-gd php"${PHP_VER}"-pspell php"${PHP_VER}"-mbstring php"${PHP_VER}"-xml libjs-jquery libjs-jquery-mousewheel libmagic1 \
sqlite3 sqlite3
apt_install php"${PHP_VER}"-ldap apt_install php"${PHP_VER}"-ldap
@ -49,8 +49,8 @@ apt_install php"${PHP_VER}"-ldap
# https://github.com/mstilkerich/rcmcarddav/releases # https://github.com/mstilkerich/rcmcarddav/releases
# The easiest way to get the package hashes is to run this script and get the hash from # The easiest way to get the package hashes is to run this script and get the hash from
# the error message. # the error message.
VERSION=1.6.8 VERSION=1.6.10
HASH=00586f5163b3f6c1b0798be745982e3547b1b24a HASH=0cfbb457e230793df8c56c2e6d3655cf3818f168
PERSISTENT_LOGIN_VERSION=version-5.3.0 PERSISTENT_LOGIN_VERSION=version-5.3.0
HTML5_NOTIFIER_VERSION=68d9ca194212e15b3c7225eb6085dbcf02fd13d7 # version 0.6.4+ HTML5_NOTIFIER_VERSION=68d9ca194212e15b3c7225eb6085dbcf02fd13d7 # version 0.6.4+
CARDDAV_VERSION=4.4.3 CARDDAV_VERSION=4.4.3

8
setup/zpush.sh
View File

@ -31,8 +31,8 @@ apt_install \
phpenmod -v "$PHP_VER" imap phpenmod -v "$PHP_VER" imap
# Copy Z-Push into place. # Copy Z-Push into place.
VERSION=2.7.3 VERSION=2.7.5
TARGETHASH=9d4bec41935e9a4e07880c5ff915bcddbda4443b TARGETHASH=f0b0b06e255f3496173ab9d28a4f2d985184720e
needs_update=0 #NODOC needs_update=0 #NODOC
if [ ! -f /usr/local/lib/z-push/version ]; then if [ ! -f /usr/local/lib/z-push/version ]; then
needs_update=1 #NODOC needs_update=1 #NODOC
@ -120,4 +120,6 @@ restart_service php"$PHP_VER"-fpm
# Fix states after upgrade # Fix states after upgrade
hide_output php"$PHP_VER" /usr/local/lib/z-push/z-push-admin.php -a fixstates if [ $needs_update == 1 ]; then
hide_output php"$PHP_VER" /usr/local/lib/z-push/z-push-admin.php -a fixstates
fi

18
tests/bin/lx_functions.sh
View File

@ -201,7 +201,25 @@ lx_wait_for_boot() {
echo "" echo ""
echo -n "Wait for cloud-init " echo -n "Wait for cloud-init "
lxc --project "$project" exec "$inst" -- cloud-init status --wait lxc --project "$project" exec "$inst" -- cloud-init status --wait
local rc=$?
if [ $rc -eq 0 ]; then
echo "Wait for ip address "
local ip=""
local count=0
while [ $count -lt 10 ]; do
let count+=1
ip="$(lxc --project "$project" exec "$inst" -- hostname -I | awk '{print $1}')"
rc=$?
echo " [${count}] got: $ip"
if [ $rc -ne 0 -o "$ip" != "" ];then
break
fi
sleep 5
done
fi
echo "" echo ""
return $rc
} }
lx_get_ssh_identity() { lx_get_ssh_identity() {

2
tests/bin/provision_functions.sh
View File

@ -67,7 +67,7 @@ provision_shell() {
else else
local tmp=$(mktemp) local tmp=$(mktemp)
echo "#!/bin/sh" >"$tmp" echo "#!/bin/bash" >"$tmp"
cat >>"$tmp" cat >>"$tmp"
lxc --project "$project" file push "$tmp" "${inst}${remote_path}" $lxc_flags || return 1 lxc --project "$project" file push "$tmp" "${inst}${remote_path}" $lxc_flags || return 1
rm -f "$tmp" rm -f "$tmp"

59
tests/bin/vlx
View File

@ -37,6 +37,12 @@
D=$(dirname "$BASH_SOURCE") D=$(dirname "$BASH_SOURCE")
. "$D/lx_functions.sh" || exit 1 . "$D/lx_functions.sh" || exit 1
show_cl="yes"
if [ "$1" = "-q" ]; then
show_cl="no"
shift
fi
vlx_guess() { vlx_guess() {
if [ $# -eq 2 ]; then if [ $# -eq 2 ]; then
LX_PROJECT="$1" LX_PROJECT="$1"
@ -83,7 +89,6 @@ vlx_exec() {
if [ "${args[$idx]}" = "--" ]; then if [ "${args[$idx]}" = "--" ]; then
if [ $idx -eq 3 ]; then if [ $idx -eq 3 ]; then
# format 1 with cwd # format 1 with cwd
echo "f1"
wd="$3" wd="$3"
vlx_guess "$1" "$2" || return 1 vlx_guess "$1" "$2" || return 1
shift; shift; shift; shift; shift; shift; shift; shift;
@ -92,12 +97,10 @@ vlx_exec() {
if [ "${2#/}" != "$2" ]; then if [ "${2#/}" != "$2" ]; then
# wd starts with /, so it's a path # wd starts with /, so it's a path
# format 2 # format 2
echo "f2"
wd="$2" wd="$2"
vlx_guess "" "$1" || return 1 vlx_guess "" "$1" || return 1
else else
# format 1 w/o cwd # format 1 w/o cwd
echo "f1 w/o cwd"
vlx_guess "$1" "$2" || return 1 vlx_guess "$1" "$2" || return 1
fi fi
shift; shift; shift; shift; shift; shift;
@ -106,12 +109,10 @@ vlx_exec() {
if [ "${1#/}" != "$1" ]; then if [ "${1#/}" != "$1" ]; then
# wd starts with /, so it's a path # wd starts with /, so it's a path
# format 3 # format 3
echo "f3"
wd="$1" wd="$1"
vlx_guess || return 1 vlx_guess || return 1
else else
# format 2 w/o cwd # format 2 w/o cwd
echo "f2 w/o cwd"
vlx_guess "$1" || return 1 vlx_guess "$1" || return 1
fi fi
shift; shift; shift; shift;
@ -121,7 +122,6 @@ vlx_exec() {
fi fi
else else
# format 4 # format 4
echo "f4"
vlx_guess || return 1 vlx_guess || return 1
fi fi
@ -129,28 +129,43 @@ vlx_exec() {
if [ ! -z "$wd" ]; then if [ ! -z "$wd" ]; then
xargs="--cwd $wd" xargs="--cwd $wd"
fi fi
echo lxc --project "$LX_PROJECT" exec "$LX_INST" $xargs -- "$@" [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" exec "$LX_INST" $xargs -- "$@"
lxc --project "$LX_PROJECT" exec "$LX_INST" $xargs -- "$@" lxc --project "$LX_PROJECT" exec "$LX_INST" $xargs -- "$@"
} }
vlx_shell() { vlx_shell() {
vlx_guess "$@" || return 1 vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" exec "$LX_INST" -- bash [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" exec "$LX_INST" -- bash
lxc --project "$LX_PROJECT" exec "$LX_INST" -- bash lxc --project "$LX_PROJECT" exec "$LX_INST" -- bash
} }
vlx_hostname() { vlx_hostname() {
vlx_guess "$@" || return 1 vlx_guess "$@" || return 1
local host
lxc --project "$LX_PROJECT" exec "$LX_INST" -- /usr/bin/hostname --fqdn || return 1 lxc --project "$LX_PROJECT" exec "$LX_INST" -- /usr/bin/hostname --fqdn || return 1
} }
vlx_ipaddr() {
vlx_guess "$@" || return 1
local hostip
hostip="$(lxc --project "$LX_PROJECT" exec "$LX_INST" -- /usr/bin/hostname -I)"
[ $? -ne 0 -o -z "$hostip" ] && return 1
awk '{print $1}' <<<"$hostip"
}
vlx_ssh() { vlx_ssh() {
local host="$1" local host="$1"
if [ "$host" = "--" ]; then
host=""
else
shift
fi
if [ -z "$host" ]; then if [ -z "$host" ]; then
host="$(vlx_hostname)" host="$(vlx_ipaddr)"
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Could not determine hostname, please specify" echo "Could not determine ip address, please specify"
host="" host=""
fi fi
if [ -z "$host" ]; then if [ -z "$host" ]; then
@ -161,20 +176,22 @@ vlx_ssh() {
local id="$(lx_get_ssh_identity)" local id="$(lx_get_ssh_identity)"
local known_hosts="$(lx_get_ssh_known_hosts)" local known_hosts="$(lx_get_ssh_known_hosts)"
local vmuser="vmuser" local vmuser="vmuser"
#echo ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host" #echo ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host" "$@"
echo "Connecting to $vmuser@$host ..." echo "Connecting to $vmuser@$host ..."
ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host" ssh -i "$id" -o UserKnownHostsFile="$known_hosts" -o StrictHostKeyChecking=no "$vmuser@$host" "$@"
} }
vlx_list() { vlx_list() {
vlx_guess "$1" || return 1 vlx_guess "$1" || return 1
echo lxc --project "$LX_PROJECT" list [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" list
lxc --project "$LX_PROJECT" list lxc --project "$LX_PROJECT" list
} }
vlx_images() { vlx_images() {
vlx_guess "$1" || return 1 vlx_guess "$1" || return 1
echo lxc --project "$LX_PROJECT" image list [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" image list
lxc --project "$LX_PROJECT" image list lxc --project "$LX_PROJECT" image list
} }
@ -191,19 +208,22 @@ vlx_up() {
vlx_start() { vlx_start() {
vlx_guess "$@" || return 1 vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" start "$LX_INST" [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" start "$LX_INST"
lxc --project "$LX_PROJECT" start "$LX_INST" lxc --project "$LX_PROJECT" start "$LX_INST"
} }
vlx_stop() { vlx_stop() {
vlx_guess "$@" || return 1 vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" stop "$LX_INST" [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" stop "$LX_INST"
lxc --project "$LX_PROJECT" stop "$LX_INST" lxc --project "$LX_PROJECT" stop "$LX_INST"
} }
vlx_delete() { vlx_delete() {
vlx_guess "$@" || return 1 vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" delete --force --interactive "$LX_INST" [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" delete --force --interactive "$LX_INST"
lxc --project "$LX_PROJECT" delete --force --interactive "$LX_INST" lxc --project "$LX_PROJECT" delete --force --interactive "$LX_INST"
} }
@ -224,7 +244,8 @@ vlx_status() {
vlx_restart() { vlx_restart() {
vlx_guess "$@" || return 1 vlx_guess "$@" || return 1
echo lxc --project "$LX_PROJECT" restart "$LX_INST" [ "$show_cl" = "yes" ] &&
echo lxc --project "$LX_PROJECT" restart "$LX_INST"
lxc --project "$LX_PROJECT" restart "$LX_INST" lxc --project "$LX_PROJECT" restart "$LX_INST"
} }

9
tests/lib/python/browser/NcContactsAutomation.py
View File

@ -26,10 +26,11 @@ class NcContactsAutomation(object):
els = d.find_els('div.contacts-list div.list-item-content,div.option__details') els = d.find_els('div.contacts-list div.list-item-content,div.option__details')
d.say_verbose('found %s contacts' % len(els)) d.say_verbose('found %s contacts' % len(els))
for el in els: for el in els:
# .line-one (nc 25+) # .list-item-content__name (nc 29+)
# .line-one (nc 25-28)
# .option__lineone (nc <25) # .option__lineone (nc <25)
fullname = el.find_el('.line-one,.option__lineone').content().strip() fullname = el.find_el('.line-one,.option__lineone,.list-item-content__name').content().strip()
email = el.find_el('.line-two,.option__linetwo').content().strip() email = el.find_el('.line-two,.option__linetwo,.list-item-content__subname').content().strip()
d.say_verbose('contact: "%s" <%s>', fullname, email) d.say_verbose('contact: "%s" <%s>', fullname, email)
# NC 28: email not present in html # NC 28: email not present in html
ignore_email = True if email == '' else False ignore_email = True if email == '' else False
@ -43,7 +44,7 @@ class NcContactsAutomation(object):
d = self.d d = self.d
d.say("Wait for contact to load") d.say("Wait for contact to load")
d.wait_for_el('section.contact-details', secs=secs) d.wait_for_el('section.contact-details', secs=secs)
def delete_current_contact(self): def delete_current_contact(self):
d = self.d d = self.d
d.say("Delete current contact") d.say("Delete current contact")

10
tests/lib/python/browser/NextcloudAutomation.py
View File

@ -58,7 +58,10 @@ class NextcloudAutomation(object):
d.say("Logout of Nextcloud") d.say("Logout of Nextcloud")
self.click_avatar() self.click_avatar()
el = d.find_el('[data-id="logout"] a', throws=False) # nc < 26 el = d.find_el('a#logout', throws=False)
if not el:
# nc >= 29
el = d.find_el('[data-id="logout"] a', throws=False) # nc < 26
if not el: if not el:
# nc >= 26 # nc >= 26
el = d.find_el('#logout > a', throws=False) el = d.find_el('#logout > a', throws=False)
@ -72,7 +75,10 @@ class NextcloudAutomation(object):
d = self.d d = self.d
d.say("Open contacts") d.say("Open contacts")
# nc 25+ # nc 25+
el = d.find_el('header [data-app-id="contacts"]', throws=False) el = d.find_el('header [title="Contacts"]', throws=False)
if not el:
# nc < 29
el = d.find_el('header [data-app-id="contacts"]', throws=False)
if not el: if not el:
# nc < 25 # nc < 25
el = d.find_el('header [data-id="contacts"]') el = d.find_el('header [data-id="contacts"]')

1
tests/lxd/create-backup/README.md Normal file
View File

@ -0,0 +1 @@
TODO

1
tests/lxd/majorupgrade/README.md Normal file
View File

@ -0,0 +1 @@
TODO - convert to lxd

0
tests/vagrant/majorupgrade/Vagrantfile → tests/lxd/majorupgrade/Vagrantfile vendored
View File

0
tests/vagrant/majorupgrade/majorupgrade.sh → tests/lxd/majorupgrade/majorupgrade.sh
View File

7
tests/lxd/vanilla/provision.sh
View File

@ -17,7 +17,7 @@ D=$(dirname "$BASH_SOURCE")
provision_start "" "/mailinabox" || exit 1 provision_start "" "/mailinabox" || exit 1
# Setup system # Setup system
if [ "$1" = "ciab" ]; then if [ "$TESTS" = "ciab" -o "$1" = "ciab" ]; then
# use a remote cloudinabox (does not have to be running) # use a remote cloudinabox (does not have to be running)
provision_shell <<<" provision_shell <<<"
cd /mailinabox cd /mailinabox
@ -44,7 +44,7 @@ exit \$rc
" "
provision_done $? provision_done $?
else elif [ -z "$1" ]; then
# vanilla (default - no miab integration) # vanilla (default - no miab integration)
provision_shell <<<" provision_shell <<<"
cd /mailinabox cd /mailinabox
@ -62,4 +62,7 @@ exit \$rc
" "
provision_done $? provision_done $?
else
echo "Invalid argument: $1"
exit 1
fi fi

13
tests/suites/nextcloud/contacts.py
View File

@ -7,6 +7,7 @@
##### details. ##### details.
##### #####
from browser.automation import ( from browser.automation import (
TestDriver, TestDriver,
TimeoutException, TimeoutException,
@ -50,16 +51,20 @@ try:
# open Contacts # open Contacts
# #
d.start("Open contacts app") d.start("Open contacts app")
contacts = nc.open_contacts() try:
contacts = nc.open_contacts()
except NoSuchElementException:
nc.close_first_run_wizard()
contacts = nc.open_contacts()
nc.wait_for_app_load() nc.wait_for_app_load()
# #
# handle selected operation # handle selected operation
# #
if op=='exists': if op=='exists':
d.start("Check that contact %s exists", contact['email']) d.start("Check that contact %s exists", contact['email'])
contacts.click_contact(contact) # raises NoSuchElementException if not found contacts.click_contact(contact) # raises NoSuchElementException if not found
elif op=='delete': elif op=='delete':
d.start("Delete contact %s", contact['email']) d.start("Delete contact %s", contact['email'])
contacts.click_contact(contact) contacts.click_contact(contact)
@ -68,7 +73,7 @@ try:
elif op=='nop': elif op=='nop':
pass pass
else: else:
raise ValueError('Invalid operation: %s' % op) raise ValueError('Invalid operation: %s' % op)

4
tests/suites/remote-nextcloud.sh
View File

@ -172,7 +172,7 @@ test_web_config() {
record "output=$REST_OUTPUT" record "output=$REST_OUTPUT"
if [ $code -eq 0 ]; then if [ $code -eq 0 ]; then
test_failure "carddav url works, but expecting 401/NotAuthenticated from server" test_failure "carddav url works, but expecting 401/NotAuthenticated from server"
elif [ $code -eq 1 -o $REST_HTTP_CODE -ne 401 ] || ! grep "NotAuthenticated" <<<"$REST_OUTPUT" >/dev/null; then elif [ $REST_HTTP_CODE -ne 401 ]; then
test_failure "carddav url doesn't work: $REST_ERROR" test_failure "carddav url doesn't work: $REST_ERROR"
fi fi
fi fi
@ -189,7 +189,7 @@ test_web_config() {
record "output=$REST_OUTPUT" record "output=$REST_OUTPUT"
if [ $code -eq 0 ]; then if [ $code -eq 0 ]; then
test_failure "caldav url works, but expecting 401/NotAuthenticated from server" test_failure "caldav url works, but expecting 401/NotAuthenticated from server"
elif [ $code -eq 1 -o $REST_HTTP_CODE -ne 401 ] || ! grep "NotAuthenticated" <<<"$REST_OUTPUT" >/dev/null; then elif [ $REST_HTTP_CODE -ne 401 ]; then
test_failure "caldav url doesn't work: $REST_ERROR" test_failure "caldav url doesn't work: $REST_ERROR"
fi fi
fi fi

3
tests/vagrant/.gitignore vendored
View File

@ -1,3 +0,0 @@
.vagrant
out
*-console.log

104
tests/vagrant/Vagrantfile vendored
View File

@ -1,104 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
load './funcs.rb'
Vagrant.configure("2") do |config|
config.vm.synced_folder "../..", "/mailinabox", id: "mailinabox", automount: false
use_preloaded_box config, "ubuntu/jammy64"
# fresh install with encryption-at-rest
if ENV['tests']=='all'
config.vm.define "remote-nextcloud-docker-ehdd" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
export PRIMARY_HOSTNAME=qa1.abc.com
export FEATURE_MUNIN=false
export EHDD_KEYFILE=$HOME/keyfile
echo -n "boo" >$EHDD_KEYFILE
tests/system-setup/remote-nextcloud-docker.sh || exit 1
tests/runner.sh -no-smtp-remote remote-nextcloud ehdd default || exit 2
SH
end
end
# remote-nextcloud-docker w/basic data
config.vm.define "remote-nextcloud-docker" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
export PRIMARY_HOSTNAME=qa2.abc.com
export FEATURE_MUNIN=false
tests/system-setup/remote-nextcloud-docker.sh upgrade --populate=basic || exit 1
tests/runner.sh -no-smtp-remote remote-nextcloud upgrade-basic default || exit 2
SH
end
# upgrade-from-upstream
config.vm.define "upgrade-from-upstream" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
export PRIMARY_HOSTNAME=qa3.abc.com
# TODO: change UPSTREAM_TAG to 'main' once upstream is installable
export UPSTREAM_TAG=v67
tests/system-setup/upgrade-from-upstream.sh --populate=basic --populate=totpuser || exit 1
tests/runner.sh -no-smtp-remote upgrade-basic upgrade-totpuser default || exit 2
SH
end
# upgrade
# this test is only needed when testing migrations from miabldap
# to a newer miabldap with a migration step
#
# upgrade will handle testing upgrades of
# miabldap with or without a new migration step
config.vm.define "upgrade" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
# TODO: remove DEB_PYTHON_INSTALL_LAYOUT once MIABLDAP_RELEASE_TAG >= v66 (see https://github.com/downtownallday/mailinabox-ldap/commit/371f5bc1b236de40a1ed5d9118140ee13fddf5dc)
export DEB_PYTHON_INSTALL_LAYOUT='deb'
export PRIMARY_HOSTNAME=upgrade.abc.com
tests/system-setup/upgrade.sh --populate=basic --populate=totpuser || exit 1
tests/runner.sh -no-smtp-remote upgrade-basic upgrade-totpuser default || exit 2
SH
end
# unsetvars: because miab sets bash '-e' to fail any setup script
# when a script command returns a non-zero exit code, and more
# importantly '-u' which fails scripts when any unset variable is
# accessed, this definition sets a minimal number of environment
# variables prior to running start.sh. Doing so will test that no
# failures occur during setup in the most common use case because
# other vagrant definitions in this file load
# tests/system-setup/setup-default.sh, which pre-assign a value to
# most variables.
if ENV['tests']=='all' or ENV['tests']=='pre-commit'
config.vm.define "unsetvars" do |m1|
m1.vm.hostname = "mailinabox.lan"
m1.vm.network "private_network", ip: "192.168.56.4"
m1.vm.provision :shell, :inline => <<-SH
export NONINTERACTIVE=1
export PUBLIC_IP=auto
export PUBLIC_IPV6=auto
export PRIMARY_HOSTNAME=auto
export SKIP_NETWORK_CHECKS=1
cd /mailinabox
setup/start.sh
SH
end
end
end

25
tests/vagrant/funcs.rb
View File

@ -1,25 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
def use_preloaded_box(obj, name, preloaded_dir=".")
obj.vm.box = String.new(name)
_name=name.sub! '/','-' # ubuntu/bionic64 => ubuntu-bionic64
if File.file?("#{preloaded_dir}/preloaded/preloaded-#{_name}.box")
# box name needs to be unique on the system
obj.vm.box = "preloaded-miabldap-#{_name}"
obj.vm.box_url = "file://" + Dir.pwd + "/#{preloaded_dir}/preloaded/preloaded-#{_name}.box"
if Vagrant.has_plugin?('vagrant-vbguest')
# do not update additions when booting this machine
obj.vbguest.auto_update = false
end
end
end
# Grab the name of the default interface
$default_network_interface = `ip route | awk '/^default/ {printf "%s", $5; exit 0}'`

86
tests/vagrant/parallel.sh
View File

@ -1,86 +0,0 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# Parallel provisioning for virtualbox because "The Vagrant VirtualBox
# provider does not support parallel execution at this time"
# (https://www.vagrantup.com/docs/providers/virtualbox/usage.html)
#
# Credit to:
# https://dzone.com/articles/parallel-provisioning-speeding
#
. "$(dirname "$0")/../lib/color-output.sh"
. "$(dirname "$0")/../lib/misc.sh"
if [ -z "$tests" ]; then
export tests="pre-commit"
fi
OUTPUT_DIR=out
#rm -rf "$OUTPUT_DIR"
mkdir -p "$OUTPUT_DIR"
# set total parallel vms to (#cores minus 1)
MAX_PROCS=$(cat /proc/cpuinfo | grep processor | wc -l)
let MAX_PROCS-=1
parallel_provision() {
while read box; do
outfile="$OUTPUT_DIR/$box.out.txt"
rm -f "$outfile"
echo "Provisioning '$box'. Output will be in: $outfile" 1>&2
echo $box
done | xargs -P $MAX_PROCS -I"BOXNAME" \
sh -c 'vagrant provision BOXNAME >'"$OUTPUT_DIR/"'BOXNAME.out.txt 2>&1 && echo "EXITCODE: 0" >> '"$OUTPUT_DIR/"'BOXNAME.out.txt || echo "EXITCODE: $?" >>'"$OUTPUT_DIR/"'BOXNAME.out.txt'
}
## -- main -- ##
start_time="$(date +%s)"
# start boxes sequentially to avoid vbox explosions
vagrant up --no-provision
# but run provision tasks in parallel
boxes="$(vagrant status | awk '/running \(/ {print $1}')"
echo "$boxes" | parallel_provision
# output overall result - Vagrantfile script must output "EXITCODE: <num>"
H1 "Results"
rc=0
for box in $boxes; do
file="$OUTPUT_DIR"/$box.out.txt
exitcode="$(tail "$file" | grep EXITCODE: | awk '{print $NF}')"
echo -n "$box: "
if [ -z "$exitcode" ]; then
danger "NO EXITCODE!"
[ $rc -eq 0 ] && rc=2
elif [ "$exitcode" == "0" ]; then
success "SUCCESS"
else
danger "FAILURE ($exitcode)"
rc=1
fi
done
# output elapsed time
end_time="$(date +%s)"
echo ""
echo "Elapsed time: $(elapsed_pretty $start_time $end_time)"
# exit
echo ""
echo "Guest VMs are running! Destroy them with 'vagrant destroy -f'"
exit $rc

2
tests/vagrant/preloaded/.gitignore vendored
View File

@ -1,2 +0,0 @@
*.box
src/

58
tests/vagrant/preloaded/Vagrantfile vendored
View File

@ -1,58 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
def checkout_tag_and_mount(obj, tag)
if "#{tag}" == ""
obj.vm.synced_folder "../../..", "/mailinabox", id: "mailinabox", automount: false
else
_srcdir="src/maibldap-#{tag}"
if not Dir.exist?(_srcdir)
puts "Cloning tag #{tag} to #{_srcdir}"
if tag.size==40 and tag.match?(/\A[0-9a-fA-F]+\Z/)
system("git clone #{ENV['MIABLDAP_GIT']} #{_srcdir}")
system("cd #{_srcdir}; git reset --hard #{tag}")
else
system("git clone -b #{tag} --depth 1 #{ENV['MIABLDAP_GIT']} #{_srcdir}")
end
end
obj.vm.synced_folder _srcdir, "/mailinabox", id: "mailinabox", automount: false
end
end
Vagrant.configure("2") do |config|
checkout_tag_and_mount config, ENV['RELEASE_TAG']
config.vm.define "preloaded-ubuntu-bionic64" do |m1|
m1.vm.box = "ubuntu/bionic64"
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
tests/vagrant/preloaded/prepvm.sh --no-dry-run
rc=$?
echo "$rc" > "/vagrant/prepcode.txt"
[ $rc -gt 0 ] && exit 1
exit 0
SH
end
config.vm.define "preloaded-ubuntu-jammy64" do |m1|
m1.vm.box = "ubuntu/jammy64"
m1.vm.boot_timeout = 30
m1.vm.provision :shell, :inline => <<-SH
cd /mailinabox
tests/vagrant/preloaded/prepvm.sh --no-dry-run
rc=$?
echo "$rc" > "/vagrant/prepcode.txt"
[ $rc -gt 0 ] && exit 1
exit 0
SH
end
end

134
tests/vagrant/preloaded/create_preloaded.sh
View File

@ -1,134 +0,0 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# load defaults for MIABLDAP_GIT and MIABLDAP_FINAL_RELEASE_TAG_BIONIC64 (make available to Vagrantfile)
pushd "../../.." >/dev/null
source tests/lib/color-output.sh
source tests/system-setup/setup-defaults.sh || exit 1
popd >/dev/null
H1 "Destroy any running boxes"
vagrant destroy -f
rm -f prepcode.txt
H1 "Ensure plugins are installed"
for plugin in "vagrant-vbguest" "vagrant-reload"
do
if ! vagrant plugin list | grep -F "$plugin" >/dev/null; then
vagrant plugin install "$plugin" || exit 1
fi
done
H1 "Upgrade base boxes"
vagrant box update
boxes=(
"preloaded-ubuntu-jammy64"
"preloaded-ubuntu-bionic64"
)
# preload packages from source of the following git tags. empty string
# means use the current source tree
tags=(
""
"$MIABLDAP_FINAL_RELEASE_TAG_BIONIC64"
)
try_reboot=(
true
false
)
idx=0
for box in "${boxes[@]}"
do
if [ -z "$1" ]; then
# no cli arguments - only process first box
[ $idx -ge 1 ] && break
else
# cli argument specifies "all" or a named box
if [ "$1" != "all" -a "$1" != "$box" ]; then
let idx+=1
continue
fi
fi
H1 "Provision: $box"
export RELEASE_TAG="${tags[$idx]}"
vagrant up $box | tee /tmp/$box.out
upcode=$?
if [ $upcode -eq 0 -a ! -e "./prepcode.txt" ] && ${try_reboot[$idx]} && grep -F 'Authentication failure' /tmp/$box.out >/dev/null; then
# note: upcode is 0 only if config.vm.boot_timeout is set.
# If this works it may be an indication that ruby's internal
# ssh does not support the algorithm required by the server,
# or the public key does not match (vagrant and vm out of
# sync)
echo ""
echo "VAGRANT AUTHENTICATION FAILURE - TRYING LOOSER ALLOWED SSHD ALGS"
if vagrant ssh $box -c "sudo bash -c 'echo PubkeyAcceptedAlgorithms +ssh-rsa > /etc/ssh/sshd_config.d/miabldap.conf; sudo systemctl restart sshd'"; then
vagrant halt $box
vagrant up $box
upcode=$?
fi
fi
if [ $upcode -ne 0 -a ! -e "./prepcode.txt" ] && ${try_reboot[$idx]}
then
# a reboot may be necessary if guest addtions was newly
# compiled by vagrant plugin "vagrant-vbguest"
echo ""
echo "VAGRANT UP RETURNED $upcode -- RETRYING AFTER REBOOT"
vagrant halt $box
vagrant up $box
upcode=$?
fi
rm -f /tmp/$box.out
let idx+=1
prepcode=$(cat "./prepcode.txt")
rm -f prepcode.txt
echo ""
echo "VAGRANT UP RETURNED $upcode"
echo "PREPVM RETURNED $prepcode"
if [ "$prepcode" != "0" -o $upcode -ne 0 ]; then
echo "FAILED!!!!!!!!"
vagrant destroy -f $box
exit 1
fi
if vagrant ssh $box -- cat /var/run/reboot-required >/dev/null 2>&1; then
echo "REBOOT REQUIRED"
vagrant reload $box
else
echo "REBOOT NOT REQUIRED"
fi
vagrant halt $box
vagrant package $box
rm -f $box.box
mv package.box $box.box
vagrant destroy -f $box
cached_name="$(sed 's/preloaded-/preloaded-miabldap-/' <<<"$box")"
echo "Removing cached box $cached_name"
if [ -e "../funcs.rb" ]; then
pushd .. > /dev/null
vagrant box remove $cached_name
code=$?
popd > /dev/null
else
vagrant box remove $cached_name
code=$?
fi
echo "Remove cache box result: $code - ignoring"
done

230
tests/vagrant/preloaded/prepvm.sh
View File

@ -1,230 +0,0 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# Run this on a VM to pre-install all the packages, then
# take a snapshot - it will greatly speed up subsequent
# test installs
#
# What won't be installed:
#
# Nextcloud and Roundcube are downloaded with wget by the setup
# scripts, so they are not included
#
# slapd - we want to test installation with setup/ldap.sh
#
if [ ! -d "setup" ]; then
echo "Run from the miab root directory"
exit 1
fi
source tests/lib/misc.sh
source tests/lib/system.sh
source tests/lib/color-output.sh
dry_run=true
start=$(date +%s)
if [ "$1" == "--no-dry-run" ]; then
dry_run=false
fi
if $dry_run; then
echo "WARNING: dry run is TRUE, no changes will be made"
fi
# prevent apt from running needrestart(1)
export NEEDRESTART_SUSPEND=true
# prevent interaction during package install
export DEBIAN_FRONTEND=noninteractive
# what major version of ubuntu are we installing on?
OS_MAJOR=$(. /etc/os-release; echo $VERSION_ID | awk -F. '{print $1}')
remove_line_continuation() {
local file="$1"
awk '
BEGIN { C=0 }
C==1 && /[^\\]$/ { C=0; print $0; next }
C==1 { printf("%s",substr($0,0,length($0)-1)); next }
/\\$/ { C=1; printf("%s",substr($0,0,length($0)-1)); next }
{ print $0 }' \
"$file"
}
install_packages() {
local return_code=0
while read line; do
pkgs=""
case "$line" in
apt_install* )
pkgs="$(cut -c12- <<<"$line")"
;;
"apt-get install"* )
pkgs="$(cut -c16- <<<"$line")"
;;
"apt install"* )
pkgs="$(cut -c12- <<<"$line")"
;;
esac
# don't install slapd
pkgs="$(sed 's/slapd//g' <<< "$pkgs")"
# manually set PHP_VER if necessary
if grep "PHP_VER" <<<"$pkgs" >/dev/null; then
pkgs="$(sed "s/\"\?\${*PHP_VER}*\"\?/$PHP_VER/g" <<< "$pkgs")"
fi
if [ ! -z "$pkgs" ]; then
H2 "install: $pkgs"
if ! $dry_run; then
exec_no_output apt-get install -y $pkgs
let return_code+=$?
fi
fi
done
return $return_code
}
install_ppas() {
H1 "Add apt repositories"
grep 'hide_output add-apt-repository' setup/system.sh |
while read line; do
line=$(sed 's/^hide_output //' <<< "$line")
H2 "$line"
if ! $dry_run; then
exec_no_output $line
fi
done
}
add_swap() {
H1 "Add a swap file to the system"
if ! $dry_run; then
dd if=/dev/zero of=/swapfile bs=1024 count=$[1024*1024] status=none
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo "/swapfile none swap sw 0 0" >> /etc/fstab
fi
}
# install PPAs from sources
install_ppas
# add swap file
add_swap
# obtain PHP_VER variable from sources
PHP_VER=$(source setup/functions.sh; echo $PHP_VER)
if ! $dry_run; then
H1 "Upgrade system"
H2 "apt update"
exec_no_output apt-get update -y || exit 1
H2 "apt upgrade"
exec_no_output apt-get upgrade -y --with-new-pkgs || exit 1
H2 "apt autoremove"
exec_no_output apt-get autoremove -y
fi
# without using the same installation order as setup/start.sh, we end
# up with the system's php getting installed in addition to the
# non-system php that may also installed by setup (don't know why,
# probably one of the packages has a dependency). create an ordered
# list of files to process so we get a similar system setup.
setup_files=( $(ls setup/*.sh) )
desired_order=(
setup/functions.sh
setup/preflight.sh
setup/questions.sh
setup/network-checks.sh
setup/system.sh
setup/ssl.sh
setup/dns.sh
setup/ldap.sh
setup/mail-postfix.sh
setup/mail-dovecot.sh
setup/mail-users.sh
setup/dkim.sh
setup/spamassassin.sh
setup/web.sh
setup/webmail.sh
setup/nextcloud.sh
setup/zpush.sh
setup/management.sh
setup/management-capture.sh
setup/munin.sh
setup/firstuser.sh
)
ordered_files=()
for file in "${desired_order[@]}" "${setup_files[@]}"; do
if [ -e "$file" ] && ! array_contains "$file" "${ordered_files[@]}"; then
ordered_files+=( "$file" )
fi
done
failed=0
for file in ${ordered_files[@]}; do
H1 "$file"
remove_line_continuation "$file" | install_packages
[ $? -ne 0 ] && let failed+=1
done
if ! $dry_run; then
# bonus
H1 "install extras"
H2 "openssh, emacs, ntpdate, net-tools, jq"
exec_no_output apt-get install -y openssh-server emacs-nox ntpdate net-tools jq || let failed+=1
# these are added by system-setup scripts and needed for test runner
H2 "python3-dnspython"
exec_no_output apt-get install -y python3-dnspython || let failed+=1
H2 "pyotp(pip)"
exec_no_output python3 -m pip install pyotp --quiet || let failed+=1
# ...and for browser-based tests
#H2 "x11" # needed for chromium w/head (not --headless)
#exec_no_output apt-get install -y xorg openbox xvfb gtk2-engines-pixbuf dbus-x11 xfonts-base xfonts-100dpi xfonts-75dpi xfonts-cyrillic xfonts-scalable x11-apps imagemagick || let failed+=1
H2 "chromium"
#exec_no_output apt-get install -y chromium-browser || let failed+=1
exec_no_output snap install chromium || let failed+=1
H2 "selenium(pip)"
exec_no_output python3 -m pip install selenium --quiet || let failed+=1
# remove apache, which is what setup will do
H2 "remove apache2"
exec_no_output apt-get -y purge apache2 apache2-\*
fi
end=$(date +%s)
echo ""
echo ""
if [ $failed -gt 0 ]; then
echo "$failed failures! ($(elapsed_pretty $start $end))"
echo ""
exit 1
else
echo "Successfully prepped in $(elapsed_pretty $start $end). Take a snapshot...."
echo ""
exit 0
fi

2
tests/vagrant/vanilla/.gitignore vendored
View File

@ -1,2 +0,0 @@
.vagrant
*-console.log

87
tests/vagrant/vanilla/Vagrantfile vendored
View File

@ -1,87 +0,0 @@
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
load '../funcs.rb'
Vagrant.configure("2") do |config|
config.vm.synced_folder "../../..", "/mailinabox", id: "mailinabox", automount: false
config.vm.network "public_network", bridge: "#$default_network_interface"
use_preloaded_box config, "ubuntu/jammy64", ".."
if ENV['tests']=='ciab'
# vanilla connected to ciab (ciab does not need to be up)
config.vm.define "vanilla" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cat >/tmp/provision.sh <<EOF
#!/bin/bash
if [ \\$EUID -ne 0 ]; then
echo "Must be root"
exit 1
fi
cd /mailinabox
export PRIMARY_HOSTNAME=vanilla.local
export NC_PROTO=https
export NC_HOST=vanilla-ciab.local
export NC_PORT=443
export NC_PREFIX=/
export SKIP_SYSTEM_UPDATE=0
tests/system-setup/vanilla.sh --qa-ca --enable-mod=remote-nextcloud
if ! ufw status | grep remote_nextcloud >/dev/null; then
# firewall rules aren't added when ciab is down
echo "For testing, allow ldaps from anywhere"
ufw allow ldaps
fi
echo "Add smart host alias - so \\$NC_HOST can send mail to/via this host"
(
source tests/lib/all.sh
rest_urlencoded POST /admin/mail/aliases/add qa@abc.com Test_1234 "address=@\\$NC_HOST" "description=smart-host" "permitted_senders=qa@abc.com" 2>/dev/null
echo "\\$REST_HTTP_CODE: \\$REST_OUTPUT"
)
EOF
chmod +x /tmp/provision.sh
/tmp/provision.sh
SH
end # vanilla connected to ciab
else
# vanilla (default) install
config.vm.define "vanilla" do |m1|
m1.vm.provision :shell, :inline => <<-SH
cat >/tmp/provision.sh <<EOF
#!/bin/bash
if [ \\$EUID -ne 0 ]; then
echo "Must be root"
exit 1
fi
start=\\$(date +%s)
cd /mailinabox
export PRIMARY_HOSTNAME=vanilla.local
#export FEATURE_MUNIN=false
#export FEATURE_NEXTCLOUD=false
export SKIP_SYSTEM_UPDATE=0
tests/system-setup/vanilla.sh
# --enable-mod=roundcube-master \
# --enable-mod=roundcube-debug \
# --enable-mod=rcmcarddav-composer
end=\\$(date +%s)
echo "Provisioning took \\$(source tests/lib/misc.sh; elapsed_pretty \\$start \\$end)"
EOF
chmod +x /tmp/provision.sh
/tmp/provision.sh
SH
end # vanilla (default)
end
end

27
tools/ssl_cleanup Executable file
View File

@ -0,0 +1,27 @@
#!/bin/bash
#####
##### This file is part of Mail-in-a-Box-LDAP which is released under the
##### terms of the GNU Affero General Public License as published by the
##### Free Software Foundation, either version 3 of the License, or (at
##### your option) any later version. See file LICENSE or go to
##### https://github.com/downtownallday/mailinabox-ldap for full license
##### details.
#####
# Cleanup SSL certificates which expired more than 7 days ago from $STORAGE_ROOT/ssl and move them to $STORAGE_ROOT/ssl.expired
source /etc/mailinabox.conf
shopt -s extglob nullglob
retain_after="$(date --date="7 days ago" +%Y%m%d)"
mkdir -p $STORAGE_ROOT/ssl.expired
ls $STORAGE_ROOT/ssl/*-+([0-9])-+([0-9a-f]).pem 2>/dev/null | while read file
do
pem="$(basename "$file")"
not_valid_after="$(cut -d- -f1 <<< "${pem: -21}")"
if [ "$not_valid_after" -lt "$retain_after" ]; then
mv "$file" "$STORAGE_ROOT/ssl.expired/${pem}"
fi
done