In the earlier commit, I added a Dovecot userdb lookup. Without a userdb lookup, Dovecot would use the password db for user lookups. With a userdb lookup we can support iterating over users.
But I forgot the WHERE clause in the query, resulting in every incoming message being accepted if the user database contained any users at all. Since the mailbox path template is the same for all users, mail was delivered correctly except that mail that should have been rejected was delivered too.
On some machines localhost is defined as something other than 127.0.0.1, and if we mix "127.0.0.1" and "localhost" then some connections won't be to to the address a service is actually running on.
This was the case with DKIM: It was running on "localhost" but Postfix was connecting to it at 127.0.0.1. (https://discourse.mailinabox.email/t/opendkim-is-not-running-port-8891/1188/12.)
I suppose "localhost" could be an alias to an IPv6 address? We don't really want local services binding on IPv6, so use "127.0.0.1" to be explicit and don't use "localhost" to be sure we get an IPv4 address.
Fixes#797
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJW/mJqAAoJELkgQfTBC92B/F8H/2s6wKhzzeoqkhLU2nvYJh0B
Q1d0SbtdQWIWrTQbcjIR3aGYwJzJ+HC7rylrwS4lB2ugpJBA0MnfD+ktwbe/EyDa
pN6WLlmnXyAw28//ubq0FQqC8Gawsj4WMfmSEw/XuDShik8XJmU7QUEnewClJ7So
ko4eVp9KL8MU3Rj/DebhyoW0EjpB/qrJvLSqtj4KCxKYES9J8nUVBFVRDL48yNx4
2KTIjqreGZmtW0/wxPnganMeV6DZn3B6vBmqOYYvw7bf6r/cY0ZkNK/ENlo+ntJD
3jFKki4TJChhGVWH5T4Tw2bys4Cua1+SA3cleNRH1rYSvRWyOCwK+LS4YBJHYp4=
=umMp
-----END PGP SIGNATURE-----
merge hotfix release tag 'v0.17c' into master
The hotfixes were all already applied to master in original PRs. This merge merely brings over the CHANGELOG and the updated install instructions (v0.17b=>v0.17c), including to bootstrap.sh which is what triggers v0.17c being the latest release.
----------------------
ownCloud moved their source code to a new location, breaking our installation script.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJW1YnLAAoJELkgQfTBC92BZg8H/RVfJRrhkM6B0UocYOmp2LQB
u6b/KxSM5JbBq9BTyQTAkQ0Dj6caWpBwAATGgxvyAYt0Du/QTx/ycfgenkP2DWcI
UYm+AkV2Eg9BiXpFqG33vJ/aWjATB+S3XM+77jMbeMb4kK+PfOHguvmxnsS4x8HY
h+unkXPy8fr1rATaOGg32fTZrKkVWAriHFfJqy3kkoTHS+9fj7MVGbTBPntPaC46
/42IsCFiQsNdz6QeHmXzFnP6AuqmkbcN9dpsFe36qfSfwVhNqC776lxxP+r1nK3p
AWs/q5VQ4V1ght3MzDch9dOlmGuqBVB1WkGdQVV7RemYrdNi5S1h+dXXUDiNXY8=
=C7LW
-----END PGP SIGNATURE-----
merge point release v0.17b
ownCloud moved their source code to a new location, breaking our installation script.
https://discourse.mailinabox.email/t/status-check-emails-empty-after-upgrading-to-v0-16/1082/3
A user on that thread suggests an alternate solution, adding `PYTHONIOENCODING=utf-8` to `/etc/environment`. Python docs say that affects stdin/out/err. But we also use these environment variables elsewhere to ensure that config files we read/write are opened with UTF8 too. Maybe all that can be simplified too.
The default timeout for Postfix's maximal_queue_lifetime and bounce_queue_lifetime is 5 days. This is way too long if you expect someone to have an answer and after 5 days you'll get the message that it's not delivered. This disrupts communication. It would be more responsive if the user got the 'can't deliver' error after 24 hours.
Sending mail through Exchange/ActiveSync (Z-Push) had been broken since v0.14. This is now fixed.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJWkQ9rAAoJELkgQfTBC92BPjQH/ibFBZHgma7C53Q4X9iPfUk2
dPK75rNBx06d6yW4LNAYuWVnNO1Mb0khb2k2UPwg4noImYWqsNS9hXih7C6oPHiK
Szz4ubCc5MEqnmhxiNkzLdIBvsyKmz8IfmCl+LCXu8uk0Fb+pB6zbSdAGxjtaSPL
itCwz8+ApTC4bl1CoNYPn2zudHmHNeC7L6INYdb+xbtc/Tz6mO/xMaBVPDiKeq9P
LqLTOXiJNENz7vKSZytlWGOTdtSTZqwd7JBXuBg0QFz5C9yg8EV4LWB9wOm5aTIf
Fol3WK5ZHA5YeihZKmZSjz9+p4iwv5hqR5osKL2n46LeVHyafESl+QnZSXRlDjE=
=i3ei
-----END PGP SIGNATURE-----
Merge release branch v0.15a
v0.15a
Sending mail through Exchange/ActiveSync (Z-Push) had been broken since v0.14. This is now fixed.
-----------------------
Mail:
* Updated Roundcube to version 1.1.3.
* Auto-create aliases for abuse@, as required by RFC2142.
* The DANE TLSA record is changed to use the certificate subject public key rather than the whole certificate, which means the record remains valid after certificate changes (so long as the private key remains the same, which it does for us).
Control panel:
* When IPv6 is enabled, check that system services are accessible over IPv6 too, that the box's hostname resolves over IPv6, and that reverse DNS is setup correctly for IPv6.
* Explanatory text for setting up secondary nameserver is added/fixed.
* DNS checks now have a timeout in case a DNS server is not responding, so the checks don't stall indefinitely.
* Better messages if external DNS is used and, weirdly, custom secondary nameservers are set.
* Add POP to the mail client settings documentation.
* The box's IP address is added to the fail2ban whitelist so that the status checks don't trigger the machine banning itself, which results in the status checks showing services down even though they are running.
* For SSL certificates, rather than asking you what country you are in during setup, ask at the time a CSR is generated. The default system self-signed certificate now omits a country in the subject (it was never needed). The CSR_COUNTRY Mail-in-a-Box setting is dropped entirely.
System:
* Nightly backups and system status checks are now moved to 3am in the system's timezone.
* fail2ban's recidive jail is now active, which guards against persistent brute force login attacks over long periods of time.
* Setup (first run only) now asks for your timezone to set the system time.
* The Exchange/ActiveSync server is now taken offline during nightly backups (along with SMTP and IMAP).
* The machine's random number generator (/dev/urandom) is now seeded with Ubuntu Pollinate and a blocking read on /dev/random.
* DNSSEC key generation during install now uses /dev/urandom (instead of /dev/random), which is faster.
* The $STORAGE_ROOT/ssl directory is flattened by a migration script and the system SSL certificate path is now a symlink to the actual certificate.
* If ownCloud sends out email, it will use the box's administrative address now (admin@yourboxname).
* Z-Push (Exchange/ActiveSync) logs now exclude warnings and are now rotated to save disk space.
* Fix pip command that might have not installed all necessary Python packages.
* The control panel and backup would not work on Google Compute Engine because GCE installs a conflicting boto package.
* Added a new command `management/backup.py --restore` to restore files from a backup to a target directory (command line arguments are passed to `duplicity restore`).
v0.14 (November 4, 2015)
------------------------
Mail:
* Spamassassin's network-based tests (Pyzor, others) and DKIM tests are now enabled. (Pyzor had always been installed but was not active due to a misconfiguration.)
* Moving spam out of the Spam folder and into Trash would incorrectly train Spamassassin that those messages were not spam.
* Automatically create the Sent and Archive folders for new users.
* The HTML5_Notifier plugin for Roundcube is now included, which when turned on in Roundcube settings provides desktop notifications for new mail.
* The Exchange/ActiveSync backend Z-Push has been updated to fix a problem with CC'd emails not being sent to the CC recipients.
Calender/Contacts:
* CalDAV/CardDAV and Exchange/ActiveSync for calendar/contacts wasn't working in some network configurations.
Web:
* When a new domain is added to the box, rather than applying a new self-signed certificate for that domain, the SSL certificate for the box's primary hostname will be used instead.
* If a custom DNS record is set on a domain or 'www'+domain, web would not be served for that domain. If the custom DNS record is just the box's IP address, that's a configuration mistake, but allow it and let web continue to be served.
* Accommodate really long domain names by increasing an nginx setting.
Control panel:
* Added an option to check for new Mail-in-a-Box versions within status checks. It is off by default so that boxes don't "phone home" without permission.
* Added a random password generator on the users page to simplify creating new accounts.
* When S3 backup credentials are set, the credentials are now no longer ever sent back from the box to the client, for better security.
* Fixed the jumpiness when a modal is displayed.
* Focus is put into the login form fields when the login form is displayed.
* Status checks now include a warning if a custom DNS record has been set on a domain that would normally serve web and as a result that domain no longer is serving web.
* Status checks now check that secondary nameservers, if specified, are actually serving the domains.
* Some errors in the control panel when there is invalid data in the database or an improperly named archived user account have been suppressed.
* Added subresource integrity attributes to all remotely-sourced resources (i.e. via CDNs) to guard against CDNs being used as an attack vector.
System:
* Tweaks to fail2ban settings.
* Fixed a spurrious warning while installing munin.
Seems like ownCloud 8.1.1 now doesn't play nice with trusted_domains. Whatever is put in ahead of time gets reset to an array containing 'localhost' only, probably because we invoke autoconfiguration from the command line where it doesn't know the hostname it's being accessed from. We now set this value after running autoconfig.
This has the added benefit of also fixing the problem that if PRIMARY_HOSTNAME changes, trusted_domains wasn't updated. Now it is. Fixes#503.
See #514.
Work-around for ownCloud 8.1.1 upgrade bug and tweaking munin's setup.
v0.13a (August 23, 2015)
------------------------
Note: v0.13 (no 'a', August 19, 2015) was pulled immediately due to an ownCloud bug that prevented upgrades. v0.13a works around that problem.
Mail:
* Outbound mail headers (the Recieved: header) are tweaked to possibly improve deliverability.
* Some MIME messages would hang Roundcube due to a missing package.
* The users permitted to send as an alias can now be different from where an alias forwards to.
DNS:
* The secondary nameservers option in the control panel now accepts more than one nameserver and a special xfr:IP format to specify zone-transfer-only IP addresses.
* A TLSA record is added for HTTPS for DNSSEC-aware clients that support it.
System:
* Backups can now be turned off, or stored in Amazon S3, through new control panel options.
* Munin was not working on machines confused about their hostname and had lots of errors related to PANGO, NTP peers and network interfaces that were not up.
* ownCloud updated to version 8.1.1 (with upgrade work-around), its memcached caching enabled.
* When upgrading, network checks like blocked port 25 are now skipped.
* Tweaks to the intrusion detection rules for IMAP.
* Mail-in-a-Box's setup is a lot quieter, hiding lots of irrelevant messages.
Control panel:
* SSL certificate checks were failing on OVH/OpenVZ servers due to missing /dev/stdin.
* Improve the sort order of the domains in the status checks.
* Some links in the control panel were only working in Chrome.
* let munin choose what it wants to monitor, it may need an update periodically
* deactivate monitoring of NTP peers (this makes no sense)
* deactivate monitoring of network interfaces that aren't up (also makes no sense)
* we get errors in munin-node.log if we don't create /var/lib/munin-node/plugin-state/
The antispam-plugin configuration is not ignoring deleted email (mail sent to a trash folder). This causes a big problem because if someone decides to "clean up" their Spam folder by deleting all of the emails, then sa-learn --ham is run on what is usually correctly caught spam messages. This causes big problems with the accuracy of the bayes scoring! It should really only be learning ham if someone drags the email to a non-Trash folder.
I ran "sa-learn --dbpath /home/user-data/mail/spamassassin --dump magic" before these changes and saw the nham counter increment with every message deleted from Spam. With this new config sa-learn --ham is not run when email is deleted from Spam so the bayes database is not incorrectly trained.
v0.13 (August 19, 2015)
-----------------------
Mail:
* Outbound mail headers (the Recieved: header) are tweaked to possibly improve deliverability.
* Some MIME messages would hang Roundcube due to a missing package.
* The users permitted to send as an alias can now be different from where an alias forwards to.
DNS:
* The secondary nameservers option in the control panel now accepts more than one nameserver and a special xfr:IP format to specify zone-transfer-only IP addresses.
* A TLSA record is added for HTTPS for DNSSEC-aware clients that support it.
System:
* Backups can now be turned off, or stored in Amazon S3, through new control panel options.
* Munin was not working on machines confused about their hostname.
* ownCloud updated to version 8.1.1, its memcached caching enabled.
* When upgrading, network checks like blocked port 25 are now skipped.
* Tweaks to the intrusion detection rules for IMAP.
* Mail-in-a-Box's setup is a lot quieter, hiding lots of irrelevant messages.
Control panel:
* SSL certificate checks were failing on OVH/OpenVZ servers due to missing /dev/stdin.
* Improve the sort order of the domains in the status checks.
* Some links in the control panel were only working in Chrome.
Contacts and Calendar are now versioned separately from ownCloud core.
Contacts appears to be in a v0.4.x.x series. This is the hash of v0.4.0.0.
Per https://github.com/owncloud/calendar/issues/892, Calendar is in a 0.7.x series. This is the hash for 0.7.2.
Backup location and maximum age can now be configured in the admin panel.
For now only S3 is supported, but adding other duplicity supported backends should be straightforward.
Some users report munin is broken because munin and munin-node disagree about the name of the machine. I think this occurs if hostname (used by munin-node) reports a different name than PRIMARY_HOSTNAME (which we put in the munin config).
Hard-code PRIMARY_HOSTNAME in munin-node.conf.
Fixes#474.
See https://discourse.mailinabox.email/t/404-not-found-on-admin-munin/623/24.
This is an extension of #427. Building on that change it adds support in the
aliases table for flagging aliases as:
1. Applicable to inbound and outbound mail.
2. Applicable to inbound mail only.
3. Applicable to outbound mail only.
4. Disabled.
The aliases UI is also updated to allow administrators to set the direction of
each alias.
Using this extra information, the sqlite queries executed by Postfix are
updated so only the relevant alias types are checked.
The goal and result of this change is that outbound-only catch-all aliases can
now be defined (in fact catch-all aliases of any type can be defined).
This allow us to continue supporting relaying as described at
https://mailinabox.email/advanced-configuration.html#relay
without requiring that administrators either create regular aliases for each
outbound *relay* address, or that they create a catch-all alias and then face a
flood of spam.
I have tested the code as it is in this commit and fixed every issue I found,
so in that regard the change is complete. However I see room for improvement
in terms of updating terminology to make the UI etc. easier to understand.
I'll make those changes as subsequent commits so that this tested checkpoint is
not lost, but also so they can be rejected independently of the actual change
if not wanted.
remove live dependency on Sourceforge
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJVq5kxAAoJELkgQfTBC92B9uMIALcrGjq7weaL3qRHYRoeVs5C
/Ov1Lg9QY7PGRl3HtBmFvw50E3coxFCFBfEycK0D9Rue6xF2PHyg8n0DvX5Q2wSD
A9EWAv27ZPoup8/ggv970lTZSpJzseJs1Km0QeOaapfgzPFFtDDwUbkV8sHQxXi4
KCFzmlE72rmvsley/u3IlS/dCb07QdLhdIa/ZJYxSIMJdvMqj0enefBOELoeomYC
ZoNzzzB08eCiyTVd6BTFPBz6CWI6yW203JWoQsSjaz9qEB/N6m9u/PrHBT8VPIRM
Q/a4gn598eAzcGEjub3ZYmJlnbBSlhvczfljmYgNcgizy/SwByaA1AaAemdwI5s=
=2FnK
-----END PGP SIGNATURE-----
Merge tag 'v0.12c'
v0.12c
remove live dependency on Sourceforge
everything was already on master
--------------------
This is a minor update to v0.11, which was a major update. Please read v0.11's advisories.
* The administrator@ alias was incorrectly created starting with v0.11. If your first install was v0.11, check that the administrator@ alias forwards mail to you.
* Intrusion detection rules (fail2ban) are relaxed (i.e. less is blocked).
* SSL certificates could not be installed for the new automatic 'www.' redirect domains.
* PHP's default character encoding is changed from no default to UTF8. The effect of this change is unclear but should prevent possible future text conversion issues.
* User-installed SSL private keys in the BEGIN PRIVATE KEY format were not accepted.
* SSL certificates with SAN domains with IDNA encoding were broken in v0.11.
* Some IDNA functionality was using IDNA 2003 rather than IDNA 2008.
Modify outgoing_mail_header_filters and mail-postfix.sh
files to result in the primary hostname, and the public
ip of the server showing in the first mail header route
instead of unknown and 127.0.0.1. This could help lower
the spam score of mail sent from your server to some
public mail services.
---------------------
Advisories:
* Users can no longer spoof arbitrary email addresses in outbound mail. When sending mail, the email address configured in your mail client must match the SMTP login username being used, or the email address must be an alias with the SMTP login username listed as one of the alias's targets.
* This update replaces your DKIM signing key with a stronger key. Because of DNS caching/propagation, mail sent within a few hours after this update could be marked as spam by recipients. If you use External DNS, you will need to update your DNS records.
* The box will now install software from a new Mail-in-a-Box PPA on Launchpad.net, where we are distributing two of our own packages: a patched postgrey and dovecot-lucene.
Mail:
* Greylisting will now let some reputable senders pass through immediately.
* Searching mail (via IMAP) will now be much faster using the dovecot lucene full text search plugin.
* Users can no longer spoof arbitrary email addresses in outbound mail (see above).
* Fix for deleting admin@ and postmaster@ addresses.
* Roundcube is updated to version 1.1.2, plugins updated.
* Exchange/ActiveSync autoconfiguration was not working on all devices (e.g. iPhone) because of a case-sensitive URL.
* The DKIM signing key has been increased to 2048 bits, from 1024, replacing the existing key.
Web:
* 'www' subdomains now automatically redirect to their parent domain (but you'll need to install an SSL certificate).
* OCSP no longer uses Google Public DNS.
* The installed PHP version is no longer exposed through HTTP response headers, for better security.
DNS:
* Default IPv6 AAAA records were missing since version 0.09.
Control panel:
* Resetting a user's password now forces them to log in again everywhere.
* Status checks were not working if an ssh server was not installed.
* SSL certificate validation now uses the Python cryptography module in some places where openssl was used.
* There is a new tab to show the installed version of Mail-in-a-Box and to fetch the latest released version.
System:
* The munin system monitoring tool is now installed and accessible at /admin/munin.
* ownCloud updated to version 8.0.4. The ownCloud installation step now is reslient to download problems. The ownCloud configuration file is now stored in STORAGE_ROOT to fix loss of data when moving STORAGE_ROOT to a new machine.
* The setup scripts now run `apt-get update` prior to installing anything to ensure the apt database is in sync with the packages actually available.
* Add a migration to delete any existing DKIM key so that existing machines get a fresh 2048-bit key. (Sadly we don't support key rotation so the change is immediate.)
* Because the DNS record for a 2048-bit key is so much longer, the way we read OpenDKIM's DNS record text file had to be modified to combine an arbitrary number of TXT record quoted ("...") strings.
* When writing out the TXT record value, the string must be split into quoted ("...") strings with a maximum length of 255 bytes each, per the DNS spec.
* Added a changelog entry.
Currently MiaB creates 1024 bit keys which is seen as a minimum standard
by several providers such as Google who already uses a 2048 bit key.
Increasing the keysize beyond 2048 is an issue as it often goes beyond
supported DNS record sizes.
If the downloaded file doesn't pass hash verification, the script exits and leaves a broken system
Just make hash verification before moving owncloud directory
* Use `cryptography` instead of parsing openssl's output.
* When checking if we can reuse the primary domain certificate or a www-parent-domain certificate for a domain, avoid shelling out to openssl entirely.
* SMTP Submission (port 587) began offering the insecure SSLv3 protocol due to a misconfiguration in the previous version.
* Roundcube now allows persistent logins using Roundcube-Persistent-Login-Plugin.
* ownCloud is updated to version 8.0.3.
* SPF records for non-mail domains were tightened.
* The minimum greylisting delay has been reduced from 5 minutes to 3 minutes.
* Users and aliases weren't working if they were entered with any uppercase letters. Now only lowercase is allowed.
* After installing an SSL certificate from the control panel, the page wasn't being refreshed.
* Backups broke if the box's hostname was changed after installation.
* Dotfiles (i.e. .svn) stored in ownCloud Files were not accessible from ownCloud's mobile/desktop clients.
* Fix broken install on OVH VPS's.
A lot of legit mail servers try again between 200 and 285 seconds, then
3 hours later. Why? RFC is not strict about retry timer so postfix and
other MTA have their own intervals. To fix the problem of receiving
these e-mail really latter, I reduced the delay of postgrey to
180 seconds (default is 300 seconds).
* install the munin-node package
* don't install munin-plugins-extra (if the user wants it they can add it)
* expose the munin www directory via the management daemon so that it can handle authorization, rather than manintaining a separate password file
