Work-around for ownCloud 8.1.1 upgrade bug and tweaking munin's setup.
v0.13a (August 23, 2015)
------------------------
Note: v0.13 (no 'a', August 19, 2015) was pulled immediately due to an ownCloud bug that prevented upgrades. v0.13a works around that problem.
Mail:
* Outbound mail headers (the Recieved: header) are tweaked to possibly improve deliverability.
* Some MIME messages would hang Roundcube due to a missing package.
* The users permitted to send as an alias can now be different from where an alias forwards to.
DNS:
* The secondary nameservers option in the control panel now accepts more than one nameserver and a special xfr:IP format to specify zone-transfer-only IP addresses.
* A TLSA record is added for HTTPS for DNSSEC-aware clients that support it.
System:
* Backups can now be turned off, or stored in Amazon S3, through new control panel options.
* Munin was not working on machines confused about their hostname and had lots of errors related to PANGO, NTP peers and network interfaces that were not up.
* ownCloud updated to version 8.1.1 (with upgrade work-around), its memcached caching enabled.
* When upgrading, network checks like blocked port 25 are now skipped.
* Tweaks to the intrusion detection rules for IMAP.
* Mail-in-a-Box's setup is a lot quieter, hiding lots of irrelevant messages.
Control panel:
* SSL certificate checks were failing on OVH/OpenVZ servers due to missing /dev/stdin.
* Improve the sort order of the domains in the status checks.
* Some links in the control panel were only working in Chrome.
* let munin choose what it wants to monitor, it may need an update periodically
* deactivate monitoring of NTP peers (this makes no sense)
* deactivate monitoring of network interfaces that aren't up (also makes no sense)
* we get errors in munin-node.log if we don't create /var/lib/munin-node/plugin-state/
The antispam-plugin configuration is not ignoring deleted email (mail sent to a trash folder). This causes a big problem because if someone decides to "clean up" their Spam folder by deleting all of the emails, then sa-learn --ham is run on what is usually correctly caught spam messages. This causes big problems with the accuracy of the bayes scoring! It should really only be learning ham if someone drags the email to a non-Trash folder.
I ran "sa-learn --dbpath /home/user-data/mail/spamassassin --dump magic" before these changes and saw the nham counter increment with every message deleted from Spam. With this new config sa-learn --ham is not run when email is deleted from Spam so the bayes database is not incorrectly trained.
v0.13 (August 19, 2015)
-----------------------
Mail:
* Outbound mail headers (the Recieved: header) are tweaked to possibly improve deliverability.
* Some MIME messages would hang Roundcube due to a missing package.
* The users permitted to send as an alias can now be different from where an alias forwards to.
DNS:
* The secondary nameservers option in the control panel now accepts more than one nameserver and a special xfr:IP format to specify zone-transfer-only IP addresses.
* A TLSA record is added for HTTPS for DNSSEC-aware clients that support it.
System:
* Backups can now be turned off, or stored in Amazon S3, through new control panel options.
* Munin was not working on machines confused about their hostname.
* ownCloud updated to version 8.1.1, its memcached caching enabled.
* When upgrading, network checks like blocked port 25 are now skipped.
* Tweaks to the intrusion detection rules for IMAP.
* Mail-in-a-Box's setup is a lot quieter, hiding lots of irrelevant messages.
Control panel:
* SSL certificate checks were failing on OVH/OpenVZ servers due to missing /dev/stdin.
* Improve the sort order of the domains in the status checks.
* Some links in the control panel were only working in Chrome.
Contacts and Calendar are now versioned separately from ownCloud core.
Contacts appears to be in a v0.4.x.x series. This is the hash of v0.4.0.0.
Per https://github.com/owncloud/calendar/issues/892, Calendar is in a 0.7.x series. This is the hash for 0.7.2.
Backup location and maximum age can now be configured in the admin panel.
For now only S3 is supported, but adding other duplicity supported backends should be straightforward.
Some users report munin is broken because munin and munin-node disagree about the name of the machine. I think this occurs if hostname (used by munin-node) reports a different name than PRIMARY_HOSTNAME (which we put in the munin config).
Hard-code PRIMARY_HOSTNAME in munin-node.conf.
Fixes#474.
See https://discourse.mailinabox.email/t/404-not-found-on-admin-munin/623/24.
This is an extension of #427. Building on that change it adds support in the
aliases table for flagging aliases as:
1. Applicable to inbound and outbound mail.
2. Applicable to inbound mail only.
3. Applicable to outbound mail only.
4. Disabled.
The aliases UI is also updated to allow administrators to set the direction of
each alias.
Using this extra information, the sqlite queries executed by Postfix are
updated so only the relevant alias types are checked.
The goal and result of this change is that outbound-only catch-all aliases can
now be defined (in fact catch-all aliases of any type can be defined).
This allow us to continue supporting relaying as described at
https://mailinabox.email/advanced-configuration.html#relay
without requiring that administrators either create regular aliases for each
outbound *relay* address, or that they create a catch-all alias and then face a
flood of spam.
I have tested the code as it is in this commit and fixed every issue I found,
so in that regard the change is complete. However I see room for improvement
in terms of updating terminology to make the UI etc. easier to understand.
I'll make those changes as subsequent commits so that this tested checkpoint is
not lost, but also so they can be rejected independently of the actual change
if not wanted.
remove live dependency on Sourceforge
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJVq5kxAAoJELkgQfTBC92B9uMIALcrGjq7weaL3qRHYRoeVs5C
/Ov1Lg9QY7PGRl3HtBmFvw50E3coxFCFBfEycK0D9Rue6xF2PHyg8n0DvX5Q2wSD
A9EWAv27ZPoup8/ggv970lTZSpJzseJs1Km0QeOaapfgzPFFtDDwUbkV8sHQxXi4
KCFzmlE72rmvsley/u3IlS/dCb07QdLhdIa/ZJYxSIMJdvMqj0enefBOELoeomYC
ZoNzzzB08eCiyTVd6BTFPBz6CWI6yW203JWoQsSjaz9qEB/N6m9u/PrHBT8VPIRM
Q/a4gn598eAzcGEjub3ZYmJlnbBSlhvczfljmYgNcgizy/SwByaA1AaAemdwI5s=
=2FnK
-----END PGP SIGNATURE-----
Merge tag 'v0.12c'
v0.12c
remove live dependency on Sourceforge
everything was already on master
--------------------
This is a minor update to v0.11, which was a major update. Please read v0.11's advisories.
* The administrator@ alias was incorrectly created starting with v0.11. If your first install was v0.11, check that the administrator@ alias forwards mail to you.
* Intrusion detection rules (fail2ban) are relaxed (i.e. less is blocked).
* SSL certificates could not be installed for the new automatic 'www.' redirect domains.
* PHP's default character encoding is changed from no default to UTF8. The effect of this change is unclear but should prevent possible future text conversion issues.
* User-installed SSL private keys in the BEGIN PRIVATE KEY format were not accepted.
* SSL certificates with SAN domains with IDNA encoding were broken in v0.11.
* Some IDNA functionality was using IDNA 2003 rather than IDNA 2008.
Modify outgoing_mail_header_filters and mail-postfix.sh
files to result in the primary hostname, and the public
ip of the server showing in the first mail header route
instead of unknown and 127.0.0.1. This could help lower
the spam score of mail sent from your server to some
public mail services.
---------------------
Advisories:
* Users can no longer spoof arbitrary email addresses in outbound mail. When sending mail, the email address configured in your mail client must match the SMTP login username being used, or the email address must be an alias with the SMTP login username listed as one of the alias's targets.
* This update replaces your DKIM signing key with a stronger key. Because of DNS caching/propagation, mail sent within a few hours after this update could be marked as spam by recipients. If you use External DNS, you will need to update your DNS records.
* The box will now install software from a new Mail-in-a-Box PPA on Launchpad.net, where we are distributing two of our own packages: a patched postgrey and dovecot-lucene.
Mail:
* Greylisting will now let some reputable senders pass through immediately.
* Searching mail (via IMAP) will now be much faster using the dovecot lucene full text search plugin.
* Users can no longer spoof arbitrary email addresses in outbound mail (see above).
* Fix for deleting admin@ and postmaster@ addresses.
* Roundcube is updated to version 1.1.2, plugins updated.
* Exchange/ActiveSync autoconfiguration was not working on all devices (e.g. iPhone) because of a case-sensitive URL.
* The DKIM signing key has been increased to 2048 bits, from 1024, replacing the existing key.
Web:
* 'www' subdomains now automatically redirect to their parent domain (but you'll need to install an SSL certificate).
* OCSP no longer uses Google Public DNS.
* The installed PHP version is no longer exposed through HTTP response headers, for better security.
DNS:
* Default IPv6 AAAA records were missing since version 0.09.
Control panel:
* Resetting a user's password now forces them to log in again everywhere.
* Status checks were not working if an ssh server was not installed.
* SSL certificate validation now uses the Python cryptography module in some places where openssl was used.
* There is a new tab to show the installed version of Mail-in-a-Box and to fetch the latest released version.
System:
* The munin system monitoring tool is now installed and accessible at /admin/munin.
* ownCloud updated to version 8.0.4. The ownCloud installation step now is reslient to download problems. The ownCloud configuration file is now stored in STORAGE_ROOT to fix loss of data when moving STORAGE_ROOT to a new machine.
* The setup scripts now run `apt-get update` prior to installing anything to ensure the apt database is in sync with the packages actually available.
* Add a migration to delete any existing DKIM key so that existing machines get a fresh 2048-bit key. (Sadly we don't support key rotation so the change is immediate.)
* Because the DNS record for a 2048-bit key is so much longer, the way we read OpenDKIM's DNS record text file had to be modified to combine an arbitrary number of TXT record quoted ("...") strings.
* When writing out the TXT record value, the string must be split into quoted ("...") strings with a maximum length of 255 bytes each, per the DNS spec.
* Added a changelog entry.
Currently MiaB creates 1024 bit keys which is seen as a minimum standard
by several providers such as Google who already uses a 2048 bit key.
Increasing the keysize beyond 2048 is an issue as it often goes beyond
supported DNS record sizes.
If the downloaded file doesn't pass hash verification, the script exits and leaves a broken system
Just make hash verification before moving owncloud directory
* Use `cryptography` instead of parsing openssl's output.
* When checking if we can reuse the primary domain certificate or a www-parent-domain certificate for a domain, avoid shelling out to openssl entirely.
* SMTP Submission (port 587) began offering the insecure SSLv3 protocol due to a misconfiguration in the previous version.
* Roundcube now allows persistent logins using Roundcube-Persistent-Login-Plugin.
* ownCloud is updated to version 8.0.3.
* SPF records for non-mail domains were tightened.
* The minimum greylisting delay has been reduced from 5 minutes to 3 minutes.
* Users and aliases weren't working if they were entered with any uppercase letters. Now only lowercase is allowed.
* After installing an SSL certificate from the control panel, the page wasn't being refreshed.
* Backups broke if the box's hostname was changed after installation.
* Dotfiles (i.e. .svn) stored in ownCloud Files were not accessible from ownCloud's mobile/desktop clients.
* Fix broken install on OVH VPS's.
A lot of legit mail servers try again between 200 and 285 seconds, then
3 hours later. Why? RFC is not strict about retry timer so postfix and
other MTA have their own intervals. To fix the problem of receiving
these e-mail really latter, I reduced the delay of postgrey to
180 seconds (default is 300 seconds).
* install the munin-node package
* don't install munin-plugins-extra (if the user wants it they can add it)
* expose the munin www directory via the management daemon so that it can handle authorization, rather than manintaining a separate password file
The submission port began offering SSLv3.
With `encrypt`, the smtpd_tls_protocols option is ignored and smtpd_tls_mandatory_protocols must be set instead.
see e39b777abc
=====
May 8, 2015
Mail:
* Spam checking is now performed on messages larger than the previous limit of 64KB.
* POP3S is now enabled (port 995).
* Roundcube is updated to version 1.1.1.
* Minor security improvements (more mail headers with user agent info are anonymized; crypto settings were tightened).
ownCloud:
* Downloading files you uploaded to ownCloud broke because of a change in ownCloud 8.
DNS:
* Internationalized Domain Names (IDNs) should now work in email. If you had custom DNS or custom web settings for internationalized domains, check that they are still working.
* It is now possible to set multiple TXT and other types of records on the same domain in the control panel.
* The custom DNS API was completely rewritten to support setting multiple records of the same type on a domain. Any existing client code using the DNS API will have to be rewritten. (Existing code will just get 404s back.)
* On some systems the `nsd` service failed to start if network inferfaces were not ready.
System / Control Panel:
* In order to guard against misconfiguration that can lead to domain control validation hijacking, email addresses that begin with admin, administrator, postmaster, hostmaster, and webmaster can no longer be used for (new) mail user accounts, and aliases for these addresses may direct mail only to the box's administrator(s).
* Backups now use duplicity's built-in gpg symmetric AES256 encryption rather than my home-brewed encryption. Old backups will be incorporated inside the first backup after this update but then deleted from disk (i.e. your backups from the previous few days will be backed up).
* There was a race condition between backups and the new nightly status checks.
* The control panel would sometimes lock up with an unnecessary loading indicator.
* You can no longer delete your own account from the control panel.
Setup:
* All Mail-in-a-Box release tags are now signed on github, instructions for verifying the signature are added to the README, and the integrity of some packages downloaded during setup is now verified against a SHA1 hash stored in the tag itself.
* Bugs in first user account creation were fixed.
Even though SMTP (on port 25) is typically opportunistic and a MitM attack can't be prevented, we may as well only offer ciphers that provide some level of security. If a client is so old or misconfigured that it doesn't support newer ciphers, it should hopefully fall back to a non-TLS connection.
Postfix's default was basically anything goes (anonymous and 40-bit ciphers!). Google's MTA's only offer ciphers at 112 bits at greater, and this change approximates that with Postfix's "medium" setting.
Fixes#371
This disallows aNULL and other bad ciphers in the Postfix submission server.
I missed an option in 45e93f7dcc recommended by the blog post I was reading.
Fixes#389.
Prior to nsd 4.0.1-1ubuntu0.1, we had to create the nsd user before installing the nsd package.
This was our issue #25 (see 4e6037c0e1, c7e1e29d) and I reported it upstream at https://bugs.launchpad.net/ubuntu/+source/nsd/+bug/1311886. The new package was published by Ubuntu on 2015-01-15 so this work-around is no longer needed.
I changed my mind. In 1bf8f1991f I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.
It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).
This partially reverts commit 1bf8f1991f, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.
A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.
Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.
CHANGELOG
=========
v0.08 (April 1, 2015)
---------------------
Mail:
* The Roundcube vacation_sieve plugin by @arodier is now installed to make it easier to set vacation auto-reply messages from within Roundcube.
* Authentication-Results headers for DMARC, added in v0.07, were mistakenly added for outbound mail --- that's now removed.
* The Trash folder is now created automatically for new mail accounts, addressing a Roundcube error.
DNS:
* Custom DNS TXT records were not always working and they can now override the default SPF, DKIM, and DMARC records.
System:
* ownCloud updated to version 8.0.2.
* Brute-force SSH and IMAP login attempts are now prevented by properly configuring fail2ban.
* Status checks are run each night and any changes from night to night are emailed to the box administrator (the first user account).
Control panel:
* The new check that system services are running mistakenly checked that the Dovecot Managesieve service is publicly accessible. Although the service binds to the public network interface we don't open the port in ufw. On some machines it seems that ufw blocks the connection from the status checks (which seems correct) and on some machines (mine) it doesn't, which is why I didn't notice the problem.
* The current backup chain will now try to predict how many days until it is deleted (always at least 3 days after the next full backup).
* The list of aliases that forward to a user are removed from the Mail Users page because when there are many alises it is slow and times-out.
* Some status check errors are turned into warnings, especially those that might not apply if External DNS is used.
I added OpenDMARC's milter in fba4d4702e. But this started
setting Authentication-Results headers on outbound mail with failures. Not sure why it
fails at that point, but it shouldn't be set at all. The failure might cause recipients
to junk the mail. See #358.
This commit removes the milter from the SMTP submission (port 587) listener.
If the migration file is missing but the storage directory exists, assume this is a fresh directory -- don't bother trying to migrate, and do write the migration file with the current migration ID.
cc333b3965 worked for fresh systems, but if the system already had the daemon running the api.key file would already exist and the test would pass to early. Now removing the file first.
fixes#322
/proc/meminfo reports kibibytes. Lower the minimum memory requirement so that 768 MB (not MiB) also is allowed.
Report the detected memory in MB (not KiB), to be clearer.
Fixes#289.
* `git fetch` wasn't done right for shallow clones
* the test for whether mailinabox has already been cloned wasn't looking at the right directory if the script was not run from $HOME
When executed "cat bootstrap.sh | bash", apt-get mangled stdin. The script would terminate at the end of the if block containing apt-get (that seems to be as much as bash read from the pipe) and the remainder of the script was output to the console. This was very weird.
Ensuring that apt-get and git have their stdins redirected from /dev/null seems to fix the problem.
see #224
Aliases have precedence over mail users. A catch-all address would grab mail intended for a mail user and send it elsewhere. This adds some SQL hackery to create dummy aliases for all mail users.
fixes#200closes#214 another way
nginx/postfix use a new pre-generated dh2048.pem file. dovecot generates the bits on its own.
ssllabs.com reports that TLS_DHE ciphers went from 1024 to 2048 bits as expected. The ECDHE ciphers remain at 256 bits --- no idea what that really means. (This tests nginx only. I haven't tested postfix/dovecot.)
see https://discourse.mailinabox.email/t/fips-ready-for-ssl-dhec-key-exchange/76/3
Create directory in $STORAGE_ROOT for bayes database.
Added --username arg to sa-learn as the user mail does not have permission to edit files in $STORAGE_ROOT. There is probably a better solution to this...
Since it runs before the real setup begins, we must make sure that packages are installed.
Also removing bind9-host's installation from system.sh. In 189dd6000e I added this so we could use `host`
to aid Docker autoconfiguration. Docker support was since removed but this hadn't gotten removed, which lead me to think it was
normally installed by Ubuntu. It's now installed in `network-checks.sh`.
fixes#180
The user might acquire the sources via some means other than a git clone. On Vagrant, the files come in via Vagrant. So test for git before running `git describe`.
this will always download the latest ownCloud and upgrade if ownCloud install dir exist, this apphroach allows us to keep existing user plugins. currently not checking if currently installed version is equal to the one we're downloading as I couldn't find a proper solution for that
Joshua Tauberer
derekrspencer
Leo Koppelkamm
David Piggott
PortableTech
Hnk Reno
aLeX
Morteza Milani
Toilal
Norman Stanke
Joaquin Bravo
Xoib
Brock Tice
H8H
Stefan Wienert
pierreozoux
Michael Dec
fpgaminer
jmar71n
Bretos
Joel Kåberg
Helmuth Gronewold
Christian Koptein
Stephan Brauer
jkaberg