A lot of legit mail servers try again between 200 and 285 seconds, then
3 hours later. Why? RFC is not strict about retry timer so postfix and
other MTA have their own intervals. To fix the problem of receiving
these e-mail really latter, I reduced the delay of postgrey to
180 seconds (default is 300 seconds).
* install the munin-node package
* don't install munin-plugins-extra (if the user wants it they can add it)
* expose the munin www directory via the management daemon so that it can handle authorization, rather than manintaining a separate password file
The submission port began offering SSLv3.
With `encrypt`, the smtpd_tls_protocols option is ignored and smtpd_tls_mandatory_protocols must be set instead.
see e39b777abc
=====
May 8, 2015
Mail:
* Spam checking is now performed on messages larger than the previous limit of 64KB.
* POP3S is now enabled (port 995).
* Roundcube is updated to version 1.1.1.
* Minor security improvements (more mail headers with user agent info are anonymized; crypto settings were tightened).
ownCloud:
* Downloading files you uploaded to ownCloud broke because of a change in ownCloud 8.
DNS:
* Internationalized Domain Names (IDNs) should now work in email. If you had custom DNS or custom web settings for internationalized domains, check that they are still working.
* It is now possible to set multiple TXT and other types of records on the same domain in the control panel.
* The custom DNS API was completely rewritten to support setting multiple records of the same type on a domain. Any existing client code using the DNS API will have to be rewritten. (Existing code will just get 404s back.)
* On some systems the `nsd` service failed to start if network inferfaces were not ready.
System / Control Panel:
* In order to guard against misconfiguration that can lead to domain control validation hijacking, email addresses that begin with admin, administrator, postmaster, hostmaster, and webmaster can no longer be used for (new) mail user accounts, and aliases for these addresses may direct mail only to the box's administrator(s).
* Backups now use duplicity's built-in gpg symmetric AES256 encryption rather than my home-brewed encryption. Old backups will be incorporated inside the first backup after this update but then deleted from disk (i.e. your backups from the previous few days will be backed up).
* There was a race condition between backups and the new nightly status checks.
* The control panel would sometimes lock up with an unnecessary loading indicator.
* You can no longer delete your own account from the control panel.
Setup:
* All Mail-in-a-Box release tags are now signed on github, instructions for verifying the signature are added to the README, and the integrity of some packages downloaded during setup is now verified against a SHA1 hash stored in the tag itself.
* Bugs in first user account creation were fixed.
Even though SMTP (on port 25) is typically opportunistic and a MitM attack can't be prevented, we may as well only offer ciphers that provide some level of security. If a client is so old or misconfigured that it doesn't support newer ciphers, it should hopefully fall back to a non-TLS connection.
Postfix's default was basically anything goes (anonymous and 40-bit ciphers!). Google's MTA's only offer ciphers at 112 bits at greater, and this change approximates that with Postfix's "medium" setting.
Fixes#371
This disallows aNULL and other bad ciphers in the Postfix submission server.
I missed an option in 45e93f7dcc recommended by the blog post I was reading.
Fixes#389.
Prior to nsd 4.0.1-1ubuntu0.1, we had to create the nsd user before installing the nsd package.
This was our issue #25 (see 4e6037c0e1, c7e1e29d) and I reported it upstream at https://bugs.launchpad.net/ubuntu/+source/nsd/+bug/1311886. The new package was published by Ubuntu on 2015-01-15 so this work-around is no longer needed.
I changed my mind. In 1bf8f1991f I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.
It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).
This partially reverts commit 1bf8f1991f, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.
A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.
Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.
CHANGELOG
=========
v0.08 (April 1, 2015)
---------------------
Mail:
* The Roundcube vacation_sieve plugin by @arodier is now installed to make it easier to set vacation auto-reply messages from within Roundcube.
* Authentication-Results headers for DMARC, added in v0.07, were mistakenly added for outbound mail --- that's now removed.
* The Trash folder is now created automatically for new mail accounts, addressing a Roundcube error.
DNS:
* Custom DNS TXT records were not always working and they can now override the default SPF, DKIM, and DMARC records.
System:
* ownCloud updated to version 8.0.2.
* Brute-force SSH and IMAP login attempts are now prevented by properly configuring fail2ban.
* Status checks are run each night and any changes from night to night are emailed to the box administrator (the first user account).
Control panel:
* The new check that system services are running mistakenly checked that the Dovecot Managesieve service is publicly accessible. Although the service binds to the public network interface we don't open the port in ufw. On some machines it seems that ufw blocks the connection from the status checks (which seems correct) and on some machines (mine) it doesn't, which is why I didn't notice the problem.
* The current backup chain will now try to predict how many days until it is deleted (always at least 3 days after the next full backup).
* The list of aliases that forward to a user are removed from the Mail Users page because when there are many alises it is slow and times-out.
* Some status check errors are turned into warnings, especially those that might not apply if External DNS is used.
I added OpenDMARC's milter in fba4d4702e. But this started
setting Authentication-Results headers on outbound mail with failures. Not sure why it
fails at that point, but it shouldn't be set at all. The failure might cause recipients
to junk the mail. See #358.
This commit removes the milter from the SMTP submission (port 587) listener.
If the migration file is missing but the storage directory exists, assume this is a fresh directory -- don't bother trying to migrate, and do write the migration file with the current migration ID.
cc333b3965 worked for fresh systems, but if the system already had the daemon running the api.key file would already exist and the test would pass to early. Now removing the file first.
fixes#322
/proc/meminfo reports kibibytes. Lower the minimum memory requirement so that 768 MB (not MiB) also is allowed.
Report the detected memory in MB (not KiB), to be clearer.
Fixes#289.
* `git fetch` wasn't done right for shallow clones
* the test for whether mailinabox has already been cloned wasn't looking at the right directory if the script was not run from $HOME
When executed "cat bootstrap.sh | bash", apt-get mangled stdin. The script would terminate at the end of the if block containing apt-get (that seems to be as much as bash read from the pipe) and the remainder of the script was output to the console. This was very weird.
Ensuring that apt-get and git have their stdins redirected from /dev/null seems to fix the problem.
see #224
Aliases have precedence over mail users. A catch-all address would grab mail intended for a mail user and send it elsewhere. This adds some SQL hackery to create dummy aliases for all mail users.
fixes#200closes#214 another way
nginx/postfix use a new pre-generated dh2048.pem file. dovecot generates the bits on its own.
ssllabs.com reports that TLS_DHE ciphers went from 1024 to 2048 bits as expected. The ECDHE ciphers remain at 256 bits --- no idea what that really means. (This tests nginx only. I haven't tested postfix/dovecot.)
see https://discourse.mailinabox.email/t/fips-ready-for-ssl-dhec-key-exchange/76/3
Create directory in $STORAGE_ROOT for bayes database.
Added --username arg to sa-learn as the user mail does not have permission to edit files in $STORAGE_ROOT. There is probably a better solution to this...
Since it runs before the real setup begins, we must make sure that packages are installed.
Also removing bind9-host's installation from system.sh. In 189dd6000e I added this so we could use `host`
to aid Docker autoconfiguration. Docker support was since removed but this hadn't gotten removed, which lead me to think it was
normally installed by Ubuntu. It's now installed in `network-checks.sh`.
fixes#180
The user might acquire the sources via some means other than a git clone. On Vagrant, the files come in via Vagrant. So test for git before running `git describe`.
this will always download the latest ownCloud and upgrade if ownCloud install dir exist, this apphroach allows us to keep existing user plugins. currently not checking if currently installed version is equal to the one we're downloading as I couldn't find a proper solution for that
Doesn't seem like 2 minutes is a problem, but 4 minutes seems better. A little less bandwidth, possibly less battery usage (though we don't have evidence that's actually true), and the interval should be shorter than any peer timeouts that might occur due to inactivity
fixes#129
* check that the PUBLIC_IP is not listed in zen.spamhaus.org
* check that the PRIMARY_HOSTNAME is not listed in dbl.spamhaus.org
* check that a connection to Google's MTA is working (i.e. we're not on a residential network that blocks outbound port 25)
Rather than pass `-r /dev/random` to ldns-keygen (it was `-r /dev/urandom`),
don't pass `-r` at all since /dev/random is the default.
Merges branch 'master' of github.com:pysiak/mailinabox
/dev/random should be used for crypto-grade RNG.
To make sure use of /dev/random doesn't stall due to lack of entropy, install haveged which fills the entropy pool with sources such as network traffic, key strokes, etc.
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
modified: setup/dns.sh
modified: setup/system.sh
modified: setup/webmail.sh
This lets roundcube's manageseive plugin do cool things like vacation responses.
Also:
* Run the spam filtering sieve script out of a global sieve file that we'll place in /etc/dovecot. It is no longer necessary to create per-user sieve files for this. Remove them with a new migration. Remove the code that created them.
* Corrects the spam script. Backslashes were double-escaped probably because this script started embedded within the bash script. Not sure how this was working until now.
this adapts work by @h8h in #103
As the scripts keep growing, it's time to split them up to
keep them understandable.
This splits mail.sh into mail-postfix.sh, mail-dovecot.sh,
and mail-users.sh, which has all of the user database-related
configurations shared by Dovecot and Postfix. Also from
spamassassin.sh the core sieve configuration is moved into
mail-dovecot.sh and the virtual transport setting is moved
into mail-postfix.sh.
Also revising one of the sed scripts in mail-dovecot to
not insert a new additional # at the start of a line each
time the script is run.
Intended to be the simplest auth possible: every time the service
starts, a random key is written to `/var/lib/mailinabox/api.key`. In
order to authenticate to the service, the client must pass the contents
of `api.key` in an HTTP basic auth header. In this way, users who do not
have read access to that file are not able to communicate with the
service.
Duplicity will manage the process of creating incremental backups for us.
Although duplicity can both encrypt & copy files to a remote host, I really
don't like PGP and so I don't want to use that.
Instead, we'll back up to a local directory unencrypted, then manually
encrypt the full & incremental backup files. Synchronizing the encrypted
backup directory to a remote host is a TODO.
Testing showed that it may take a few seconds for the default values to
populate. If the help text is shown, “Enter the public IP address…,”
but no prompt is shown, the user may get confused and try to enter the
IP address before mailinabox has had a chance to figure out and display
a suitable default value.
This re-implements part of PR #69 by @mkropat, who wrote:
By default, Postfix adds a Received header — on all mail that you send —
that lists the IP of the device you sent the mail from. This feature is
great if you're a mail provider and you need to debug why one user is
having sending issues. This feature is not so great if you run your own
mail server and you don't want every recipient of every email you send
to know the device and IP you sent the email from.
To limit this filtering to outgoing mail only, we apply the filters just
to the submission port. See these guides [1] [2] for more context.
[1] http://askubuntu.com/a/78168/11259
[2] http://www.void.gr/kargig/blog/2013/11/24/anonymize-headers-in-postfix/
Addresses #3
Added support by adding parallel code wherever `$PUBLIC_IP` was used.
Providing an IPv6 address is completely optional.
Playing around on my IPv6-enabled mail server revealed that — before
this change — mailinabox might try to use an IPv6 address as the value
for `$PUBLIC_IP`, which wouldn't work out well.
* using it to enable the Postfix submission service
* per @mkropat's suggestion in #69, set an option to distinguish submission from regular smpd in syslog by giving submission a new name (doing this here to test that editconf is working right on master.cf)
Postfix, Dovecot, and nginx all read the key file while they're running
as root — before dropping permissions — so no authorization is needed on
the private key file beyond being root-readable.
Default IP+hostname values were incorrect for my VPS provider. I
improved the detection, which should give correct results results for
almost any provider. Specific issues addressed:
- icanhazip.com detection was only enabled in non-interactive mode
- `hostname` is by convention a short (non-fqdn) name in Ubuntu
- `hostname --fqdn` fails if provider does not pouplate `hosts` file
- `hostname -i` fails if provider does not populate `hosts` file
- `curl` without `--fail` will someday return crazy results
when icanhazip.com returns 500 errors or similar
* adding a Vagrantfile
* in a non-interactive setup like this, create the user's first email account for them
* let the machine auto-detect its IP address using http://icanhazip.com/
* use our own justtesting.email domain to provision a subdomain for users so they can quickly get started
* Created a new Python/flask-based management daemon.
* Moved the mail user management core code from tools/mail.py to the new daemon.
* tools/mail.py is a wrapper around the daemon and can be run as a non-root user.
* Adding a new initscript for the management daemon.
* Moving dns_update.sh to the management daemon, called via curl'ing the daemon's API.
This also now runs the DNS update after mail users and aliases are added/removed,
which sets up new domains' DNS as needed.
