I changed my mind. In 1bf8f1991f I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.
It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).
This partially reverts commit 1bf8f1991f, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.
A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.
Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.
* For non-ASCII domain names, we will keep the Unicode encoding in our users/aliases table. This is nice for the user and also simplifies things like sorting domain names (using Unicode lexicographic order is good, using ASCII lexicogrpahic order on IDNA is confusing).
* Write nsd config, nsd zone files, nginx config, and SSL CSRs with domains in IDNA-encoded ASCII.
* When checking SSL certificates, treat the CN and SANs as IDNA.
* Since Chrome has an interesting feature of converting Unicode to IDNA in <input type="email"> form fields, we'll also forcibly convert IDNA to Unicode in the domain part of email addresses before saving email addresses in the users/aliases tables so that the table is normalized to Unicode.
* Don't allow non-ASCII characters in user account email addresses. Dovecot gets confused when querying the Sqlite database (which we observed even for non-word ASCII characters too, so it may not be related to the character encoding).
Relative hostnames have a fall-back lookup with the machine's hostname appended, which makes no sense. Add a period, e.g. "my.hostname.com" => "my.hostname.com.", to prevent that.
This caused false positive Spamhaus checks. Fixes#185.
The problem was that custom records defined for a subdomain where implicit
records are otherwise defined (e.g. A/AAAA records for the root) were ignored.
Though additional_records for a subdomain are processed in the base call to
build_zone (the call for the parent domain), and so custom records that don't
override implicits were working fine, those that overrode implicits were
ignored.
This was because the recursive call to build_zone for the subdomain creates the
implicit records (including A/AAAA records for the root), and so by relying on
the base call to add the additional_records fails because has_rec returned
true.
Adding a subdomain's additional_records in the child call works because has_rec
returns false when testing whether to add an e.g. A/AAAA override for the root,
as the defaults have not yet been added.
The dns_update script didn't generate IPv6 (AAAA) glue records for the name servers.
This caused http://dnscheck.pingdom.com to complain about a mismatch between the glue records reported by the parent name server and mailinabox nsd.
Here's the failing dnscheck output for reference:
> Checking glue for ns1.my.domain.tld (1.2.3.4).
> Child glue for bgwe.eu found: ns1.my.domain.tld (1.2.3.4)
> Checking glue for ns1.my.domain.tld (1234::1).
> Missing glue at child: ns1.my.domain.tld
> Checking glue for ns2.my.domain.tld (1.2.3.4).
> Child glue for bgwe.eu found: ns2.my.domain.tld (1.2.3.4)
> Checking glue for ns2.my.domain.tld (1234::1).
> Missing glue at child: ns2.my.domain.tld
I'm not very familiar with Python and DNS, please verify ;)
Postfix has a tls_security_level called "dane" which uses DNS-Based Authentication of Named Entities (DANE)
to require, if specified in the DNS of the MX host, an encrpyted connection with a known certificate.
This commit adds TLSA records.
Addresses #3
Added support by adding parallel code wherever `$PUBLIC_IP` was used.
Providing an IPv6 address is completely optional.
Playing around on my IPv6-enabled mail server revealed that — before
this change — mailinabox might try to use an IPv6 address as the value
for `$PUBLIC_IP`, which wouldn't work out well.
* adding a Vagrantfile
* in a non-interactive setup like this, create the user's first email account for them
* let the machine auto-detect its IP address using http://icanhazip.com/
* use our own justtesting.email domain to provision a subdomain for users so they can quickly get started
* Created a new Python/flask-based management daemon.
* Moved the mail user management core code from tools/mail.py to the new daemon.
* tools/mail.py is a wrapper around the daemon and can be run as a non-root user.
* Adding a new initscript for the management daemon.
* Moving dns_update.sh to the management daemon, called via curl'ing the daemon's API.
This also now runs the DNS update after mail users and aliases are added/removed,
which sets up new domains' DNS as needed.
