The cryptography package has created all sorts of installation trouble over the last few years, probably because of mismatches between OS-installed packages and pip-installed packages. Using a virtualenv for all Python packages used by the management daemon should make sure everything is consistent.
See #1298, see #1264.
* [Issue #1159] Remove any +tag name in email alias before checking privileges
* Move priprivileged email check after the conversion to unicode so only IDNA serves as input
This is an extension of #427. Building on that change it adds support in the
aliases table for flagging aliases as:
1. Applicable to inbound and outbound mail.
2. Applicable to inbound mail only.
3. Applicable to outbound mail only.
4. Disabled.
The aliases UI is also updated to allow administrators to set the direction of
each alias.
Using this extra information, the sqlite queries executed by Postfix are
updated so only the relevant alias types are checked.
The goal and result of this change is that outbound-only catch-all aliases can
now be defined (in fact catch-all aliases of any type can be defined).
This allow us to continue supporting relaying as described at
https://mailinabox.email/advanced-configuration.html#relay
without requiring that administrators either create regular aliases for each
outbound *relay* address, or that they create a catch-all alias and then face a
flood of spam.
I have tested the code as it is in this commit and fixed every issue I found,
so in that regard the change is complete. However I see room for improvement
in terms of updating terminology to make the UI etc. easier to understand.
I'll make those changes as subsequent commits so that this tested checkpoint is
not lost, but also so they can be rejected independently of the actual change
if not wanted.
Unfortunately our users/aliases database is case sensitive. (Perhaps I should have defined the columns with COLLATE NOCASE, see https://www.sqlite.org/datatype3.html.) Postfix always queries the tables in lowecase, so mail delivery would fail if a user or alias were defined with any capital letters. It would have also been possible to add multiple euqivalent addresses into the database with different case.
This commit rejects new mail users that have capital letters and forces new aliases to lowecase. I prefer to reject rather than casefold user accounts so that the login credentials the user gave are exactly what goes into the database.
https://discourse.mailinabox.email/t/recipient-address-rejected-user-unknown-in-virtual-mailbox-table/512/4
This also includes fixes for a typo and some whitespace inconsistencies in
mailconfig.py. In fact the capitalisation change and those fixes are the
remnants of a patch I had been running that changed the default aliases - it
was through developing it that I found the issues.
(I wanted to bring the number of patches I apply before deploying to zero and
in the case of this one I've come to view the way MIAB already is as superior,
so I've undone the core of my patch and these tiny issues are all that remain).
I changed my mind. In 1bf8f1991f I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.
It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).
This partially reverts commit 1bf8f1991f, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.
A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.
Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.
* For non-ASCII domain names, we will keep the Unicode encoding in our users/aliases table. This is nice for the user and also simplifies things like sorting domain names (using Unicode lexicographic order is good, using ASCII lexicogrpahic order on IDNA is confusing).
* Write nsd config, nsd zone files, nginx config, and SSL CSRs with domains in IDNA-encoded ASCII.
* When checking SSL certificates, treat the CN and SANs as IDNA.
* Since Chrome has an interesting feature of converting Unicode to IDNA in <input type="email"> form fields, we'll also forcibly convert IDNA to Unicode in the domain part of email addresses before saving email addresses in the users/aliases tables so that the table is normalized to Unicode.
* Don't allow non-ASCII characters in user account email addresses. Dovecot gets confused when querying the Sqlite database (which we observed even for non-word ASCII characters too, so it may not be related to the character encoding).
This seemed to already be technically supported but the validation is now stricter and the admin is more helpful:
* Postfix seems to allow @domain.tld as an alias destination address but only if it is the only destination address (see the virtual man page).
* Allow @domain.tld if it is the whole destination address string.
* Otherwise, do not allow email addresses without local parts in the destination.
* In the admin, add a third tab for making it clear how to add a domain alias.
closes#265
In the event the first user is an address that we'd normally create as an alias,
we'd generate a loop from the alias to the administrative alias to the first user
account (which was the alias again).
hopefully fixes#186
This reverts pull request #105 from jonessen96/master (84d2023f94) which was incorrect because it lost the "+" in DOT_ATOM_TEXT and so was not accepting any email addresses.
Am taking the opportunity to make the code cleaner while I'm here.
This lets roundcube's manageseive plugin do cool things like vacation responses.
Also:
* Run the spam filtering sieve script out of a global sieve file that we'll place in /etc/dovecot. It is no longer necessary to create per-user sieve files for this. Remove them with a new migration. Remove the code that created them.
* Corrects the spam script. Backslashes were double-escaped probably because this script started embedded within the bash script. Not sure how this was working until now.
this adapts work by @h8h in #103
* for local mail users, also disallows periods at the beginning or end of the local or domain parts
* Dovecot gets confused if the string contains any unusual characters, so local mail users are restricted to a narrow regex
* for mail aliases Postfix is not confused so use a regex based on RFC 2822
* Created a new Python/flask-based management daemon.
* Moved the mail user management core code from tools/mail.py to the new daemon.
* tools/mail.py is a wrapper around the daemon and can be run as a non-root user.
* Adding a new initscript for the management daemon.
* Moving dns_update.sh to the management daemon, called via curl'ing the daemon's API.
This also now runs the DNS update after mail users and aliases are added/removed,
which sets up new domains' DNS as needed.
Joshua Tauberer
Git Repository
Dominik Murzynowski
yodax
Ariejan de Vroom
Peter Timofejew
David Piggott
Jonas Platte