When building the part of the report about the current DS records founded, they are added in the same order as they were received when calling query_dns(), which can differ from run to run. This was making the difflib.SequenceMatcher() method to find the same line removed and added one line later, and sending an Status Checks Change Notice email with the same line added and removed when there was actually no real changes.
Port 465 with "implicit" (i.e. always-on) TLS is a more secure approach than port 587 with explicit (i.e. optional and only on with STARTTLS). Although we reject credentials on port 587 without STARTTLS, by that point credentials have already been sent.
* Stop generating RSASHA1-NSEC3-SHA1 keys on new installs since it is no longer recommended, but preserve the key on existing installs so that we continue to sign zones with existing keys to retain the chain of trust with existing DS records.
* Start generating ECDSAP256SHA256 keys during setup, the current best practice (in addition to RSASHA256 which is also ok). See https://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml#dns-sec-alg-numbers-1 and https://www.cloudflare.com/dns/dnssec/ecdsa-and-dnssec/.
* Sign zones using all available keys rather than choosing just one based on the TLD to enable rotation/migration to the new key and to give the user some options since not every registrar/TLD supports every algorithm.
* Allow a user to drop a key from signing specific domains using DOMAINS= in our key configuration file. Signing the zones with extraneous keys may increase the size of DNS responses, which isn't ideal, although I don't know if this is a problem in practice. (Although a user can delete the RSASHA1-NSEC3-SHA1 key file, the other keys will be re-generated on upgrade.)
* When generating zonefiles, add a hash of all of the DNSSEC signing keys so that when the keys change the zone is definitely regenerated and re-signed.
* In status checks, if DNSSEC is not active (or not valid), offer to use all of the keys that have been generated (for RSASHA1-NSEC3-SHA1 on existing installs, RSASHA256, and now ECDSAP256SHA256) with all digest types, since not all registers support everything, but list them in an order that guides users to the best practice.
* In status checks, if the deployed DS record doesn't use a ECDSAP256SHA256 key, prompt the user to update their DS record.
* In status checks, if multiple DS records are set, only fail if none are valid. If some use ECDSAP256SHA256 and some don't, remind the user to delete the DS records that don't.
* Don't fail if the DS record uses the SHA384 digest (by pre-generating a DS record with that digest type) but don't recommend it because it is not in the IANA mandatory list yet (https://www.iana.org/assignments/ds-rr-types/ds-rr-types.xhtml).
See #1953
* Only spawn a thread pool when strictly needed
For --check-primary-hostname, the pool is not used.
When exiting, the other processes are left alive and will hang.
* Acquire pools with the 'with' statement
@joshdata squashed pull request #1398, removed some comments, and added these notes:
* The old init.d script for the management daemon is replaced with a systemd service.
* A systemd service configuration is added to configure permissions for munin on startup.
* nginx SSL settings are updated because nginx's options and defaults have changed, and we now enable http2.
* Automatic SSHFP record generation is updated to know that 22 is the default SSH daemon port, since it is no longer explicit in sshd_config.
* The dovecot-lucene package is dropped because the Mail-in-a-Box PPA where we built the package has not been updated for Ubuntu 18.04.
* The stock postgrey package is installed instead of the one from our PPA (which we no longer support), which loses the automatic whitelisting of DNSWL.org-whitelisted senders.
* Drop memcached and the status check for memcached, which we used to use with ownCloud long ago but are no longer installing.
* Other minor changes.
The cryptography package has created all sorts of installation trouble over the last few years, probably because of mismatches between OS-installed packages and pip-installed packages. Using a virtualenv for all Python packages used by the management daemon should make sure everything is consistent.
See #1298, see #1264.
Refactor by moving the email-the-admin code out of the status checks and into a new separate tool.
This is why I suppressed non-error output of the backups last commit - so it doesn't send a daily email.
For HTTPS for the non-primary domains, instead of selecting an SSL certificate by expecting it to be in a directory named after the domain name (with special-case lookups
for www domains, and reusing the server certificate where possible), now scan all of the certificates that have been installed and just pick the best to use for each domain.
If no certificate is available, don't create a self-signed certificate anymore. This wasn't ever really necessary. Instead just use the server certificate.
This is an extension of #427. Building on that change it adds support in the
aliases table for flagging aliases as:
1. Applicable to inbound and outbound mail.
2. Applicable to inbound mail only.
3. Applicable to outbound mail only.
4. Disabled.
The aliases UI is also updated to allow administrators to set the direction of
each alias.
Using this extra information, the sqlite queries executed by Postfix are
updated so only the relevant alias types are checked.
The goal and result of this change is that outbound-only catch-all aliases can
now be defined (in fact catch-all aliases of any type can be defined).
This allow us to continue supporting relaying as described at
https://mailinabox.email/advanced-configuration.html#relay
without requiring that administrators either create regular aliases for each
outbound *relay* address, or that they create a catch-all alias and then face a
flood of spam.
I have tested the code as it is in this commit and fixed every issue I found,
so in that regard the change is complete. However I see room for improvement
in terms of updating terminology to make the UI etc. easier to understand.
I'll make those changes as subsequent commits so that this tested checkpoint is
not lost, but also so they can be rejected independently of the actual change
if not wanted.
* Use `cryptography` instead of parsing openssl's output.
* When checking if we can reuse the primary domain certificate or a www-parent-domain certificate for a domain, avoid shelling out to openssl entirely.
* Split the nginx templates again so we have just the part needed to make a domain do a redirect separate from the rest.
* Add server blocks to the nginx config for these domains.
* List these domains in the SSL certificate install admin panel.
* Generate default 'www' records just for domains we provide default redirects for.
Fixes#321.
I changed my mind. In 1bf8f1991f I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.
It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).
This partially reverts commit 1bf8f1991f, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.
A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.
Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.
* For non-ASCII domain names, we will keep the Unicode encoding in our users/aliases table. This is nice for the user and also simplifies things like sorting domain names (using Unicode lexicographic order is good, using ASCII lexicogrpahic order on IDNA is confusing).
* Write nsd config, nsd zone files, nginx config, and SSL CSRs with domains in IDNA-encoded ASCII.
* When checking SSL certificates, treat the CN and SANs as IDNA.
* Since Chrome has an interesting feature of converting Unicode to IDNA in <input type="email"> form fields, we'll also forcibly convert IDNA to Unicode in the domain part of email addresses before saving email addresses in the users/aliases tables so that the table is normalized to Unicode.
* Don't allow non-ASCII characters in user account email addresses. Dovecot gets confused when querying the Sqlite database (which we observed even for non-word ASCII characters too, so it may not be related to the character encoding).
If bind9 isn't running, dont proceed with other checks because we can't do DNS checks. Even though we skip, add error handling so that a failed call to rndc doesn't crash and that a timeout in a DNS check doesn't crash the status checks.
