From 10bedad3a3f83a5e10551994e78a76da182df91f Mon Sep 17 00:00:00 2001 From: Joshua Tauberer Date: Sun, 17 May 2020 12:10:38 -0400 Subject: [PATCH] MTA-STS tweaks, add status check using postfix-mta-sts-resolver, change to enforce --- CHANGELOG.md | 14 +++++---- README.md | 19 +++++++----- management/dns_update.py | 58 ++++++++++++++++++++++--------------- management/status_checks.py | 20 +++++++++++++ security.md | 14 ++++----- setup/management.sh | 2 +- setup/start.sh | 12 +++----- setup/web.sh | 9 +++--- 8 files changed, 90 insertions(+), 58 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b792510b..01f860fe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,14 @@ CHANGELOG ========= +In Development +-------------- + +Mail: + +* An MTA-STS policy for incoming mail is now published (in DNS and over HTTPS) when the primary hostname and email address domain both have a signed TLS certificate installed. +* MTA-STS reporting is enabled with reports sent to administrator@ the primary hostname. + v0.45 (May 16, 2020) -------------------- @@ -24,12 +32,6 @@ Web: * Add a new hidden feature to set nginx alias in www/custom.yaml. -MTA-STS: - -* Added support for client side MTA-STS when there is a valid SSL Certificate on the primary domain -* Automatically adds reporting when alias "tlsrpt@" is added. -* Starts default on 'testing', but changes will be kept between MiaB Upgrades. - Setup: * Improved error handling. diff --git a/README.md b/README.md index e787c8d8..9a3f0ee8 100644 --- a/README.md +++ b/README.md @@ -28,15 +28,20 @@ It is a one-click email appliance. There are no user-configurable setup options. The components installed are: -* SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), Exchange ActiveSync ([z-push](http://z-push.org/)) -* Webmail ([Roundcube](http://roundcube.net/)), static website hosting ([nginx](http://nginx.org/)) -* Spam filtering ([spamassassin](https://spamassassin.apache.org/)), greylisting ([postgrey](http://postgrey.schweikert.ch/)) -* DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), and [SSHFP](https://tools.ietf.org/html/rfc4255) records automatically set -* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), system monitoring ([munin](http://munin-monitoring.org/)) +* SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), and Exchange ActiveSync ([z-push](http://z-push.org/)) servers +* Webmail ([Roundcube](http://roundcube.net/)), mail filter rules (also using dovecot), email client autoconfig settings (served by [nginx](http://nginx.org/)) +* Spam filtering ([spamassassin](https://spamassassin.apache.org/)) and greylisting ([postgrey](http://postgrey.schweikert.ch/)) +* DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), [MTA-STS](https://tools.ietf.org/html/rfc8461), and [SSHFP](https://tools.ietf.org/html/rfc4255) policy records automatically set +* TLS certificates automatically provisioned [Let's Encrypt](https://letsencrypt.org/) +* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), and basic system monitoring ([munin](http://munin-monitoring.org/)) -It also includes: +It also includes system management tools: -* A control panel and API for adding/removing mail users, aliases, custom DNS records, etc. and detailed system monitoring. +* Comprehensive health monitoring that checks each day that services are running, ports are open, TLS certificates are valid, and DNS records are correct +* A control panel for adding/removing mail users, aliases, custom DNS records, configuring backups, etc. +* An API for all of the actions on the control panel + +It also supports static website hosting since the box is serving HTTPS anyway. For more information on how Mail-in-a-Box handles your privacy, see the [security details page](security.md). diff --git a/management/dns_update.py b/management/dns_update.py index 822af4d9..98db700f 100755 --- a/management/dns_update.py +++ b/management/dns_update.py @@ -304,37 +304,47 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en if not has_rec(qname, rtype): records.append((qname, rtype, value, explanation)) - # If this is a domain name that there are email addresses configured for, i.e. "something@" - # this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461) - # Policy Domain. + # If this is a domain name that there are email addresses configured for, i.e. "something@" + # this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461) + # Policy Domain. # - # A "_mta-sts" TXT record signals the presence of a MTA-STS policy, and an effectively random policy - # ID is used to signal that a new policy may (or may not) be deployed any time the DNS is - # updated. - # - # The policy itself is served at the "mta-sts" (no underscore) subdomain over HTTPS. The - # TLS certificate used by Postfix for STARTTLS must be a valid certificate for the MX - # name (PRIMARY_HOSTNAME), so we do not set an MTA-STS policy if the certificate is not - # valid (e.g. because it is self-signed and a valid certificate has not yet been provisioned). - get_prim_cert = get_ssl_certificates(env)[env['PRIMARY_HOSTNAME']] - response = check_certificate(env['PRIMARY_HOSTNAME'], get_prim_cert['certificate'],get_prim_cert['private-key']) - - if response[0] == 'OK' and domain in get_mail_domains(env): + # A "_mta-sts" TXT record signals the presence of a MTA-STS policy, and an effectively random policy + # ID is used to signal that a new policy may (or may not) be deployed any time the DNS is + # updated. + # + # The policy itself is served at the "mta-sts" (no underscore) subdomain over HTTPS. Therefore + # the TLS certificate used by Postfix for STARTTLS must be a valid certificate for the MX + # domain name (PRIMARY_HOSTNAME) *and* the TLS certificate used by nginx for HTTPS on the mta-sts + # subdomain must be valid certificate for that domain. Do not set an MTA-STS policy if either + # certificate in use is not valid (e.g. because it is self-signed and a valid certificate has not + # yet been provisioned). + mta_sts_enabled = False + if domain in get_mail_domains(env): + # Check that PRIMARY_HOSTNAME and the mta_sts domain both have valid certificates. + for d in (env['PRIMARY_HOSTNAME'], "mta-sts." + domain): + cert = get_ssl_certificates(env).get(d) + if not cert: + break # no certificate provisioned for this domain + cert_status = check_certificate(d, cert['certificate'], cert['private-key']) + if cert_status[0] != 'OK': + break # certificate is not valid + else: + # 'break' was not encountered above, so both domains are good + mta_sts_enabled = True + if mta_sts_enabled: mta_sts_records = [ - ("mta-sts", "A", env["PUBLIC_IP"], "Provides MTA-STS support"), - ("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Provides MTA-STS support"), - ("_mta-sts", "TXT", "v=STSv1; id=%sZ" % datetime.datetime.now().strftime("%Y%m%d%H%M%S"), "Enables MTA-STS support") + ("mta-sts", "A", env["PUBLIC_IP"], "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."), + ("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."), + ("_mta-sts", "TXT", "v=STSv1; id=%sZ" % datetime.datetime.now().strftime("%Y%m%d%H%M%S"), "Optional. Part of the MTA-STS policy for incoming mail. If set, a MTA-STS policy must also be published.") ] # Rules can be custom configured accoring to https://tools.ietf.org/html/rfc8460. # Skip if the rules below if the user has set a custom _smtp._tls record. if not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"): - # if the alias 'tlsrpt@PRIMARY_HOSTNAME' is configured, automaticly, reporting will be enabled to this email address - tls_rpt_email = "tlsrpt@%s" % env['PRIMARY_HOSTNAME'] tls_rpt_string = "" - for alias in get_mail_aliases(env): - if alias[0] == tls_rpt_email: - tls_rpt_string = " rua=mailto:%s" % tls_rpt_email - mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1;%s" % tls_rpt_string, "For reporting, add an email alias: 'tlsrpt@%s' or add a custom TXT record like 'v=TLSRPTv1; rua=mailto:[youremail]@%s' for reporting" % (env["PRIMARY_HOSTNAME"], env["PRIMARY_HOSTNAME"]))) + tls_rpt_email = env.get("MTA_STS_TLSRPT_EMAIL", "postmaster@%s" % env['PRIMARY_HOSTNAME']) + if tls_rpt_email: # if a reporting address is not cleared + tls_rpt_string = " rua=mailto:%s" % tls_rpt_email + mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1;%s" % tls_rpt_string, "Optional. Enables MTA-STS reporting.")) for qname, rtype, value, explanation in mta_sts_records: if value is None or value.strip() == "": continue # skip IPV6 if not set if not has_rec(qname, rtype): diff --git a/management/status_checks.py b/management/status_checks.py index a9d0595c..2cd56f74 100755 --- a/management/status_checks.py +++ b/management/status_checks.py @@ -5,11 +5,13 @@ # what to do next. import sys, os, os.path, re, subprocess, datetime, multiprocessing.pool +import asyncio import dns.reversename, dns.resolver import dateutil.parser, dateutil.tz import idna import psutil +import postfix_mta_sts_resolver.resolver from dns_update import get_dns_zones, build_tlsa_record, get_custom_dns_config, get_secondary_dns, get_custom_dns_records from web_update import get_web_domains, get_domains_with_a_records @@ -327,6 +329,11 @@ def run_domain_checks(rounded_time, env, output, pool): def run_domain_checks_on_domain(domain, rounded_time, env, dns_domains, dns_zonefiles, mail_domains, web_domains, domains_with_a_records): output = BufferedOutput() + # When running inside Flask, the worker threads don't get a thread pool automatically. + # Also this method is called in a forked worker pool, so creating a new loop is probably + # a good idea. + asyncio.set_event_loop(asyncio.new_event_loop()) + # we'd move this up, but this returns non-pickleable values ssl_certificates = get_ssl_certificates(env) @@ -611,6 +618,19 @@ def check_mail_domain(domain, env, output): if mx != recommended_mx: good_news += " This configuration is non-standard. The recommended configuration is '%s'." % (recommended_mx,) output.print_ok(good_news) + + # Check MTA-STS policy. + loop = asyncio.get_event_loop() + sts_resolver = postfix_mta_sts_resolver.resolver.STSResolver(loop=loop) + valid, policy = loop.run_until_complete(sts_resolver.resolve(domain)) + if valid == postfix_mta_sts_resolver.resolver.STSFetchResult.VALID: + if policy[1].get("mx") == [env['PRIMARY_HOSTNAME']] and policy[1].get("mode") == "enforce": # policy[0] is the policyid + output.print_ok("MTA-STS policy is present.") + else: + output.print_error("MTA-STS policy is present but has unexpected settings. [{}]".format(policy[1])) + else: + output.print_error("MTA-STS policy is missing: {}".format(valid)) + else: output.print_error("""This domain's DNS MX record is incorrect. It is currently set to '%s' but should be '%s'. Mail will not be delivered to this box. It may take several hours for public DNS to update after a change. This problem may result from diff --git a/security.md b/security.md index ae77f339..ba3e3847 100644 --- a/security.md +++ b/security.md @@ -101,20 +101,18 @@ The box restricts the envelope sender address (also called the return path or MA Incoming Mail ------------- -### Encryption +### Encryption Settings -As discussed above, there is no way to require on-the-wire encryption of mail. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh)) +As with outbound email, there is no way to require on-the-wire encryption of incoming mail from all senders. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh)) + +### MTA-STS + +The box publishes a SMTP MTA Strict Transport Security ([SMTP MTA-STS](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_MTA_Strict_Transport_Security)) policy (via DNS and HTTPS) in "enforce" mode. Senders that support MTA-STS will use a secure SMTP connection. (MTA-STS tells senders to connect and expect a signed TLS certificate for the "MX" domain without permitting a fallback to an unencrypted connection.) ### DANE When DNSSEC is enabled at the box's domain name's registrar, [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities) records are automatically published in DNS. Senders supporting DANE will enforce encryption on-the-wire between them and the box --- see the section on DANE for outgoing mail above. ([source](management/dns_update.py)) -### MTA-STS - -SMTP MTA Strict Transport Security ([SMTP MTA-STS for short](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_MTA_Strict_Transport_Security)). - -MTA-STS is a mechanism that instructs an SMTP server that the communication with the other SMTP server MUST be encrypted and that the domain name on the certificate should match the domain in the policy. It uses a combination of DNS and HTTPS to publish a policy that tells the sending party what to do when an encrypted channel can not be negotiated. - ### Filters Incoming mail is run through several filters. Email is bounced if the sender's IP address is listed in the [Spamhaus Zen blacklist](http://www.spamhaus.org/zen/) or if the sender's domain is listed in the [Spamhaus Domain Block List](http://www.spamhaus.org/dbl/). Greylisting (with [postgrey](http://postgrey.schweikert.ch/)) is also used to cut down on spam. ([source](setup/mail-postfix.sh)) diff --git a/setup/management.sh b/setup/management.sh index 9d7c762c..4b398aa2 100755 --- a/setup/management.sh +++ b/setup/management.sh @@ -50,7 +50,7 @@ hide_output $venv/bin/pip install --upgrade pip hide_output $venv/bin/pip install --upgrade \ rtyaml "email_validator>=1.0.0" "exclusiveprocess" \ flask dnspython python-dateutil \ - "idna>=2.0.0" "cryptography==2.2.2" boto psutil + "idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver # CONFIGURATION diff --git a/setup/start.sh b/setup/start.sh index 9e2db214..cedc426d 100755 --- a/setup/start.sh +++ b/setup/start.sh @@ -82,14 +82,10 @@ if [ ! -f $STORAGE_ROOT/mailinabox.version ]; then chown $STORAGE_USER.$STORAGE_USER $STORAGE_ROOT/mailinabox.version fi -# Default policy (initial) for MTA_STS = testing in the current state of inclusion. -# it can be changed to "none", "testing" or "enforce". With this extention, this is preserved by -# future upgrades - -MTA_STS="${DEFAULT_MTA_STS:-testing}" - # Save the global options in /etc/mailinabox.conf so that standalone -# tools know where to look for data. +# tools know where to look for data. The default MTA_STS_MODE setting +# is blank unless set by an environment variable, but see web.sh for +# how that is interpreted. cat > /etc/mailinabox.conf << EOF; STORAGE_USER=$STORAGE_USER STORAGE_ROOT=$STORAGE_ROOT @@ -98,7 +94,7 @@ PUBLIC_IP=$PUBLIC_IP PUBLIC_IPV6=$PUBLIC_IPV6 PRIVATE_IP=$PRIVATE_IP PRIVATE_IPV6=$PRIVATE_IPV6 -MTA_STS=$MTA_STS +MTA_STS_MODE=${MTA_STS_MODE-} EOF # Start service configuration. diff --git a/setup/web.sh b/setup/web.sh index c384c00e..42c301ec 100755 --- a/setup/web.sh +++ b/setup/web.sh @@ -126,12 +126,13 @@ chmod a+r /var/lib/mailinabox/mozilla-autoconfig.xml # nginx configuration at /.well-known/mta-sts.txt # more documentation is available on: # https://www.uriports.com/blog/mta-sts-explained/ -# default mode is "testing", which means: "Messages will be delivered as -# though there was no failure but a report will be sent if TLS-RPT is configured" -# other valid modes are: "enforce" and "none". +# default mode is "enforce". Change to "testing" which means +# "Messages will be delivered as though there was no failure +# but a report will be sent if TLS-RPT is configured" if you +# are not sure you want this yet. Or "none". PUNY_PRIMARY_HOSTNAME=$(echo "$PRIMARY_HOSTNAME" | idn2) cat conf/mta-sts.txt \ - | sed "s/MODE/$MTA_STS/" \ + | sed "s/MODE/${MTA_STS_MODE:-enforce}/" \ | sed "s/PRIMARY_HOSTNAME/$PUNY_PRIMARY_HOSTNAME/" \ > /var/lib/mailinabox/mta-sts.txt chmod a+r /var/lib/mailinabox/mta-sts.txt