MTA-STS tweaks, add status check using postfix-mta-sts-resolver, change to enforce

This commit is contained in:
Joshua Tauberer 2020-05-17 12:10:38 -04:00
parent afc9f9686a
commit 10bedad3a3
8 changed files with 90 additions and 58 deletions

View File

@ -1,6 +1,14 @@
CHANGELOG CHANGELOG
========= =========
In Development
--------------
Mail:
* An MTA-STS policy for incoming mail is now published (in DNS and over HTTPS) when the primary hostname and email address domain both have a signed TLS certificate installed.
* MTA-STS reporting is enabled with reports sent to administrator@ the primary hostname.
v0.45 (May 16, 2020) v0.45 (May 16, 2020)
-------------------- --------------------
@ -24,12 +32,6 @@ Web:
* Add a new hidden feature to set nginx alias in www/custom.yaml. * Add a new hidden feature to set nginx alias in www/custom.yaml.
MTA-STS:
* Added support for client side MTA-STS when there is a valid SSL Certificate on the primary domain
* Automatically adds reporting when alias "tlsrpt@<primary-domain>" is added.
* Starts default on 'testing', but changes will be kept between MiaB Upgrades.
Setup: Setup:
* Improved error handling. * Improved error handling.

View File

@ -28,15 +28,20 @@ It is a one-click email appliance. There are no user-configurable setup options.
The components installed are: The components installed are:
* SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), Exchange ActiveSync ([z-push](http://z-push.org/)) * SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), and Exchange ActiveSync ([z-push](http://z-push.org/)) servers
* Webmail ([Roundcube](http://roundcube.net/)), static website hosting ([nginx](http://nginx.org/)) * Webmail ([Roundcube](http://roundcube.net/)), mail filter rules (also using dovecot), email client autoconfig settings (served by [nginx](http://nginx.org/))
* Spam filtering ([spamassassin](https://spamassassin.apache.org/)), greylisting ([postgrey](http://postgrey.schweikert.ch/)) * Spam filtering ([spamassassin](https://spamassassin.apache.org/)) and greylisting ([postgrey](http://postgrey.schweikert.ch/))
* DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), and [SSHFP](https://tools.ietf.org/html/rfc4255) records automatically set * DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), [MTA-STS](https://tools.ietf.org/html/rfc8461), and [SSHFP](https://tools.ietf.org/html/rfc4255) policy records automatically set
* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), system monitoring ([munin](http://munin-monitoring.org/)) * TLS certificates automatically provisioned [Let's Encrypt](https://letsencrypt.org/)
* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), and basic system monitoring ([munin](http://munin-monitoring.org/))
It also includes: It also includes system management tools:
* A control panel and API for adding/removing mail users, aliases, custom DNS records, etc. and detailed system monitoring. * Comprehensive health monitoring that checks each day that services are running, ports are open, TLS certificates are valid, and DNS records are correct
* A control panel for adding/removing mail users, aliases, custom DNS records, configuring backups, etc.
* An API for all of the actions on the control panel
It also supports static website hosting since the box is serving HTTPS anyway.
For more information on how Mail-in-a-Box handles your privacy, see the [security details page](security.md). For more information on how Mail-in-a-Box handles your privacy, see the [security details page](security.md).

View File

@ -304,37 +304,47 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
if not has_rec(qname, rtype): if not has_rec(qname, rtype):
records.append((qname, rtype, value, explanation)) records.append((qname, rtype, value, explanation))
# If this is a domain name that there are email addresses configured for, i.e. "something@" # If this is a domain name that there are email addresses configured for, i.e. "something@"
# this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461) # this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461)
# Policy Domain. # Policy Domain.
# #
# A "_mta-sts" TXT record signals the presence of a MTA-STS policy, and an effectively random policy # A "_mta-sts" TXT record signals the presence of a MTA-STS policy, and an effectively random policy
# ID is used to signal that a new policy may (or may not) be deployed any time the DNS is # ID is used to signal that a new policy may (or may not) be deployed any time the DNS is
# updated. # updated.
# #
# The policy itself is served at the "mta-sts" (no underscore) subdomain over HTTPS. The # The policy itself is served at the "mta-sts" (no underscore) subdomain over HTTPS. Therefore
# TLS certificate used by Postfix for STARTTLS must be a valid certificate for the MX # the TLS certificate used by Postfix for STARTTLS must be a valid certificate for the MX
# name (PRIMARY_HOSTNAME), so we do not set an MTA-STS policy if the certificate is not # domain name (PRIMARY_HOSTNAME) *and* the TLS certificate used by nginx for HTTPS on the mta-sts
# valid (e.g. because it is self-signed and a valid certificate has not yet been provisioned). # subdomain must be valid certificate for that domain. Do not set an MTA-STS policy if either
get_prim_cert = get_ssl_certificates(env)[env['PRIMARY_HOSTNAME']] # certificate in use is not valid (e.g. because it is self-signed and a valid certificate has not
response = check_certificate(env['PRIMARY_HOSTNAME'], get_prim_cert['certificate'],get_prim_cert['private-key']) # yet been provisioned).
mta_sts_enabled = False
if response[0] == 'OK' and domain in get_mail_domains(env): if domain in get_mail_domains(env):
# Check that PRIMARY_HOSTNAME and the mta_sts domain both have valid certificates.
for d in (env['PRIMARY_HOSTNAME'], "mta-sts." + domain):
cert = get_ssl_certificates(env).get(d)
if not cert:
break # no certificate provisioned for this domain
cert_status = check_certificate(d, cert['certificate'], cert['private-key'])
if cert_status[0] != 'OK':
break # certificate is not valid
else:
# 'break' was not encountered above, so both domains are good
mta_sts_enabled = True
if mta_sts_enabled:
mta_sts_records = [ mta_sts_records = [
("mta-sts", "A", env["PUBLIC_IP"], "Provides MTA-STS support"), ("mta-sts", "A", env["PUBLIC_IP"], "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Provides MTA-STS support"), ("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
("_mta-sts", "TXT", "v=STSv1; id=%sZ" % datetime.datetime.now().strftime("%Y%m%d%H%M%S"), "Enables MTA-STS support") ("_mta-sts", "TXT", "v=STSv1; id=%sZ" % datetime.datetime.now().strftime("%Y%m%d%H%M%S"), "Optional. Part of the MTA-STS policy for incoming mail. If set, a MTA-STS policy must also be published.")
] ]
# Rules can be custom configured accoring to https://tools.ietf.org/html/rfc8460. # Rules can be custom configured accoring to https://tools.ietf.org/html/rfc8460.
# Skip if the rules below if the user has set a custom _smtp._tls record. # Skip if the rules below if the user has set a custom _smtp._tls record.
if not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"): if not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"):
# if the alias 'tlsrpt@PRIMARY_HOSTNAME' is configured, automaticly, reporting will be enabled to this email address
tls_rpt_email = "tlsrpt@%s" % env['PRIMARY_HOSTNAME']
tls_rpt_string = "" tls_rpt_string = ""
for alias in get_mail_aliases(env): tls_rpt_email = env.get("MTA_STS_TLSRPT_EMAIL", "postmaster@%s" % env['PRIMARY_HOSTNAME'])
if alias[0] == tls_rpt_email: if tls_rpt_email: # if a reporting address is not cleared
tls_rpt_string = " rua=mailto:%s" % tls_rpt_email tls_rpt_string = " rua=mailto:%s" % tls_rpt_email
mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1;%s" % tls_rpt_string, "For reporting, add an email alias: 'tlsrpt@%s' or add a custom TXT record like 'v=TLSRPTv1; rua=mailto:[youremail]@%s' for reporting" % (env["PRIMARY_HOSTNAME"], env["PRIMARY_HOSTNAME"]))) mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1;%s" % tls_rpt_string, "Optional. Enables MTA-STS reporting."))
for qname, rtype, value, explanation in mta_sts_records: for qname, rtype, value, explanation in mta_sts_records:
if value is None or value.strip() == "": continue # skip IPV6 if not set if value is None or value.strip() == "": continue # skip IPV6 if not set
if not has_rec(qname, rtype): if not has_rec(qname, rtype):

View File

@ -5,11 +5,13 @@
# what to do next. # what to do next.
import sys, os, os.path, re, subprocess, datetime, multiprocessing.pool import sys, os, os.path, re, subprocess, datetime, multiprocessing.pool
import asyncio
import dns.reversename, dns.resolver import dns.reversename, dns.resolver
import dateutil.parser, dateutil.tz import dateutil.parser, dateutil.tz
import idna import idna
import psutil import psutil
import postfix_mta_sts_resolver.resolver
from dns_update import get_dns_zones, build_tlsa_record, get_custom_dns_config, get_secondary_dns, get_custom_dns_records from dns_update import get_dns_zones, build_tlsa_record, get_custom_dns_config, get_secondary_dns, get_custom_dns_records
from web_update import get_web_domains, get_domains_with_a_records from web_update import get_web_domains, get_domains_with_a_records
@ -327,6 +329,11 @@ def run_domain_checks(rounded_time, env, output, pool):
def run_domain_checks_on_domain(domain, rounded_time, env, dns_domains, dns_zonefiles, mail_domains, web_domains, domains_with_a_records): def run_domain_checks_on_domain(domain, rounded_time, env, dns_domains, dns_zonefiles, mail_domains, web_domains, domains_with_a_records):
output = BufferedOutput() output = BufferedOutput()
# When running inside Flask, the worker threads don't get a thread pool automatically.
# Also this method is called in a forked worker pool, so creating a new loop is probably
# a good idea.
asyncio.set_event_loop(asyncio.new_event_loop())
# we'd move this up, but this returns non-pickleable values # we'd move this up, but this returns non-pickleable values
ssl_certificates = get_ssl_certificates(env) ssl_certificates = get_ssl_certificates(env)
@ -611,6 +618,19 @@ def check_mail_domain(domain, env, output):
if mx != recommended_mx: if mx != recommended_mx:
good_news += " This configuration is non-standard. The recommended configuration is '%s'." % (recommended_mx,) good_news += " This configuration is non-standard. The recommended configuration is '%s'." % (recommended_mx,)
output.print_ok(good_news) output.print_ok(good_news)
# Check MTA-STS policy.
loop = asyncio.get_event_loop()
sts_resolver = postfix_mta_sts_resolver.resolver.STSResolver(loop=loop)
valid, policy = loop.run_until_complete(sts_resolver.resolve(domain))
if valid == postfix_mta_sts_resolver.resolver.STSFetchResult.VALID:
if policy[1].get("mx") == [env['PRIMARY_HOSTNAME']] and policy[1].get("mode") == "enforce": # policy[0] is the policyid
output.print_ok("MTA-STS policy is present.")
else:
output.print_error("MTA-STS policy is present but has unexpected settings. [{}]".format(policy[1]))
else:
output.print_error("MTA-STS policy is missing: {}".format(valid))
else: else:
output.print_error("""This domain's DNS MX record is incorrect. It is currently set to '%s' but should be '%s'. Mail will not output.print_error("""This domain's DNS MX record is incorrect. It is currently set to '%s' but should be '%s'. Mail will not
be delivered to this box. It may take several hours for public DNS to update after a change. This problem may result from be delivered to this box. It may take several hours for public DNS to update after a change. This problem may result from

View File

@ -101,20 +101,18 @@ The box restricts the envelope sender address (also called the return path or MA
Incoming Mail Incoming Mail
------------- -------------
### Encryption ### Encryption Settings
As discussed above, there is no way to require on-the-wire encryption of mail. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh)) As with outbound email, there is no way to require on-the-wire encryption of incoming mail from all senders. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh))
### MTA-STS
The box publishes a SMTP MTA Strict Transport Security ([SMTP MTA-STS](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_MTA_Strict_Transport_Security)) policy (via DNS and HTTPS) in "enforce" mode. Senders that support MTA-STS will use a secure SMTP connection. (MTA-STS tells senders to connect and expect a signed TLS certificate for the "MX" domain without permitting a fallback to an unencrypted connection.)
### DANE ### DANE
When DNSSEC is enabled at the box's domain name's registrar, [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities) records are automatically published in DNS. Senders supporting DANE will enforce encryption on-the-wire between them and the box --- see the section on DANE for outgoing mail above. ([source](management/dns_update.py)) When DNSSEC is enabled at the box's domain name's registrar, [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities) records are automatically published in DNS. Senders supporting DANE will enforce encryption on-the-wire between them and the box --- see the section on DANE for outgoing mail above. ([source](management/dns_update.py))
### MTA-STS
SMTP MTA Strict Transport Security ([SMTP MTA-STS for short](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_MTA_Strict_Transport_Security)).
MTA-STS is a mechanism that instructs an SMTP server that the communication with the other SMTP server MUST be encrypted and that the domain name on the certificate should match the domain in the policy. It uses a combination of DNS and HTTPS to publish a policy that tells the sending party what to do when an encrypted channel can not be negotiated.
### Filters ### Filters
Incoming mail is run through several filters. Email is bounced if the sender's IP address is listed in the [Spamhaus Zen blacklist](http://www.spamhaus.org/zen/) or if the sender's domain is listed in the [Spamhaus Domain Block List](http://www.spamhaus.org/dbl/). Greylisting (with [postgrey](http://postgrey.schweikert.ch/)) is also used to cut down on spam. ([source](setup/mail-postfix.sh)) Incoming mail is run through several filters. Email is bounced if the sender's IP address is listed in the [Spamhaus Zen blacklist](http://www.spamhaus.org/zen/) or if the sender's domain is listed in the [Spamhaus Domain Block List](http://www.spamhaus.org/dbl/). Greylisting (with [postgrey](http://postgrey.schweikert.ch/)) is also used to cut down on spam. ([source](setup/mail-postfix.sh))

View File

@ -50,7 +50,7 @@ hide_output $venv/bin/pip install --upgrade pip
hide_output $venv/bin/pip install --upgrade \ hide_output $venv/bin/pip install --upgrade \
rtyaml "email_validator>=1.0.0" "exclusiveprocess" \ rtyaml "email_validator>=1.0.0" "exclusiveprocess" \
flask dnspython python-dateutil \ flask dnspython python-dateutil \
"idna>=2.0.0" "cryptography==2.2.2" boto psutil "idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver
# CONFIGURATION # CONFIGURATION

View File

@ -82,14 +82,10 @@ if [ ! -f $STORAGE_ROOT/mailinabox.version ]; then
chown $STORAGE_USER.$STORAGE_USER $STORAGE_ROOT/mailinabox.version chown $STORAGE_USER.$STORAGE_USER $STORAGE_ROOT/mailinabox.version
fi fi
# Default policy (initial) for MTA_STS = testing in the current state of inclusion.
# it can be changed to "none", "testing" or "enforce". With this extention, this is preserved by
# future upgrades
MTA_STS="${DEFAULT_MTA_STS:-testing}"
# Save the global options in /etc/mailinabox.conf so that standalone # Save the global options in /etc/mailinabox.conf so that standalone
# tools know where to look for data. # tools know where to look for data. The default MTA_STS_MODE setting
# is blank unless set by an environment variable, but see web.sh for
# how that is interpreted.
cat > /etc/mailinabox.conf << EOF; cat > /etc/mailinabox.conf << EOF;
STORAGE_USER=$STORAGE_USER STORAGE_USER=$STORAGE_USER
STORAGE_ROOT=$STORAGE_ROOT STORAGE_ROOT=$STORAGE_ROOT
@ -98,7 +94,7 @@ PUBLIC_IP=$PUBLIC_IP
PUBLIC_IPV6=$PUBLIC_IPV6 PUBLIC_IPV6=$PUBLIC_IPV6
PRIVATE_IP=$PRIVATE_IP PRIVATE_IP=$PRIVATE_IP
PRIVATE_IPV6=$PRIVATE_IPV6 PRIVATE_IPV6=$PRIVATE_IPV6
MTA_STS=$MTA_STS MTA_STS_MODE=${MTA_STS_MODE-}
EOF EOF
# Start service configuration. # Start service configuration.

View File

@ -126,12 +126,13 @@ chmod a+r /var/lib/mailinabox/mozilla-autoconfig.xml
# nginx configuration at /.well-known/mta-sts.txt # nginx configuration at /.well-known/mta-sts.txt
# more documentation is available on: # more documentation is available on:
# https://www.uriports.com/blog/mta-sts-explained/ # https://www.uriports.com/blog/mta-sts-explained/
# default mode is "testing", which means: "Messages will be delivered as # default mode is "enforce". Change to "testing" which means
# though there was no failure but a report will be sent if TLS-RPT is configured" # "Messages will be delivered as though there was no failure
# other valid modes are: "enforce" and "none". # but a report will be sent if TLS-RPT is configured" if you
# are not sure you want this yet. Or "none".
PUNY_PRIMARY_HOSTNAME=$(echo "$PRIMARY_HOSTNAME" | idn2) PUNY_PRIMARY_HOSTNAME=$(echo "$PRIMARY_HOSTNAME" | idn2)
cat conf/mta-sts.txt \ cat conf/mta-sts.txt \
| sed "s/MODE/$MTA_STS/" \ | sed "s/MODE/${MTA_STS_MODE:-enforce}/" \
| sed "s/PRIMARY_HOSTNAME/$PUNY_PRIMARY_HOSTNAME/" \ | sed "s/PRIMARY_HOSTNAME/$PUNY_PRIMARY_HOSTNAME/" \
> /var/lib/mailinabox/mta-sts.txt > /var/lib/mailinabox/mta-sts.txt
chmod a+r /var/lib/mailinabox/mta-sts.txt chmod a+r /var/lib/mailinabox/mta-sts.txt