v0.54

Changelog entries
Enable and recommend port 465 for mail submission instead of port 587 (fixes #1849 )
2026-03-13 17:17:23 +01:00 · 2021-06-20 15:50:04 -04:00 · 2021-05-15 16:50:19 -04:00 · 2021-05-15 16:42:14 -04:00 · 2021-05-15 16:42:14 -04:00 · 2021-05-09 08:16:07 -04:00
51 changed files with 1922 additions and 655 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,6 +1,106 @@
 CHANGELOG
 =========

+v0.54 (June 20, 2021)
+---------------------
+
+Mail:
+
+* Forwarded mail using mail filter rules (in Roundcube; "sieve" rules) stopped re-writing the envelope address at some point, causing forwarded mail to often be marked as spam by the final recipient. These forwards will now re-write the envelope as the Mail-in-a-Box user receiving the mail to comply with SPF/DMARC rules.
+* Sending mail is now possible on port 465 with the "SSL" or "TLS" option in mail clients, and this is now the recommended setting. Port 587 with STARTTLS remains available but should be avoided when configuring new mail clients.
+* Roundcube's login cookie is updated to use a new encryption algorithm (AES-256-CBC instead of DES-EDE-CBC).
+
+DNS:
+
+* The ECDSAP256SHA256 DNSSEC algorithm is now available. If a DS record is set for any of your domain names that have DNS hosted on your box, you will be prompted by status checks to update the DS record at your convenience.
+* Null MX records are added for domains that do not serve mail.
+
+Contacts/calendar:
+
+* Updated Nextcloud to 20.0.8, contacts to 3.5.1, calendar to 2.2.0 (#1960).
+
+Control panel:
+
+* Fixed a crash in the status checks.
+* Small wording improvements.
+
+Setup:
+
+* Minor improvements to the setup scripts.
+
+v0.53a (May 8, 2021)
+--------------------
+
+The download URL for Z-Push has been revised becaue the old URL stopped working.
+
+v0.53 (April 12, 2021)
+----------------------
+
+Software updates:
+
+* Upgraded Roundcube to version 1.4.11 addressing a security issue, and its desktop notifications plugin.
+* Upgraded Z-Push (for Exchange/ActiveSync) to version 2.6.2.
+
+Control panel:
+
+* Backblaze B2 is now a supported backup protocol.
+* Fixed an issue in the daily mail reports.
+* Sort the Custom DNS by zone and qname, and add an option to go back to the old sort order (creation order).
+
+Mail:
+
+* Enable sending DMARC failure reports to senders that request them.
+
+Setup:
+
+* Fixed error when upgrading from Nextcloud 13.
+
+v0.52 (January 31, 2021)
+------------------------
+
+Software updates:
+
+* Upgraded Roundcube to version 1.4.10.
+* Upgraded Z-Push to 2.6.1.
+
+Mail:
+
+* Incoming emails with SPF/DKIM/DMARC failures now get a higher spam score, and these messages are more likely to appear in the junk folder, since they are often spam/phishing.
+* Fixed the MTA-STS policy file's line endings.
+
+Control panel:
+
+* A new Download button in the control panel's External DNS page can be used to download the required DNS records in zonefile format.
+* Fixed the problem when the control panel would report DNS entries as Not Set by increasing a bind query limit.
+* Fixed a control panel startup bug on some systems.
+* Improved an error message on a DNS lookup timeout.
+* A typo was fixed.
+
+DNS:
+
+* The TTL for NS records has been increased to 1 day to comply with some registrar requirements.
+
+System:
+
+* Nextcloud's photos, dashboard, and activity apps are disabled since we only support contacts and calendar.
+
+v0.51 (November 14, 2020)
+-------------------------
+
+Software updates:
+
+* Upgraded Nextcloud from 17.0.6 to 20.0.1 (with Contacts from 3.3.0 to 3.4.1 and Calendar from 2.0.3 to 2.1.2)
+* Upgraded Roundcube to version 1.4.9.
+
+Mail:
+
+* The MTA-STA max_age value was increased to the normal one week.
+
+Control panel:
+
+* Two-factor authentication can now be enabled for logins to the control panel. However, keep in mind that many online services (including domain name registrars, cloud server providers, and TLS certificate providers) may allow an attacker to take over your account or issue a fraudulent TLS certificate with only access to your email address, and this new two-factor authentication does not protect access to your inbox. It therefore remains very important that user accounts with administrative email addresses have strong passwords.
+* TLS certificate expiry dates are now shown in ISO8601 format for clarity.
+
 v0.50 (September 25, 2020)
 --------------------------

@@ -22,7 +122,7 @@ TLS:

 * TLS certificates are now provisioned in groups by parent domain to limit easy domain enumeration and make provisioning more resilient to errors for particular domains.

-Control Panel:
+Control panel:

 * The control panel API is now fully documented at https://mailinabox.email/api-docs.html.
 * User passwords can now have spaces.
--- a/README.md
+++ b/README.md
@@ -54,36 +54,18 @@ See the [setup guide](https://mailinabox.email/guide.html) for detailed, user-fr

 For experts, start with a completely fresh (really, I mean it) Ubuntu 18.04 LTS 64-bit machine. On the machine...

-Clone this repository:
+Clone this repository and checkout the tag corresponding to the most recent release:

 	$ git clone https://github.com/mail-in-a-box/mailinabox
 	$ cd mailinabox
-
-_Optional:_ Download Josh's PGP key and then verify that the sources were signed
-by him:
-
-	$ curl -s https://keybase.io/joshdata/key.asc | gpg --import
-	gpg: key C10BDD81: public key "Joshua Tauberer <jt@occams.info>" imported
-
-	$ git verify-tag v0.50
-	gpg: Signature made ..... using RSA key ID C10BDD81
-	gpg: Good signature from "Joshua Tauberer <jt@occams.info>"
-	gpg: WARNING: This key is not certified with a trusted signature!
-	gpg:          There is no indication that the signature belongs to the owner.
-	Primary key fingerprint: 5F4C 0E73 13CC D744 693B  2AEA B920 41F4 C10B DD81
-
-You'll get a lot of warnings, but that's OK. Check that the primary key fingerprint matches the
-fingerprint in the key details at [https://keybase.io/joshdata](https://keybase.io/joshdata)
-and on his [personal homepage](https://razor.occams.info/). (Of course, if this repository has been compromised you can't trust these instructions.)
-
-Checkout the tag corresponding to the most recent release:
-
-	$ git checkout v0.50
+	$ git checkout v0.54

 Begin the installation.

 	$ sudo setup/start.sh

+The installation will install, uninstall, and configure packages to turn the machine into a working, good mail server.
+
 For help, DO NOT contact Josh directly --- I don't do tech support by email or tweet (no exceptions).

 Post your question on the [discussion forum](https://discourse.mailinabox.email/) instead, where maintainers and Mail-in-a-Box users may be able to help you.
@@ -91,6 +73,7 @@ Post your question on the [discussion forum](https://discourse.mailinabox.email/
 Note that while we want everything to "just work," we can't control the rest of the Internet. Other mail services might block or spam-filter email sent from your Mail-in-a-Box.
 This is a challenge faced by everyone who runs their own mail server, with or without Mail-in-a-Box. See our discussion forum for tips about that.

+
 Contributing and Development
 ----------------------------

--- a/api/mailinabox.yml
+++ b/api/mailinabox.yml
@@ -8,14 +8,14 @@ info:
    This API is documented in [**OpenAPI format**](http://spec.openapis.org/oas/v3.0.3).
    ([View the full HTTP specification](https://raw.githubusercontent.com/mail-in-a-box/mailinabox/api-spec/api/mailinabox.yml).)

-    All endpoints are relative to `https://{host}/admin` and are secured with [`Basic Access` authentication](https://en.wikipedia.org/wiki/Basic_access_authentication).
+    All endpoints are relative to `https://{host}/admin` and are secured with [`Basic Access` authentication](https://en.wikipedia.org/wiki/Basic_access_authentication). If you have multi-factor authentication enabled, authentication with a `user:password` combination will fail unless a valid OTP is supplied via the `x-auth-token` header. Authentication via a `user:user_key` pair is possible without the header being present.
  contact:
    name: Mail-in-a-Box support
    url: https://mailinabox.email/
  license:
    name: CC0 1.0 Universal
    url: https://creativecommons.org/publicdomain/zero/1.0/legalcode
-  version: 0.47.0
+  version: 0.51.0
  x-logo:
    url: https://mailinabox.email/static/logo.png
    altText: Mail-in-a-Box logo
@@ -46,6 +46,9 @@ tags:
  - name: Web
    description: |
      Static web hosting operations, which include getting domain information and updating domain root directories.
+  - name: MFA
+    description: |
+      Manage multi-factor authentication schemes. Currently, only TOTP is supported.
  - name: System
    description: |
      System operations, which include system status checks, new version checks
@@ -740,6 +743,38 @@ paths:
            text/html:
              schema:
                type: string
+  /dns/zonefile/{zone}:
+    parameters:
+      - in: path
+        name: zone
+        schema:
+          $ref: '#/components/schemas/Hostname'
+        required: true
+        description: Hostname
+    get:
+      tags:
+        - DNS
+      summary: Get DNS zonefile
+      description: Returns a DNS zone file for a hostname.
+      operationId: getDnsZonefile
+      x-codeSamples:
+        - lang: curl
+          source: |
+            curl -X GET "https://{host}/admin/dns/zonefile/<zone>" \
+              -u "<email>:<password>"
+      responses:
+        200:
+          description: Successful operation
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/DNSZonefileResponse'
+        403:
+          description: Forbidden
+          content:
+            text/html:
+              schema:
+                type: string
  /dns/update:
    post:
      tags:
@@ -1662,6 +1697,101 @@ paths:
            text/html:
              schema:
                type: string
+  /mfa/status:
+    post:
+      tags:
+        - MFA
+      summary: Retrieve MFA status for you or another user
+      description: Retrieves which type of MFA is used and configuration
+      operationId: mfaStatus
+      x-codeSamples:
+        - lang: curl
+          source: |
+            curl -X POST "https://{host}/admin/mfa/status" \
+              -u "<email>:<password>"
+      responses:
+        200:
+          description: Successful operation
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/MfaStatusResponse'
+        403:
+          description: Forbidden
+          content:
+            text/html:
+              schema:
+                type: string
+  /mfa/totp/enable:
+    post:
+      tags:
+        - MFA
+      summary: Enable TOTP authentication
+      description: Enables TOTP authentication for the currently logged-in admin user
+      operationId: mfaTotpEnable
+      x-codeSamples:
+        - lang: curl
+          source: |
+            curl -X POST "https://{host}/admin/mfa/totp/enable" \
+              -d "code=123456" \
+              -d "secret=<string>" \
+              -u "<email>:<password>"
+      requestBody:
+        required: true
+        content:
+          application/x-www-form-urlencoded:
+            schema:
+              $ref: '#/components/schemas/MfaEnableRequest'
+      responses:
+        200:
+          description: Successful operation
+          content:
+            text/html:
+              schema:
+                $ref: '#/components/schemas/MfaEnableSuccessResponse'
+        400:
+          description: Bad request
+          content:
+            text/html:
+              schema:
+                type: string
+        403:
+          description: Forbidden
+          content:
+            text/html:
+              schema:
+                type: string
+  /mfa/disable:
+    post:
+      tags:
+        - MFA
+      summary: Disable multi-factor authentication for you or another user
+      description: Disables multi-factor authentication for the currently logged-in admin user or another user if a 'user' parameter is submitted. Either disables all multi-factor authentication methods or the method corresponding to the optional property `mfa_id`.
+      operationId: mfaTotpDisable
+      requestBody:
+        required: false
+        content:
+          application/x-www-form-urlencoded:
+            schema:
+              $ref: '#/components/schemas/MfaDisableRequest'
+      x-codeSamples:
+        - lang: curl
+          source: |
+            curl -X POST "https://{host}/admin/mfa/totp/disable" \
+              -u "<email>:<user_key>"
+      responses:
+        200:
+          description: Successful operation
+          content:
+            text/html:
+              schema:
+                $ref: '#/components/schemas/MfaDisableSuccessResponse'
+        403:
+          description: Forbidden
+          content:
+            text/html:
+              schema:
+                type: string
 components:
  securitySchemes:
    basicAuth:
@@ -1683,7 +1813,7 @@ components:
        text/plain:
          schema:
            type: string
-            example: 1.2.3.4
+            example: '1.2.3.4'
            description: The value of the DNS record.
          example: '1.2.3.4'
  schemas:
@@ -1952,6 +2082,8 @@ components:
      items:
        $ref: '#/components/schemas/Hostname'
      description: DNS zones response.
+    DNSZonefileResponse:
+      type: string
    DNSSecondaryNameserverResponse:
      type: object
      required:
@@ -2529,3 +2661,47 @@ components:
      type: string
      example: web updated
      description: Web update response.
+    MfaStatusResponse:
+      type: object
+      properties:
+        enabled_mfa:
+          type: object
+          properties:
+            id:
+              type: string
+            type:
+              type: string
+            label:
+              type: string
+          nullable: true
+        new_mfa:
+          type: object
+          properties:
+            type:
+              type: string
+            secret:
+              type: string
+            qr_code_base64:
+              type: string
+    MfaEnableRequest:
+      type: object
+      required:
+        - secret
+        - code
+      properties:
+        secret:
+          type: string
+        code:
+          type: string
+        label:
+          type: string
+    MfaEnableSuccessResponse:
+      type: string
+    MfaDisableRequest:
+      type: object
+      properties:
+        mfa_id:
+          type: string
+          nullable: true
+    MfaDisableSuccessResponse:
+      type: string
--- a/conf/fail2ban/jails.conf
+++ b/conf/fail2ban/jails.conf
@@ -38,6 +38,14 @@ logpath  = STORAGE_ROOT/owncloud/nextcloud.log
 maxretry = 20
 findtime = 120

+[miab-postfix465]
+enabled  = true
+port     = 465
+filter   = miab-postfix-submission
+logpath  = /var/log/mail.log
+maxretry = 20
+findtime = 30
+
 [miab-postfix587]
 enabled  = true
 port     = 587
--- a/conf/ios-profile.xml
+++ b/conf/ios-profile.xml
@@ -53,7 +53,7 @@
      <key>OutgoingMailServerHostName</key>
      <string>PRIMARY_HOSTNAME</string>
      <key>OutgoingMailServerPortNumber</key>
-      <integer>587</integer>
+      <integer>465</integer>
      <key>OutgoingMailServerUseSSL</key>
      <true/>
      <key>OutgoingPasswordSameAsIncomingPassword</key>
--- a/conf/mozilla-autoconfig.xml
+++ b/conf/mozilla-autoconfig.xml
@@ -16,8 +16,8 @@

      <outgoingServer type="smtp">
         <hostname>PRIMARY_HOSTNAME</hostname>
-         <port>587</port>
-         <socketType>STARTTLS</socketType>
+         <port>465</port>
+         <socketType>SSL</socketType>
         <username>%EMAILADDRESS%</username>
         <authentication>password-cleartext</authentication>
         <addThisServer>true</addThisServer>
--- a/conf/mta-sts.txt
+++ b/conf/mta-sts.txt
@@ -1,4 +1,4 @@
 version: STSv1
 mode: MODE
 mx: PRIMARY_HOSTNAME
-max_age: 86400
+max_age: 604800
--- a/conf/zpush/backend_imap.php
+++ b/conf/zpush/backend_imap.php
@@ -49,7 +49,7 @@ define('IMAP_FROM_LDAP_FULLNAME', '#givenname #sn');
 define('IMAP_SMTP_METHOD', 'sendmail');

 global $imap_smtp_params;
-$imap_smtp_params = array('host' => 'ssl://127.0.0.1', 'port' => 587, 'auth' => true, 'username' => 'imap_username', 'password' => 'imap_password');
+$imap_smtp_params = array('host' => 'ssl://127.0.0.1', 'port' => 465, 'auth' => true, 'username' => 'imap_username', 'password' => 'imap_password');

 define('MAIL_MIMEPART_CRLF', "\r\n");
 define('IMAP_MEETING_USE_CALDAV', true);
--- a/management/auth.py
+++ b/management/auth.py
@@ -1,9 +1,10 @@
-import base64, os, os.path, hmac
+import base64, os, os.path, hmac, json

 from flask import make_response

 import utils
 from mailconfig import get_mail_password, get_mail_user_privileges
+from mfa import get_hash_mfa_state, validate_auth_mfa

 DEFAULT_KEY_PATH   = '/var/lib/mailinabox/api.key'
 DEFAULT_AUTH_REALM = 'Mail-in-a-Box Management Server'
@@ -72,17 +73,19 @@ class KeyAuthService:
 		if username in (None, ""):
 			raise ValueError("Authorization header invalid.")
 		elif username == self.key:
-			# The user passed the API key which grants administrative privs.
+			# The user passed the master API key which grants administrative privs.
 			return (None, ["admin"])
 		else:
-			# The user is trying to log in with a username and user-specific
-			# API key or password. Raises or returns privs.
-			return (username, self.get_user_credentials(username, password, env))
+			# The user is trying to log in with a username and either a password
+			# (and possibly a MFA token) or a user-specific API key.
+			return (username, self.check_user_auth(username, password, request, env))

-	def get_user_credentials(self, email, pw, env):
-		# Validate a user's credentials. On success returns a list of
-		# privileges (e.g. [] or ['admin']). On failure raises a ValueError
-		# with a login error message. 
+	def check_user_auth(self, email, pw, request, env):
+		# Validate a user's login email address and password. If MFA is enabled,
+		# check the MFA token in the X-Auth-Token header.
+		#
+		# On success returns a list of privileges (e.g. [] or ['admin']). On login
+		# failure, raises a ValueError with a login error message.

 		# Sanity check.
 		if email == "" or pw == "":
@@ -112,6 +115,12 @@ class KeyAuthService:
 				# Login failed.
 				raise ValueError("Invalid password.")

+			# If MFA is enabled, check that MFA passes.
+			status, hints = validate_auth_mfa(email, request, env)
+			if not status:
+				# Login valid. Hints may have more info.
+				raise ValueError(",".join(hints))
+
 		# Get privileges for authorization. This call should never fail because by this
 		# point we know the email address is a valid user. But on error the call will
 		# return a tuple of an error message and an HTTP status code.
@@ -122,16 +131,27 @@ class KeyAuthService:
 		return privs

 	def create_user_key(self, email, env):
-		# Store an HMAC with the client. The hashed message of the HMAC will be the user's
-		# email address & hashed password and the key will be the master API key. The user of
-		# course has their own email address and password. We assume they do not have the master
-		# API key (unless they are trusted anyway). The HMAC proves that they authenticated
-		# with us in some other way to get the HMAC. Including the password means that when
-		# a user's password is reset, the HMAC changes and they will correctly need to log
-		# in to the control panel again. This method raises a ValueError if the user does
-		# not exist, due to get_mail_password.
+		# Create a user API key, which is a shared secret that we can re-generate from
+		# static information in our database. The shared secret contains the user's
+		# email address, current hashed password, and current MFA state, so that the
+		# key becomes invalid if any of that information changes.
+		#
+		# Use an HMAC to generate the API key using our master API key as a key,
+		# which also means that the API key becomes invalid when our master API key
+		# changes --- i.e. when this process is restarted.
+		#
+		# Raises ValueError via get_mail_password if the user doesn't exist.
+
+		# Construct the HMAC message from the user's email address and current password.
 		msg = b"AUTH:" + email.encode("utf8") + b" " + get_mail_password(email, env).encode("utf8")
-		return hmac.new(self.key.encode('ascii'), msg, digestmod="sha256").hexdigest()
+
+		# Add to the message the current MFA state, which is a list of MFA information.
+		# Turn it into a string stably.
+		msg += b" " + json.dumps(get_hash_mfa_state(email, env), sort_keys=True).encode("utf8")
+
+		# Make the HMAC.
+		hash_key = self.key.encode('ascii')
+		return hmac.new(hash_key, msg, digestmod="sha256").hexdigest()

 	def _generate_key(self):
 		raw_key = os.urandom(32)
--- a/management/backup.py
+++ b/management/backup.py
@@ -456,6 +456,23 @@ def list_target_files(config):
 			raise ValueError(e.reason)

 		return [(key.name[len(path):], key.size) for key in bucket.list(prefix=path)]
+	elif target.scheme == 'b2':
+		from b2sdk.v1 import InMemoryAccountInfo, B2Api
+		from b2sdk.v1.exception import NonExistentBucket
+		info = InMemoryAccountInfo()
+		b2_api = B2Api(info)
+		
+		# Extract information from target
+		b2_application_keyid = target.netloc[:target.netloc.index(':')]
+		b2_application_key = target.netloc[target.netloc.index(':')+1:target.netloc.index('@')]
+		b2_bucket = target.netloc[target.netloc.index('@')+1:]
+
+		try:
+			b2_api.authorize_account("production", b2_application_keyid, b2_application_key)
+			bucket = b2_api.get_bucket_by_name(b2_bucket)
+		except NonExistentBucket as e:
+			raise ValueError("B2 Bucket does not exist. Please double check your information!")
+		return [(key.file_name, key.size) for key, _ in bucket.ls()]

 	else:
 		raise ValueError(config["target"])
--- a/management/cli.py
+++ b/management/cli.py
@@ -0,0 +1,150 @@
+#!/usr/bin/python3
+#
+# This is a command-line script for calling management APIs
+# on the Mail-in-a-Box control panel backend. The script
+# reads /var/lib/mailinabox/api.key for the backend's
+# root API key. This file is readable only by root, so this
+# tool can only be used as root.
+
+import sys, getpass, urllib.request, urllib.error, json, re, csv
+
+def mgmt(cmd, data=None, is_json=False):
+	# The base URL for the management daemon. (Listens on IPv4 only.)
+	mgmt_uri = 'http://127.0.0.1:10222'
+
+	setup_key_auth(mgmt_uri)
+
+	req = urllib.request.Request(mgmt_uri + cmd, urllib.parse.urlencode(data).encode("utf8") if data else None)
+	try:
+		response = urllib.request.urlopen(req)
+	except urllib.error.HTTPError as e:
+		if e.code == 401:
+			try:
+				print(e.read().decode("utf8"))
+			except:
+				pass
+			print("The management daemon refused access. The API key file may be out of sync. Try 'service mailinabox restart'.", file=sys.stderr)
+		elif hasattr(e, 'read'):
+			print(e.read().decode('utf8'), file=sys.stderr)
+		else:
+			print(e, file=sys.stderr)
+		sys.exit(1)
+	resp = response.read().decode('utf8')
+	if is_json: resp = json.loads(resp)
+	return resp
+
+def read_password():
+    while True:
+        first = getpass.getpass('password: ')
+        if len(first) < 8:
+            print("Passwords must be at least eight characters.")
+            continue
+        second = getpass.getpass(' (again): ')
+        if first != second:
+            print("Passwords not the same. Try again.")
+            continue
+        break
+    return first
+
+def setup_key_auth(mgmt_uri):
+	key = open('/var/lib/mailinabox/api.key').read().strip()
+
+	auth_handler = urllib.request.HTTPBasicAuthHandler()
+	auth_handler.add_password(
+		realm='Mail-in-a-Box Management Server',
+		uri=mgmt_uri,
+		user=key,
+		passwd='')
+	opener = urllib.request.build_opener(auth_handler)
+	urllib.request.install_opener(opener)
+
+if len(sys.argv) < 2:
+	print("""Usage:
+  {cli} user                                     (lists users)
+  {cli} user add user@domain.com [password]
+  {cli} user password user@domain.com [password]
+  {cli} user remove user@domain.com
+  {cli} user make-admin user@domain.com
+  {cli} user remove-admin user@domain.com
+  {cli} user admins                              (lists admins)
+  {cli} user mfa show user@domain.com            (shows MFA devices for user, if any)
+  {cli} user mfa disable user@domain.com [id]    (disables MFA for user)
+  {cli} alias                                    (lists aliases)
+  {cli} alias add incoming.name@domain.com sent.to@other.domain.com
+  {cli} alias add incoming.name@domain.com 'sent.to@other.domain.com, multiple.people@other.domain.com'
+  {cli} alias remove incoming.name@domain.com
+
+Removing a mail user does not delete their mail folders on disk. It only prevents IMAP/SMTP login.
+""".format(
+	cli="management/cli.py"
+		))
+
+elif sys.argv[1] == "user" and len(sys.argv) == 2:
+	# Dump a list of users, one per line. Mark admins with an asterisk.
+	users = mgmt("/mail/users?format=json", is_json=True)
+	for domain in users:
+		for user in domain["users"]:
+			if user['status'] == 'inactive': continue
+			print(user['email'], end='')
+			if "admin" in user['privileges']:
+				print("*", end='')
+			print()
+
+elif sys.argv[1] == "user" and sys.argv[2] in ("add", "password"):
+	if len(sys.argv) < 5:
+		if len(sys.argv) < 4:
+			email = input("email: ")
+		else:
+			email = sys.argv[3]
+		pw = read_password()
+	else:
+		email, pw = sys.argv[3:5]
+
+	if sys.argv[2] == "add":
+		print(mgmt("/mail/users/add", { "email": email, "password": pw }))
+	elif sys.argv[2] == "password":
+		print(mgmt("/mail/users/password", { "email": email, "password": pw }))
+
+elif sys.argv[1] == "user" and sys.argv[2] == "remove" and len(sys.argv) == 4:
+	print(mgmt("/mail/users/remove", { "email": sys.argv[3] }))
+
+elif sys.argv[1] == "user" and sys.argv[2] in ("make-admin", "remove-admin") and len(sys.argv) == 4:
+	if sys.argv[2] == "make-admin":
+		action = "add"
+	else:
+		action = "remove"
+	print(mgmt("/mail/users/privileges/" + action, { "email": sys.argv[3], "privilege": "admin" }))
+
+elif sys.argv[1] == "user" and sys.argv[2] == "admins":
+	# Dump a list of admin users.
+	users = mgmt("/mail/users?format=json", is_json=True)
+	for domain in users:
+		for user in domain["users"]:
+			if "admin" in user['privileges']:
+				print(user['email'])
+
+elif sys.argv[1] == "user" and len(sys.argv) == 5 and sys.argv[2:4] == ["mfa", "show"]:
+	# Show MFA status for a user.
+	status = mgmt("/mfa/status", { "user": sys.argv[4] }, is_json=True)
+	W = csv.writer(sys.stdout)
+	W.writerow(["id", "type", "label"])
+	for mfa in status["enabled_mfa"]:
+		W.writerow([mfa["id"], mfa["type"], mfa["label"]])
+
+elif sys.argv[1] == "user" and len(sys.argv) in (5, 6) and sys.argv[2:4] == ["mfa", "disable"]:
+	# Disable MFA (all or a particular device) for a user.
+	print(mgmt("/mfa/disable", { "user": sys.argv[4], "mfa-id": sys.argv[5] if len(sys.argv) == 6 else None }))
+
+elif sys.argv[1] == "alias" and len(sys.argv) == 2:
+	print(mgmt("/mail/aliases"))
+
+elif sys.argv[1] == "alias" and sys.argv[2] == "add" and len(sys.argv) == 5:
+	print(mgmt("/mail/aliases/add", { "address": sys.argv[3], "forwards_to": sys.argv[4] }))
+
+elif sys.argv[1] == "alias" and sys.argv[2] == "remove" and len(sys.argv) == 4:
+	print(mgmt("/mail/aliases/remove", { "address": sys.argv[3] }))
+
+else:
+	print("Invalid command-line arguments.")
+	sys.exit(1)
+
--- a/management/daemon.py
+++ b/management/daemon.py
@@ -1,14 +1,24 @@
+#!/usr/local/lib/mailinabox/env/bin/python3
+#
+# During development, you can start the Mail-in-a-Box control panel
+# by running this script, e.g.:
+#
+# service mailinabox stop # stop the system process
+# DEBUG=1 management/daemon.py
+# service mailinabox start # when done debugging, start it up again
+
 import os, os.path, re, json, time
-import subprocess
+import multiprocessing.pool, subprocess

 from functools import wraps

 from flask import Flask, request, render_template, abort, Response, send_from_directory, make_response

-import auth, utils, multiprocessing.pool
+import auth, utils
 from mailconfig import get_mail_users, get_mail_users_ex, get_admins, add_mail_user, set_mail_password, remove_mail_user
 from mailconfig import get_mail_user_privileges, add_remove_mail_user_privilege
 from mailconfig import get_mail_aliases, get_mail_aliases_ex, get_mail_domains, add_mail_alias, remove_mail_alias
+from mfa import get_public_mfa_state, provision_totp, validate_totp_secret, enable_mfa, disable_mfa

 env = utils.load_environment()

@@ -35,23 +45,31 @@ app = Flask(__name__, template_folder=os.path.abspath(os.path.join(os.path.dirna
 def authorized_personnel_only(viewfunc):
 	@wraps(viewfunc)
 	def newview(*args, **kwargs):
-		# Authenticate the passed credentials, which is either the API key or a username:password pair.
+		# Authenticate the passed credentials, which is either the API key or a username:password pair
+		# and an optional X-Auth-Token token.
 		error = None
+		privs = []
+
 		try:
 			email, privs = auth_service.authenticate(request, env)
 		except ValueError as e:
-			# Authentication failed.
-			privs = []
-			error = "Incorrect username or password"
-
 			# Write a line in the log recording the failed login
 			log_failed_login(request)

+			# Authentication failed.
+			error = str(e)
+
 		# Authorized to access an API view?
 		if "admin" in privs:
+			# Store the email address of the logged in user so it can be accessed
+			# from the API methods that affect the calling user.
+			request.user_email = email
+			request.user_privs = privs
+
 			# Call view func.
 			return viewfunc(*args, **kwargs)
-		elif not error:
+
+		if not error:
 			error = "You are not an administrator."

 		# Not authorized. Return a 401 (send auth) and a prompt to authorize by default.
@@ -83,8 +101,8 @@ def authorized_personnel_only(viewfunc):
 def unauthorized(error):
 	return auth_service.make_unauthorized_response()

-def json_response(data):
-	return Response(json.dumps(data, indent=2, sort_keys=True)+'\n', status=200, mimetype='application/json')
+def json_response(data, status=200):
+	return Response(json.dumps(data, indent=2, sort_keys=True)+'\n', status=status, mimetype='application/json')

 ###################################

@@ -119,12 +137,17 @@ def me():
 	try:
 		email, privs = auth_service.authenticate(request, env)
 	except ValueError as e:
+		if "missing-totp-token" in str(e):
+			return json_response({
+				"status": "missing-totp-token",
+				"reason": str(e),
+			})
+		else:
 			# Log the failed login
 			log_failed_login(request)
-
 			return json_response({
 				"status": "invalid",
-			"reason": "Incorrect username or password",
+				"reason": str(e),
 			})

 	resp = {
@@ -254,17 +277,50 @@ def dns_set_secondary_nameserver():
@app.route('/dns/custom')
@authorized_personnel_only
 def dns_get_records(qname=None, rtype=None):
-	from dns_update import get_custom_dns_config
-	return json_response([
+	# Get the current set of custom DNS records.
+	from dns_update import get_custom_dns_config, get_dns_zones
+	records = get_custom_dns_config(env, only_real_records=True)
+
+	# Filter per the arguments for the more complex GET routes below.
+	records = [r for r in records
+		if (not qname or r[0] == qname)
+		and (not rtype or r[1] == rtype) ]
+
+	# Make a better data structure.
+	records = [
        {
                "qname": r[0],
                "rtype": r[1],
                "value": r[2],
-	}
-	for r in get_custom_dns_config(env)
-	if r[0] != "_secondary_nameserver"
-		and (not qname or r[0] == qname)
-		and (not rtype or r[1] == rtype) ])
+		"sort-order": { },
+        } for r in records ]
+
+	# To help with grouping by zone in qname sorting, label each record with which zone it is in.
+	# There's an inconsistency in how we handle zones in get_dns_zones and in sort_domains, so
+	# do this first before sorting the domains within the zones.
+	zones = utils.sort_domains([z[0] for z in get_dns_zones(env)], env)
+	for r in records:
+		for z in zones:
+			if r["qname"] == z or r["qname"].endswith("." + z):
+				r["zone"] = z
+				break
+
+	# Add sorting information. The 'created' order follows the order in the YAML file on disk,
+	# which tracs the order entries were added in the control panel since we append to the end.
+	# The 'qname' sort order sorts by our standard domain name sort (by zone then by qname),
+	# then by rtype, and last by the original order in the YAML file (since sorting by value
+	# may not make sense, unless we parse IP addresses, for example).
+	for i, r in enumerate(records):
+		r["sort-order"]["created"] = i
+	domain_sort_order = utils.sort_domains([r["qname"] for r in records], env)
+	for i, r in enumerate(sorted(records, key = lambda r : (
+			zones.index(r["zone"]),
+			domain_sort_order.index(r["qname"]),
+			r["rtype"]))):
+		r["sort-order"]["qname"] = i
+
+	# Return.
+	return json_response(records)

@app.route('/dns/custom/<qname>', methods=['GET', 'POST', 'PUT', 'DELETE'])
@app.route('/dns/custom/<qname>/<rtype>', methods=['GET', 'POST', 'PUT', 'DELETE'])
@@ -324,6 +380,12 @@ def dns_get_dump():
 	from dns_update import build_recommended_dns
 	return json_response(build_recommended_dns(env))

+@app.route('/dns/zonefile/<zone>')
+@authorized_personnel_only
+def dns_get_zonefile(zone):
+	from dns_update import get_dns_zonefile
+	return Response(get_dns_zonefile(zone, env), status=200, mimetype='text/plain')
+
 # SSL

@app.route('/ssl/status')
@@ -383,6 +445,60 @@ def ssl_provision_certs():
 	requests = provision_certificates(env, limit_domains=None)
 	return json_response({ "requests": requests })

+# multi-factor auth
+
+@app.route('/mfa/status', methods=['POST'])
+@authorized_personnel_only
+def mfa_get_status():
+	# Anyone accessing this route is an admin, and we permit them to
+	# see the MFA status for any user if they submit a 'user' form
+	# field. But we don't include provisioning info since a user can
+	# only provision for themselves.
+	email = request.form.get('user', request.user_email) # user field if given, otherwise the user making the request
+	try:
+		resp = {
+			"enabled_mfa": get_public_mfa_state(email, env)
+		}
+		if email == request.user_email:
+			resp.update({
+				"new_mfa": {
+					"totp": provision_totp(email, env)
+				}
+			})
+	except ValueError as e:
+		return (str(e), 400)
+	return json_response(resp)
+
+@app.route('/mfa/totp/enable', methods=['POST'])
+@authorized_personnel_only
+def totp_post_enable():
+	secret = request.form.get('secret')
+	token = request.form.get('token')
+	label = request.form.get('label')
+	if type(token) != str:
+		return ("Bad Input", 400)
+	try:
+		validate_totp_secret(secret)
+		enable_mfa(request.user_email, "totp", secret, token, label, env)
+	except ValueError as e:
+		return (str(e), 400)
+	return "OK"
+
+@app.route('/mfa/disable', methods=['POST'])
+@authorized_personnel_only
+def totp_post_disable():
+	# Anyone accessing this route is an admin, and we permit them to
+	# disable the MFA status for any user if they submit a 'user' form
+	# field.
+	email = request.form.get('user', request.user_email) # user field if given, otherwise the user making the request
+	try:
+		result = disable_mfa(email, request.form.get('mfa-id') or None, env) # convert empty string to None
+	except ValueError as e:
+		return (str(e), 400)
+	if result: # success
+		return "OK"
+	else: # error
+		return ("Invalid user or MFA id.", 400)

 # WEB

@@ -604,7 +720,22 @@ def log_failed_login(request):
 # APP

 if __name__ == '__main__':
-	if "DEBUG" in os.environ: app.debug = True
+	if "DEBUG" in os.environ:
+		# Turn on Flask debugging.
+		app.debug = True
+
+		# Use a stable-ish master API key so that login sessions don't restart on each run.
+		# Use /etc/machine-id to seed the key with a stable secret, but add something
+		# and hash it to prevent possibly exposing the machine id, using the time so that
+		# the key is not valid indefinitely.
+		import hashlib
+		with open("/etc/machine-id") as f:
+			api_key = f.read()
+		api_key += "|" + str(int(time.time() / (60*60*2)))
+		hasher = hashlib.sha1()
+		hasher.update(api_key.encode("ascii"))
+		auth_service.key = hasher.hexdigest()
+
 	if "APIKEY" in os.environ: auth_service.key = os.environ["APIKEY"]

 	if not app.debug:
--- a/management/dns_update.py
+++ b/management/dns_update.py
@@ -9,7 +9,6 @@ import ipaddress
 import rtyaml
 import dns.resolver

-from mailconfig import get_mail_domains, get_mail_aliases
 from utils import shell, load_env_vars_from_file, safe_domain_name, sort_domains
 from ssl_certificates import get_ssl_certificates, check_certificate

@@ -20,10 +19,14 @@ from ssl_certificates import get_ssl_certificates, check_certificate
 DOMAIN_RE = "^(?!\-)(?:[*][.])?(?:[a-zA-Z\d\-_]{0,62}[a-zA-Z\d_]\.){1,126}(?!\d+)[a-zA-Z\d_]{1,63}(\.?)$"

 def get_dns_domains(env):
-	# Add all domain names in use by email users and mail aliases and ensure
-	# PRIMARY_HOSTNAME is in the list.
+	# Add all domain names in use by email users and mail aliases, any
+	# domains we serve web for (except www redirects because that would
+	# lead to infinite recursion here) and ensure PRIMARY_HOSTNAME is in the list.
+	from mailconfig import get_mail_domains
+	from web_update import get_web_domains
 	domains = set()
-	domains |= get_mail_domains(env)
+	domains |= set(get_mail_domains(env))
+	domains |= set(get_web_domains(env, include_www_redirects=False))
 	domains.add(env['PRIMARY_HOSTNAME'])
 	return domains

@@ -97,7 +100,8 @@ def do_dns_update(env, force=False):
 	if len(updated_domains) > 0:
 		shell('check_call', ["/usr/sbin/service", "nsd", "restart"])

-	# Write the OpenDKIM configuration tables for all of the domains.
+	# Write the OpenDKIM configuration tables for all of the mail domains.
+	from mailconfig import get_mail_domains
 	if write_opendkim_tables(get_mail_domains(env), env):
 		# Settings changed. Kick opendkim.
 		shell('check_call', ["/usr/sbin/service", "opendkim", "restart"])
@@ -122,18 +126,47 @@ def build_zones(env):
 	domains = get_dns_domains(env)
 	zonefiles = get_dns_zones(env)

-	# Custom records to add to zones.
-	additional_records = list(get_custom_dns_config(env))
+	# Create a dictionary of domains to a set of attributes for each
+	# domain, such as whether there are mail users at the domain.
+	from mailconfig import get_mail_domains
 	from web_update import get_web_domains
-	www_redirect_domains = set(get_web_domains(env)) - set(get_web_domains(env, include_www_redirects=False))
+	mail_domains = set(get_mail_domains(env))
+	mail_user_domains = set(get_mail_domains(env, users_only=True)) # i.e. will log in for mail, Nextcloud
+	web_domains = set(get_web_domains(env))
+	auto_domains = web_domains - set(get_web_domains(env, include_auto=False))
+	domains |= auto_domains # www redirects not included in the initial list, see above
+
+	# Add ns1/ns2+PRIMARY_HOSTNAME which must also have A/AAAA records
+	# when the box is acting as authoritative DNS server for its domains.
+	for ns in ("ns1", "ns2"):
+		d = ns + "." + env["PRIMARY_HOSTNAME"]
+		domains.add(d)
+		auto_domains.add(d)
+
+	domains = {
+		domain: {
+			"user": domain in mail_user_domains,
+			"mail": domain in mail_domains,
+			"web": domain in web_domains,
+			"auto": domain in auto_domains,
+		}
+		for domain in domains
+	}
+
+	# For MTA-STS, we'll need to check if the PRIMARY_HOSTNAME certificate is
+	# singned and valid. Check that now rather than repeatedly for each domain.
+	domains[env["PRIMARY_HOSTNAME"]]["certificate-is-valid"] = is_domain_cert_signed_and_valid(env["PRIMARY_HOSTNAME"], env)
+
+	# Load custom records to add to zones.
+	additional_records = list(get_custom_dns_config(env))

 	# Build DNS records for each zone.
 	for domain, zonefile in zonefiles:
 		# Build the records to put in the zone.
-		records = build_zone(domain, domains, additional_records, www_redirect_domains, env)
+		records = build_zone(domain, domains, additional_records, env)
 		yield (domain, zonefile, records)

-def build_zone(domain, all_domains, additional_records, www_redirect_domains, env, is_zone=True):
+def build_zone(domain, domain_properties, additional_records, env, is_zone=True):
 	records = []

 	# For top-level zones, define the authoritative name servers.
@@ -145,10 +178,10 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
 	# 'False' in the tuple indicates these records would not be used if the zone
 	# is managed outside of the box.
 	if is_zone:
-		# Obligatory definition of ns1.PRIMARY_HOSTNAME.
+		# Obligatory NS record to ns1.PRIMARY_HOSTNAME.
 		records.append((None,  "NS",  "ns1.%s." % env["PRIMARY_HOSTNAME"], False))

-		# Define ns2.PRIMARY_HOSTNAME or whatever the user overrides.
+		# NS record to ns2.PRIMARY_HOSTNAME or whatever the user overrides.
 		# User may provide one or more additional nameservers
 		secondary_ns_list = get_secondary_dns(additional_records, mode="NS") \
 			or ["ns2." + env["PRIMARY_HOSTNAME"]]
@@ -158,15 +191,6 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en

 	# In PRIMARY_HOSTNAME...
 	if domain == env["PRIMARY_HOSTNAME"]:
-		# Define ns1 and ns2.
-		# 'False' in the tuple indicates these records would not be used if the zone
-		# is managed outside of the box.
-		records.append(("ns1", "A", env["PUBLIC_IP"], False))
-		records.append(("ns2", "A", env["PUBLIC_IP"], False))
-		if env.get('PUBLIC_IPV6'):
-			records.append(("ns1", "AAAA", env["PUBLIC_IPV6"], False))
-			records.append(("ns2", "AAAA", env["PUBLIC_IPV6"], False))
-
 		# Set the A/AAAA records. Do this early for the PRIMARY_HOSTNAME so that the user cannot override them
 		# and we can provide different explanatory text.
 		records.append((None, "A", env["PUBLIC_IP"], "Required. Sets the IP address of the box."))
@@ -184,10 +208,11 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en

 	# Add DNS records for any subdomains of this domain. We should not have a zone for
 	# both a domain and one of its subdomains.
-	subdomains = [d for d in all_domains if d.endswith("." + domain)]
+	if is_zone: # don't recurse when we're just loading data for a subdomain
+		subdomains = [d for d in domain_properties if d.endswith("." + domain)]
 		for subdomain in subdomains:
 			subdomain_qname = subdomain[0:-len("." + domain)]
-		subzone = build_zone(subdomain, [], additional_records, www_redirect_domains, env, is_zone=False)
+			subzone = build_zone(subdomain, domain_properties, additional_records, env, is_zone=False)
 			for child_qname, child_rtype, child_value, child_explanation in subzone:
 				if child_qname == None:
 					child_qname = subdomain_qname
@@ -220,21 +245,23 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
 				continue
 		records.append((qname, rtype, value, "(Set by user.)"))

-	# Add defaults if not overridden by the user's custom settings (and not otherwise configured).
+	# Add A/AAAA defaults if not overridden by the user's custom settings (and not otherwise configured).
 	# Any CNAME or A record on the qname overrides A and AAAA. But when we set the default A record,
 	# we should not cause the default AAAA record to be skipped because it thinks a custom A record
 	# was set. So set has_rec_base to a clone of the current set of DNS settings, and don't update
 	# during this process.
 	has_rec_base = list(records)
+	a_expl = "Required. May have a different value. Sets the IP address that %s resolves to for web hosting and other services besides mail. The A record must be present but its value does not affect mail delivery." % domain
+	if domain_properties[domain]["auto"]:
+		if domain.startswith("ns1.") or domain.startswith("ns2."): a_expl = False # omit from 'External DNS' page since this only applies if box is its own DNS server
+		if domain.startswith("www."): a_expl = "Optional. Sets the IP address that %s resolves to so that the box can provide a redirect to the parent domain." % domain
+		if domain.startswith("mta-sts."): a_expl = "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."
+		if domain.startswith("autoconfig."): a_expl = "Provides email configuration autodiscovery support for Thunderbird Autoconfig."
+		if domain.startswith("autodiscover."): a_expl = "Provides email configuration autodiscovery support for Z-Push ActiveSync Autodiscover."
 	defaults = [
-		(None,  "A",    env["PUBLIC_IP"],       "Required. May have a different value. Sets the IP address that %s resolves to for web hosting and other services besides mail. The A record must be present but its value does not affect mail delivery." % domain),
+		(None,  "A",    env["PUBLIC_IP"], a_expl),
 		(None,  "AAAA", env.get('PUBLIC_IPV6'), "Optional. Sets the IPv6 address that %s resolves to, e.g. for web hosting. (It is not necessary for receiving mail on this domain.)" % domain),
 	]
-	if "www." + domain in www_redirect_domains:
-		defaults += [
-			("www", "A",    env["PUBLIC_IP"],       "Optional. Sets the IP address that www.%s resolves to so that the box can provide a redirect to the parent domain." % domain),
-			("www", "AAAA", env.get('PUBLIC_IPV6'), "Optional. Sets the IPv6 address that www.%s resolves to so that the box can provide a redirect to the parent domain." % domain),
-		]
 	for qname, rtype, value, explanation in defaults:
 		if value is None or value.strip() == "": continue # skip IPV6 if not set
 		if not is_zone and qname == "www": continue # don't create any default 'www' subdomains on what are themselves subdomains
@@ -248,6 +275,7 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
 	# Don't pin the list of records that has_rec checks against anymore.
 	has_rec_base = records

+	if domain_properties[domain]["mail"]:
 		# The MX record says where email for the domain should be delivered: Here!
 		if not has_rec(None, "MX", prefix="10 "):
 			records.append((None,  "MX",  "10 %s." % env["PRIMARY_HOSTNAME"], "Required. Specifies the hostname (and priority) of the machine that handles @%s mail." % domain))
@@ -272,40 +300,16 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
 		if not has_rec("_dmarc", "TXT", prefix="v=DMARC1; "):
 			records.append(("_dmarc", "TXT", 'v=DMARC1; p=quarantine', "Recommended. Specifies that mail that does not originate from the box but claims to be from @%s or which does not have a valid DKIM signature is suspect and should be quarantined by the recipient's mail system." % domain))

-	# For any subdomain with an A record but no SPF or DMARC record, add strict policy records.
-	all_resolvable_qnames = set(r[0] for r in records if r[1] in ("A", "AAAA"))
-	for qname in all_resolvable_qnames:
-		if not has_rec(qname, "TXT", prefix="v=spf1 "):
-			records.append((qname,  "TXT", 'v=spf1 -all', "Recommended. Prevents use of this domain name for outbound mail by specifying that no servers are valid sources for mail from @%s. If you do send email from this domain name you should either override this record such that the SPF rule does allow the originating server, or, take the recommended approach and have the box handle mail for this domain (simply add any receiving alias at this domain name to make this machine treat the domain name as one of its mail domains)." % (qname + "." + domain)))
-		dmarc_qname = "_dmarc" + ("" if qname is None else "." + qname)
-		if not has_rec(dmarc_qname, "TXT", prefix="v=DMARC1; "):
-			records.append((dmarc_qname, "TXT", 'v=DMARC1; p=reject', "Recommended. Prevents use of this domain name for outbound mail by specifying that the SPF rule should be honoured for mail from @%s." % (qname + "." + domain)))
-
+	if domain_properties[domain]["user"]:
 		# Add CardDAV/CalDAV SRV records on the non-primary hostname that points to the primary hostname
 		# for autoconfiguration of mail clients (so only domains hosting user accounts need it).
 		# The SRV record format is priority (0, whatever), weight (0, whatever), port, service provider hostname (w/ trailing dot).
-	if domain != env["PRIMARY_HOSTNAME"] and domain in get_mail_domains(env, users_only=True):
+		if domain != env["PRIMARY_HOSTNAME"]:
 			for dav in ("card", "cal"):
 				qname = "_" + dav + "davs._tcp"
 				if not has_rec(qname, "SRV"):
 					records.append((qname, "SRV", "0 0 443 " + env["PRIMARY_HOSTNAME"] + ".", "Recommended. Specifies the hostname of the server that handles CardDAV/CalDAV services for email addresses on this domain."))

-	# Adds autoconfiguration A records for all domains that there are user accounts at.
-	# This allows the following clients to automatically configure email addresses in the respective applications.
-	# autodiscover.* - Z-Push ActiveSync Autodiscover
-	# autoconfig.* - Thunderbird Autoconfig
-	if domain in get_mail_domains(env, users_only=True):
-		autodiscover_records = [
-			("autodiscover", "A", env["PUBLIC_IP"], "Provides email configuration autodiscovery support for Z-Push ActiveSync Autodiscover."),
-			("autodiscover", "AAAA", env["PUBLIC_IPV6"], "Provides email configuration autodiscovery support for Z-Push ActiveSync Autodiscover."),
-			("autoconfig", "A", env["PUBLIC_IP"], "Provides email configuration autodiscovery support for Thunderbird Autoconfig."),
-			("autoconfig", "AAAA", env["PUBLIC_IPV6"], "Provides email configuration autodiscovery support for Thunderbird Autoconfig.")
-		]
-		for qname, rtype, value, explanation in autodiscover_records:
-			if value is None or value.strip() == "": continue # skip IPV6 if not set
-			if not has_rec(qname, rtype):
-				records.append((qname, rtype, value, explanation))
-
 	# If this is a domain name that there are email addresses configured for, i.e. "something@"
 	# this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461)
 	# Policy Domain.
@@ -321,25 +325,12 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
 	# subdomain must be valid certificate for that domain. Do not set an MTA-STS policy if either
 	# certificate in use is not valid (e.g. because it is self-signed and a valid certificate has not
 	# yet been provisioned). Since we cannot provision a certificate without A/AAAA records, we
-	# always set them --- only the TXT records depend on there being valid certificates.
-	mta_sts_enabled = False
-	mta_sts_records = [
-		("mta-sts", "A", env["PUBLIC_IP"], "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
-		("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
-	]
-	if domain in get_mail_domains(env):
-		# Check that PRIMARY_HOSTNAME and the mta_sts domain both have valid certificates.
-		for d in (env['PRIMARY_HOSTNAME'], "mta-sts." + domain):
-			cert = get_ssl_certificates(env).get(d)
-			if not cert:
-				break # no certificate provisioned for this domain
-			cert_status = check_certificate(d, cert['certificate'], cert['private-key'])
-			if cert_status[0] != 'OK':
-				break # certificate is not valid
-		else:
-			# 'break' was not encountered above, so both domains are good
-			mta_sts_enabled = True
-	if mta_sts_enabled:
+	# always set them (by including them in the www domains) --- only the TXT records depend on there
+	# being valid certificates.
+	mta_sts_records = [ ]
+	if domain_properties[domain]["mail"] \
+	  and domain_properties[env["PRIMARY_HOSTNAME"]]["certificate-is-valid"] \
+	  and is_domain_cert_signed_and_valid("mta-sts." + domain, env):
 		# Compute an up-to-32-character hash of the policy file. We'll take a SHA-1 hash of the policy
 		# file (20 bytes) and encode it as base-64 (28 bytes, using alphanumeric alternate characters
 		# instead of '+' and '/' which are not allowed in an MTA-STS policy id) but then just take its
@@ -356,15 +347,39 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
 		if env.get("MTA_STS_TLSRPT_RUA") and not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"):
 			mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1; rua=" + env["MTA_STS_TLSRPT_RUA"], "Optional. Enables MTA-STS reporting."))
 	for qname, rtype, value, explanation in mta_sts_records:
-		if value is None or value.strip() == "": continue # skip IPV6 if not set
 		if not has_rec(qname, rtype):
 			records.append((qname, rtype, value, explanation))

+	# Add no-mail-here records for any qname that has an A or AAAA record
+	# but no MX record. This would include domain itself if domain is a
+	# non-mail domain and also may include qnames from custom DNS records.
+	# Do this once at the end of generating a zone.
+	if is_zone:
+		qnames_with_a = set(qname for (qname, rtype, value, explanation) in records if rtype in ("A", "AAAA"))
+		qnames_with_mx = set(qname for (qname, rtype, value, explanation) in records if rtype == "MX")
+		for qname in qnames_with_a - qnames_with_mx:
+			# Mark this domain as not sending mail with hard-fail SPF and DMARC records.
+			d = (qname+"." if qname else "") + domain
+			if not has_rec(qname, "TXT", prefix="v=spf1 "):
+				records.append((qname,  "TXT", 'v=spf1 -all', "Recommended. Prevents use of this domain name for outbound mail by specifying that no servers are valid sources for mail from @%s. If you do send email from this domain name you should either override this record such that the SPF rule does allow the originating server, or, take the recommended approach and have the box handle mail for this domain (simply add any receiving alias at this domain name to make this machine treat the domain name as one of its mail domains)." % d))
+			if not has_rec("_dmarc" + ("."+qname if qname else ""), "TXT", prefix="v=DMARC1; "):
+				records.append(("_dmarc" + ("."+qname if qname else ""), "TXT", 'v=DMARC1; p=reject', "Recommended. Prevents use of this domain name for outbound mail by specifying that the SPF rule should be honoured for mail from @%s." % d))
+
+			# And with a null MX record (https://explained-from-first-principles.com/email/#null-mx-record)
+			if not has_rec(qname, "MX"):
+				records.append((qname, "MX", '0 .', "Recommended. Prevents use of this domain name for incoming mail."))
+
 	# Sort the records. The None records *must* go first in the nsd zone file. Otherwise it doesn't matter.
 	records.sort(key = lambda rec : list(reversed(rec[0].split(".")) if rec[0] is not None else ""))

 	return records

+def is_domain_cert_signed_and_valid(domain, env):
+	cert = get_ssl_certificates(env).get(domain)
+	if not cert: return False # no certificate provisioned
+	cert_status = check_certificate(domain, cert['certificate'], cert['private-key'])
+	return cert_status[0] == 'OK'
+
 ########################################################################

 def build_tlsa_record(env):
@@ -429,6 +444,7 @@ def build_sshfp_records():
 	# to the zone file (that trigger bumping the serial number). However,
 	# if SSH has been configured to listen on a nonstandard port, we must
 	# specify that port to sshkeyscan.
+
 	port = 22
 	with open('/etc/ssh/sshd_config', 'r') as f:
 		for line in f:
@@ -439,8 +455,11 @@ def build_sshfp_records():
 				except ValueError:
 					pass
 				break
+
 	keys = shell("check_output", ["ssh-keyscan", "-t", "rsa,dsa,ecdsa,ed25519", "-p", str(port), "localhost"])
-	for key in sorted(keys.split("\n")):
+	keys = sorted(keys.split("\n"))
+
+	for key in keys:
 		if key.strip() == "" or key[0] == "#": continue
 		try:
 			host, keytype, pubkey = key.split(" ")
@@ -460,24 +479,27 @@ def write_nsd_zone(domain, zonefile, records, env, force):
 	# On the $ORIGIN line, there's typically a ';' comment at the end explaining
 	# what the $ORIGIN line does. Any further data after the domain confuses
 	# ldns-signzone, however. It used to say '; default zone domain'.
-
+	#
 	# The SOA contact address for all of the domains on this system is hostmaster
 	# @ the PRIMARY_HOSTNAME. Hopefully that's legit.
-
+	#
 	# For the refresh through TTL fields, a good reference is:
 	# http://www.peerwisdom.org/2013/05/15/dns-understanding-the-soa-record/
-
+	#
+	# A hash of the available DNSSEC keys are added in a comment so that when
+	# the keys change we force a re-generation of the zone which triggers
+	# re-signing it.

 	zone = """
 $ORIGIN {domain}.
-$TTL 1800           ; default time to live
+$TTL 86400          ; default time to live

@ IN SOA ns1.{primary_domain}. hostmaster.{primary_domain}. (
           __SERIAL__     ; serial number
           7200     ; Refresh (secondary nameserver update interval)
-           1800     ; Retry (when refresh fails, how often to try again)
+           86400    ; Retry (when refresh fails, how often to try again)
           1209600  ; Expire (when refresh fails, how long secondary nameserver will keep records around anyway)
-           1800     ; Negative TTL (how long negative responses are cached)
+           86400    ; Negative TTL (how long negative responses are cached)
           )
 """

@@ -502,6 +524,9 @@ $TTL 1800           ; default time to live
 			value = v2
 		zone += value + "\n"

+	# Append a stable hash of DNSSEC signing keys in a comment.
+	zone += "\n; DNSSEC signing keys hash: {}\n".format(hash_dnssec_keys(domain, env))
+
 	# DNSSEC requires re-signing a zone periodically. That requires
 	# bumping the serial number even if no other records have changed.
 	# We don't see the DNSSEC records yet, so we have to figure out
@@ -564,6 +589,17 @@ $TTL 1800           ; default time to live

 	return True # file is updated

+def get_dns_zonefile(zone, env):
+	for domain, fn in get_dns_zones(env):
+		if zone == domain:
+			break
+	else:
+		raise ValueError("%s is not a domain name that corresponds to a zone." % zone)
+
+	nsd_zonefile = "/etc/nsd/zones/" + fn
+	with open(nsd_zonefile, "r") as f:
+		return f.read()
+
 ########################################################################

 def write_nsd_conf(zonefiles, additional_records, env):
@@ -601,53 +637,77 @@ zone:

 ########################################################################

-def dnssec_choose_algo(domain, env):
-	if '.' in domain and domain.rsplit('.')[-1] in \
-		("email", "guide", "fund", "be", "lv"):
-		# At GoDaddy, RSASHA256 is the only algorithm supported
-		# for .email and .guide.
-		# A variety of algorithms are supported for .fund. This
-		# is preferred.
-		# Gandi tells me that .be does not support RSASHA1-NSEC3-SHA1
-        # Nic.lv does not support RSASHA1-NSEC3-SHA1 for .lv tld's
-		return "RSASHA256"
+def find_dnssec_signing_keys(domain, env):
+	# For key that we generated (one per algorithm)...
+	d = os.path.join(env['STORAGE_ROOT'], 'dns/dnssec')
+	keyconfs = [f for f in os.listdir(d) if f.endswith(".conf")]
+	for keyconf in keyconfs:
+		# Load the file holding the KSK and ZSK key filenames.
+		keyconf_fn = os.path.join(d, keyconf)
+		keyinfo = load_env_vars_from_file(keyconf_fn)

-	# For any domain we were able to sign before, don't change the algorithm
-	# on existing users. We'll probably want to migrate to SHA256 later.
-	return "RSASHA1-NSEC3-SHA1"
+		# Skip this key if the conf file has a setting named DOMAINS,
+		# holding a comma-separated list of domain names, and if this
+		# domain is not in the list. This allows easily disabling a
+		# key by setting "DOMAINS=" or "DOMAINS=none", other than
+		# deleting the key's .conf file, which might result in the key
+		# being regenerated next upgrade. Keys should be disabled if
+		# they are not needed to reduce the DNSSEC query response size.
+		if "DOMAINS" in keyinfo and domain not in [dd.strip() for dd in keyinfo["DOMAINS"].split(",")]:
+			continue
+
+		for keytype in ("KSK", "ZSK"):
+			yield keytype, keyinfo[keytype]
+
+def hash_dnssec_keys(domain, env):
+	# Create a stable (by sorting the items) hash of all of the private keys
+	# that will be used to sign this domain.
+	keydata = []
+	for keytype, keyfn in sorted(find_dnssec_signing_keys(domain, env)):
+		oldkeyfn = os.path.join(env['STORAGE_ROOT'], 'dns/dnssec', keyfn + ".private")
+		keydata.append(keytype)
+		keydata.append(keyfn)
+		with open(oldkeyfn, "r") as fr:
+			keydata.append( fr.read() )
+	keydata = "".join(keydata).encode("utf8")
+	return hashlib.sha1(keydata).hexdigest()

 def sign_zone(domain, zonefile, env):
-	algo = dnssec_choose_algo(domain, env)
-	dnssec_keys = load_env_vars_from_file(os.path.join(env['STORAGE_ROOT'], 'dns/dnssec/%s.conf' % algo))
+	# Sign the zone with all of the keys that were generated during
+	# setup so that the user can choose which to use in their DS record at
+	# their registrar, and also to support migration to newer algorithms.

-	# In order to use the same keys for all domains, we have to generate
-	# a new .key file with a DNSSEC record for the specific domain. We
-	# can reuse the same key, but it won't validate without a DNSSEC
-	# record specifically for the domain.
+	# In order to use the key files generated at setup which are for
+	# the domain _domain_, we have to re-write the files and place
+	# the actual domain name in it, so that ldns-signzone works.
 	#
+	# Patch each key, storing the patched version in /tmp for now.
+	# Each key has a .key and .private file. Collect a list of filenames
+	# for all of the keys (and separately just the key-signing keys).
+	all_keys = []
+	ksk_keys = []
+	for keytype, keyfn in find_dnssec_signing_keys(domain, env):
+		newkeyfn = '/tmp/' + keyfn.replace("_domain_", domain)
+
+		for ext in (".private", ".key"):
 			# Copy the .key and .private files to /tmp to patch them up.
 			#
 			# Use os.umask and open().write() to securely create a copy that only
 			# we (root) can read.
-	files_to_kill = []
-	for key in ("KSK", "ZSK"):
-		if dnssec_keys.get(key, "").strip() == "": raise Exception("DNSSEC is not properly set up.")
-		oldkeyfn = os.path.join(env['STORAGE_ROOT'], 'dns/dnssec/' + dnssec_keys[key])
-		newkeyfn = '/tmp/' + dnssec_keys[key].replace("_domain_", domain)
-		dnssec_keys[key] = newkeyfn
-		for ext in (".private", ".key"):
-			if not os.path.exists(oldkeyfn + ext): raise Exception("DNSSEC is not properly set up.")
-			with open(oldkeyfn + ext, "r") as fr:
+			oldkeyfn = os.path.join(env['STORAGE_ROOT'], 'dns/dnssec', keyfn + ext)
+			with open(oldkeyfn, "r") as fr:
 				keydata = fr.read()
-			keydata = keydata.replace("_domain_", domain) # trick ldns-signkey into letting our generic key be used by this zone
-			fn = newkeyfn + ext
+			keydata = keydata.replace("_domain_", domain)
 			prev_umask = os.umask(0o77) # ensure written file is not world-readable
 			try:
-				with open(fn, "w") as fw:
+				with open(newkeyfn + ext, "w") as fw:
 					fw.write(keydata)
 			finally:
 				os.umask(prev_umask) # other files we write should be world-readable
-			files_to_kill.append(fn)
+
+		# Put the patched key filename base (without extension) into the list of keys we'll sign with.
+		all_keys.append(newkeyfn)
+		if keytype == "KSK": ksk_keys.append(newkeyfn)

 	# Do the signing.
 	expiry_date = (datetime.datetime.now() + datetime.timedelta(days=30)).strftime("%Y%m%d")
@@ -660,32 +720,34 @@ def sign_zone(domain, zonefile, env):

 		# zonefile to sign
 		"/etc/nsd/zones/" + zonefile,
-
+	]
 		# keys to sign with (order doesn't matter -- it'll figure it out)
-		dnssec_keys["KSK"],
-		dnssec_keys["ZSK"],
-	])
+		+ all_keys
+	)

 	# Create a DS record based on the patched-up key files. The DS record is specific to the
 	# zone being signed, so we can't use the .ds files generated when we created the keys.
 	# The DS record points to the KSK only. Write this next to the zone file so we can
 	# get it later to give to the user with instructions on what to do with it.
 	#
-	# We want to be able to validate DS records too, but multiple forms may be valid depending
-	# on the digest type. So we'll write all (both) valid records. Only one DS record should
-	# actually be deployed. Preferebly the first.
+	# Generate a DS record for each key. There are also several possible hash algorithms that may
+	# be used, so we'll pre-generate all for each key. One DS record per line. Only one
+	# needs to actually be deployed at the registrar. We'll select the preferred one
+	# in the status checks.
 	with open("/etc/nsd/zones/" + zonefile + ".ds", "w") as f:
-		for digest_type in ('2', '1'):
+		for key in ksk_keys:
+			for digest_type in ('1', '2', '4'):
 				rr_ds = shell('check_output', ["/usr/bin/ldns-key2ds",
 					"-n", # output to stdout
-				"-" + digest_type, # 1=SHA1, 2=SHA256
-				dnssec_keys["KSK"] + ".key"
+					"-" + digest_type, # 1=SHA1, 2=SHA256, 4=SHA384
+					key + ".key"
 				])
 				f.write(rr_ds)

-	# Remove our temporary file.
-	for fn in files_to_kill:
-		os.unlink(fn)
+	# Remove the temporary patched key files.
+	for fn in all_keys:
+		os.unlink(fn + ".private")
+		os.unlink(fn + ".key")

 ########################################################################

@@ -742,7 +804,7 @@ def write_opendkim_tables(domains, env):

 ########################################################################

-def get_custom_dns_config(env):
+def get_custom_dns_config(env, only_real_records=False):
 	try:
 		custom_dns = rtyaml.load(open(os.path.join(env['STORAGE_ROOT'], 'dns/custom.yaml')))
 		if not isinstance(custom_dns, dict): raise ValueError() # caught below
@@ -750,6 +812,8 @@ def get_custom_dns_config(env):
 		return [ ]

 	for qname, value in custom_dns.items():
+		if qname == "_secondary_nameserver" and only_real_records: continue # skip fake record
+
 		# Short form. Mapping a domain name to a string is short-hand
 		# for creating A records.
 		if isinstance(value, str):
--- a/management/mail_log.py
+++ b/management/mail_log.py
@@ -44,9 +44,8 @@ TIME_DELTAS = OrderedDict([
    ('today', datetime.datetime.now() - datetime.datetime.now().replace(hour=0, minute=0, second=0))
 ])

-# Start date > end date!
-START_DATE = datetime.datetime.now()
-END_DATE = None
+END_DATE = NOW = datetime.datetime.now()
+START_DATE = None

 VERBOSE = False

@@ -121,7 +120,7 @@ def scan_mail_log(env):
        pass

    print("Scanning logs from {:%Y-%m-%d %H:%M:%S} to {:%Y-%m-%d %H:%M:%S}".format(
-        END_DATE, START_DATE)
+        START_DATE, END_DATE)
    )

    # Scan the lines in the log files until the date goes out of range
@@ -253,7 +252,7 @@ def scan_mail_log(env):

    if collector["postgrey"]:
        msg = "Greylisted Email {:%Y-%m-%d %H:%M:%S} and {:%Y-%m-%d %H:%M:%S}"
-        print_header(msg.format(END_DATE, START_DATE))
+        print_header(msg.format(START_DATE, END_DATE))

        print(textwrap.fill(
            "The following mail was greylisted, meaning the emails were temporarily rejected. "
@@ -291,7 +290,7 @@ def scan_mail_log(env):

    if collector["rejected"]:
        msg = "Blocked Email {:%Y-%m-%d %H:%M:%S} and {:%Y-%m-%d %H:%M:%S}"
-        print_header(msg.format(END_DATE, START_DATE))
+        print_header(msg.format(START_DATE, END_DATE))

        data = OrderedDict(sorted(collector["rejected"].items(), key=email_sort))

@@ -345,19 +344,19 @@ def scan_mail_log_line(line, collector):
    # Replaced the dateutil parser for a less clever way of parser that is roughly 4 times faster.
    # date = dateutil.parser.parse(date)
    
-    # date = datetime.datetime.strptime(date, '%b %d %H:%M:%S')
-    # date = date.replace(START_DATE.year)
-
-    # strptime fails on Feb 29 if correct year is not provided. See https://bugs.python.org/issue26460
-    date = datetime.datetime.strptime(str(START_DATE.year) + ' ' + date, '%Y %b %d %H:%M:%S')
+    # strptime fails on Feb 29 with ValueError: day is out of range for month if correct year is not provided.
+    # See https://bugs.python.org/issue26460
+    date = datetime.datetime.strptime(str(NOW.year) + ' ' + date, '%Y %b %d %H:%M:%S')
+    # if log date in future, step back a year
+    if date > NOW:
+      date = date.replace(year = NOW.year - 1)
    #print("date:", date)

    # Check if the found date is within the time span we are scanning
-    # END_DATE < START_DATE
-    if date > START_DATE:
+    if date > END_DATE:
        # Don't process, and halt
        return False
-    elif date < END_DATE:
+    elif date < START_DATE:
        # Don't process, but continue
        return True

@@ -606,7 +605,7 @@ def email_sort(email):


 def valid_date(string):
-    """ Validate the given date string fetched from the --startdate argument """
+    """ Validate the given date string fetched from the --enddate argument """
    try:
        date = dateutil.parser.parse(string)
    except ValueError:
@@ -820,12 +819,14 @@ if __name__ == "__main__":

    parser.add_argument("-t", "--timespan", choices=TIME_DELTAS.keys(), default='today',
                        metavar='<time span>',
-                        help="Time span to scan, going back from the start date. Possible values: "
+                        help="Time span to scan, going back from the end date. Possible values: "
                             "{}. Defaults to 'today'.".format(", ".join(list(TIME_DELTAS.keys()))))
-    parser.add_argument("-d", "--startdate",  action="store", dest="startdate",
-                        type=valid_date, metavar='<start date>',
-                        help="Date and time to start scanning the log file from. If no date is "
-                             "provided, scanning will start from the current date and time.")
+    # keep the --startdate arg for backward compatibility
+    parser.add_argument("-d", "--enddate", "--startdate",  action="store", dest="enddate",
+                        type=valid_date, metavar='<end date>',
+                        help="Date and time to end scanning the log file. If no date is "
+                             "provided, scanning will end at the current date and time. "
+                             "Alias --startdate is for compatibility.")
    parser.add_argument("-u", "--users", action="store", dest="users",
                        metavar='<email1,email2,email...>',
                        help="Comma separated list of (partial) email addresses to filter the "
@@ -837,13 +838,13 @@ if __name__ == "__main__":

    args = parser.parse_args()

-    if args.startdate is not None:
-        START_DATE = args.startdate
+    if args.enddate is not None:
+        END_DATE = args.enddate
        if args.timespan == 'today':
            args.timespan = 'day'
-        print("Setting start date to {}".format(START_DATE))
+        print("Setting end date to {}".format(END_DATE))

-    END_DATE = START_DATE - TIME_DELTAS[args.timespan]
+    START_DATE = END_DATE - TIME_DELTAS[args.timespan]

    VERBOSE = args.verbose

--- a/management/mailconfig.py
+++ b/management/mailconfig.py
@@ -608,7 +608,6 @@ def validate_password(pw):
 	if len(pw) < 8:
 		raise ValueError("Passwords must be at least eight characters.")

-
 if __name__ == "__main__":
 	import sys
 	if len(sys.argv) > 2 and sys.argv[1] == "validate-email":
--- a/management/mfa.py
+++ b/management/mfa.py
@@ -0,0 +1,141 @@
+import base64
+import hmac
+import io
+import os
+import pyotp
+import qrcode
+
+from mailconfig import open_database
+
+def get_user_id(email, c):
+	c.execute('SELECT id FROM users WHERE email=?', (email,))
+	r = c.fetchone()
+	if not r: raise ValueError("User does not exist.")
+	return r[0]
+
+def get_mfa_state(email, env):
+	c = open_database(env)
+	c.execute('SELECT id, type, secret, mru_token, label FROM mfa WHERE user_id=?', (get_user_id(email, c),))
+	return [
+		{ "id": r[0], "type": r[1], "secret": r[2], "mru_token": r[3], "label": r[4] }
+		for r in c.fetchall()
+	]
+
+def get_public_mfa_state(email, env):
+	mfa_state = get_mfa_state(email, env)
+	return [
+		{ "id": s["id"], "type": s["type"], "label": s["label"] }
+		for s in mfa_state
+	]
+
+def get_hash_mfa_state(email, env):
+	mfa_state = get_mfa_state(email, env)
+	return [
+		{ "id": s["id"], "type": s["type"], "secret": s["secret"] }
+		for s in mfa_state
+	]
+
+def enable_mfa(email, type, secret, token, label, env):
+	if type == "totp":
+		validate_totp_secret(secret)
+		# Sanity check with the provide current token.
+		totp = pyotp.TOTP(secret)
+		if not totp.verify(token, valid_window=1):
+			raise ValueError("Invalid token.")
+	else:
+		raise ValueError("Invalid MFA type.")
+
+	conn, c = open_database(env, with_connection=True)
+	c.execute('INSERT INTO mfa (user_id, type, secret, label) VALUES (?, ?, ?, ?)', (get_user_id(email, c), type, secret, label))
+	conn.commit()
+
+def set_mru_token(email, mfa_id, token, env):
+	conn, c = open_database(env, with_connection=True)
+	c.execute('UPDATE mfa SET mru_token=? WHERE user_id=? AND id=?', (token, get_user_id(email, c), mfa_id))
+	conn.commit()
+
+def disable_mfa(email, mfa_id, env):
+	conn, c = open_database(env, with_connection=True)
+	if mfa_id is None:
+		# Disable all MFA for a user.
+		c.execute('DELETE FROM mfa WHERE user_id=?', (get_user_id(email, c),))
+	else:
+		# Disable a particular MFA mode for a user.
+		c.execute('DELETE FROM mfa WHERE user_id=? AND id=?', (get_user_id(email, c), mfa_id))
+	conn.commit()
+	return c.rowcount > 0
+
+def validate_totp_secret(secret):
+	if type(secret) != str or secret.strip() == "":
+		raise ValueError("No secret provided.")
+	if len(secret) != 32:
+		raise ValueError("Secret should be a 32 characters base32 string")
+
+def provision_totp(email, env):
+	# Make a new secret.
+	secret = base64.b32encode(os.urandom(20)).decode('utf-8')
+	validate_totp_secret(secret) # sanity check
+
+	# Make a URI that we encode within a QR code.
+	uri = pyotp.TOTP(secret).provisioning_uri(
+		name=email,
+		issuer_name=env["PRIMARY_HOSTNAME"] + " Mail-in-a-Box Control Panel"
+	)
+
+	# Generate a QR code as a base64-encode PNG image.
+	qr = qrcode.make(uri)
+	byte_arr = io.BytesIO()
+	qr.save(byte_arr, format='PNG')
+	png_b64 = base64.b64encode(byte_arr.getvalue()).decode('utf-8')
+
+	return {
+		"type": "totp",
+		"secret": secret,
+		"qr_code_base64": png_b64
+	}
+
+def validate_auth_mfa(email, request, env):
+	# Validates that a login request satisfies any MFA modes
+	# that have been enabled for the user's account. Returns
+	# a tuple (status, [hints]). status is True for a successful
+	# MFA login, False for a missing token. If status is False,
+	# hints is an array of codes that indicate what the user
+	# can try. Possible codes are:
+	# "missing-totp-token"
+	# "invalid-totp-token"
+
+	mfa_state = get_mfa_state(email, env)
+
+	# If no MFA modes are added, return True.
+	if len(mfa_state) == 0:
+		return (True, [])
+
+	# Try the enabled MFA modes.
+	hints = set()
+	for mfa_mode in mfa_state:
+		if mfa_mode["type"] == "totp":
+			# Check that a token is present in the X-Auth-Token header.
+			# If not, give a hint that one can be supplied.
+			token = request.headers.get('x-auth-token')
+			if not token:
+				hints.add("missing-totp-token")
+				continue
+
+			# Check for a replay attack.
+			if hmac.compare_digest(token, mfa_mode['mru_token'] or ""):
+				# If the token fails, skip this MFA mode.
+				hints.add("invalid-totp-token")
+				continue
+
+			# Check the token.
+			totp = pyotp.TOTP(mfa_mode["secret"])
+			if not totp.verify(token, valid_window=1):
+				hints.add("invalid-totp-token")
+				continue
+
+			# On success, record the token to prevent a replay attack.
+			set_mru_token(email, mfa_mode['id'], token, env)
+			return (True, [])
+
+	# On a failed login, indicate failure and any hints for what the user can do instead.
+	return (False, list(hints))
--- a/management/ssl_certificates.py
+++ b/management/ssl_certificates.py
@@ -606,10 +606,10 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
 		ndays = (cert_expiration_date-now).days
 		if not rounded_time or ndays <= 10:
 			# Yikes better renew soon!
-			expiry_info = "The certificate expires in %d days on %s." % (ndays, cert_expiration_date.strftime("%x"))
+			expiry_info = "The certificate expires in %d days on %s." % (ndays, cert_expiration_date.date().isoformat())
 		else:
 			# We'll renew it with Lets Encrypt.
-			expiry_info = "The certificate expires on %s." % cert_expiration_date.strftime("%x")
+			expiry_info = "The certificate expires on %s." % cert_expiration_date.date().isoformat()

 		if warn_if_expiring_soon and ndays <= warn_if_expiring_soon:
 			# Warn on day 10 to give 4 days for us to automatically renew the
--- a/management/status_checks.py
+++ b/management/status_checks.py
@@ -34,6 +34,7 @@ def get_services():
 		{ "name": "SSH Login (ssh)", "port": get_ssh_port(), "public": True, },
 		{ "name": "Public DNS (nsd4)", "port": 53, "public": True, },
 		{ "name": "Incoming Mail (SMTP/postfix)", "port": 25, "public": True, },
+		{ "name": "Outgoing Mail (SMTP 465/postfix)", "port": 465, "public": True, },
 		{ "name": "Outgoing Mail (SMTP 587/postfix)", "port": 587, "public": True, },
 		#{ "name": "Postfix/master", "port": 10587, "public": True, },
 		{ "name": "IMAPS (dovecot)", "port": 993, "public": True, },
@@ -42,7 +43,7 @@ def get_services():
 		{ "name": "HTTPS Web (nginx)", "port": 443, "public": True, },
 	]

-def run_checks(rounded_values, env, output, pool):
+def run_checks(rounded_values, env, output, pool, domains_to_check=None):
 	# run systems checks
 	output.add_heading("System")

@@ -63,7 +64,7 @@ def run_checks(rounded_values, env, output, pool):
 	# perform other checks asynchronously

 	run_network_checks(env, output)
-	run_domain_checks(rounded_values, env, output, pool)
+	run_domain_checks(rounded_values, env, output, pool, domains_to_check=domains_to_check)

 def get_ssh_port():
 	# Returns ssh port
@@ -293,12 +294,14 @@ def run_network_checks(env, output):
 	zen = query_dns(rev_ip4+'.zen.spamhaus.org', 'A', nxdomain=None)
 	if zen is None:
 		output.print_ok("IP address is not blacklisted by zen.spamhaus.org.")
+	elif zen == "[timeout]":
+		output.print_warning("Connection to zen.spamhaus.org timed out. We could not determine whether your server's IP address is blacklisted. Please try again later.")
 	else:
 		output.print_error("""The IP address of this machine %s is listed in the Spamhaus Block List (code %s),
 			which may prevent recipients from receiving your email. See http://www.spamhaus.org/query/ip/%s."""
 			% (env['PUBLIC_IP'], zen, env['PUBLIC_IP']))

-def run_domain_checks(rounded_time, env, output, pool):
+def run_domain_checks(rounded_time, env, output, pool, domains_to_check=None):
 	# Get the list of domains we handle mail for.
 	mail_domains = get_mail_domains(env)

@@ -309,6 +312,7 @@ def run_domain_checks(rounded_time, env, output, pool):
 	# Get the list of domains we serve HTTPS for.
 	web_domains = set(get_web_domains(env))

+	if domains_to_check is None:
 		domains_to_check = mail_domains | dns_domains | web_domains

 	# Remove "www", "autoconfig", "autodiscover", and "mta-sts" subdomains, which we group with their parent,
@@ -555,61 +559,103 @@ def check_dns_zone_suggestions(domain, env, output, dns_zonefiles, domains_with_


 def check_dnssec(domain, env, output, dns_zonefiles, is_checking_primary=False):
-	# See if the domain has a DS record set at the registrar. The DS record may have
-	# several forms. We have to be prepared to check for any valid record. We've
-	# pre-generated all of the valid digests --- read them in.
+	# See if the domain has a DS record set at the registrar. The DS record must
+	# match one of the keys that we've used to sign the zone. It may use one of
+	# several hashing algorithms. We've pre-generated all possible valid DS
+	# records, although some will be preferred.
+
+	alg_name_map = { '7': 'RSASHA1-NSEC3-SHA1', '8': 'RSASHA256', '13': 'ECDSAP256SHA256' }
+	digalg_name_map = { '1': 'SHA-1', '2': 'SHA-256', '4': 'SHA-384' }
+
+	# Read in the pre-generated DS records
+	expected_ds_records = { }
 	ds_file = '/etc/nsd/zones/' + dns_zonefiles[domain] + '.ds'
 	if not os.path.exists(ds_file): return # Domain is in our database but DNS has not yet been updated.
-	ds_correct = open(ds_file).read().strip().split("\n")
-	digests = { }
-	for rr_ds in ds_correct:
+	with open(ds_file) as f:
+		for rr_ds in f:
+			rr_ds = rr_ds.rstrip()
 			ds_keytag, ds_alg, ds_digalg, ds_digest = rr_ds.split("\t")[4].split(" ")
-		digests[ds_digalg] = ds_digest

 			# Some registrars may want the public key so they can compute the digest. The DS
 			# record that we suggest using is for the KSK (and that's how the DS records were generated).
-	alg_name_map = { '7': 'RSASHA1-NSEC3-SHA1', '8': 'RSASHA256' }
+			# We'll also give the nice name for the key algorithm.
 			dnssec_keys = load_env_vars_from_file(os.path.join(env['STORAGE_ROOT'], 'dns/dnssec/%s.conf' % alg_name_map[ds_alg]))
 			dnsssec_pubkey = open(os.path.join(env['STORAGE_ROOT'], 'dns/dnssec/' + dnssec_keys['KSK'] + '.key')).read().split("\t")[3].split(" ")[3]

+			expected_ds_records[ (ds_keytag, ds_alg, ds_digalg, ds_digest) ] = {
+				"record": rr_ds,
+				"keytag": ds_keytag,
+				"alg": ds_alg,
+				"alg_name": alg_name_map[ds_alg],
+				"digalg": ds_digalg,
+				"digalg_name": digalg_name_map[ds_digalg],
+				"digest": ds_digest,
+				"pubkey": dnsssec_pubkey,
+			}
+
 	# Query public DNS for the DS record at the registrar.
-	ds = query_dns(domain, "DS", nxdomain=None)
-	ds_looks_valid = ds and len(ds.split(" ")) == 4
-	if ds_looks_valid: ds = ds.split(" ")
-	if ds_looks_valid and ds[0] == ds_keytag and ds[1] == ds_alg and ds[3] == digests.get(ds[2]):
-		if is_checking_primary: return
-		output.print_ok("DNSSEC 'DS' record is set correctly at registrar.")
+	ds = query_dns(domain, "DS", nxdomain=None, as_list=True)
+	if ds is None or isinstance(ds, str): ds = []
+
+	# There may be more that one record, so we get the result as a list.
+	# Filter out records that don't look valid, just in case, and split
+	# each record on spaces.
+	ds = [tuple(str(rr).split(" ")) for rr in ds if len(str(rr).split(" ")) == 4]
+
+	if len(ds) == 0:
+		output.print_warning("""This domain's DNSSEC DS record is not set. The DS record is optional. The DS record activates DNSSEC. See below for instructions.""")
 	else:
-		if ds == None:
-			if is_checking_primary: return
-			output.print_warning("""This domain's DNSSEC DS record is not set. The DS record is optional. The DS record activates DNSSEC.
-				To set a DS record, you must follow the instructions provided by your domain name registrar and provide to them this information:""")
+		matched_ds = set(ds) & set(expected_ds_records)
+		if matched_ds:
+			# At least one DS record matches one that corresponds with one of the ways we signed
+			# the zone, so it is valid.
+			#
+			# But it may not be preferred. Only algorithm 13 is preferred. Warn if any of the
+			# matched zones uses a different algorithm.
+			if set(r[1] for r in matched_ds) == { '13' }: # all are alg 13
+				output.print_ok("DNSSEC 'DS' record is set correctly at registrar.")
+				return
+			elif '13' in set(r[1] for r in matched_ds): # some but not all are alg 13
+				output.print_ok("DNSSEC 'DS' record is set correctly at registrar. (Records using algorithm other than ECDSAP256SHA256 should be removed.)")
+				return
+			else: # no record uses alg 13
+				output.print_warning("DNSSEC 'DS' record set at registrar is valid but should be updated to ECDSAP256SHA256 (see below).")
 		else:
 			if is_checking_primary:
 				output.print_error("""The DNSSEC 'DS' record for %s is incorrect. See further details below.""" % domain)
 				return
 			output.print_error("""This domain's DNSSEC DS record is incorrect. The chain of trust is broken between the public DNS system
 				and this machine's DNS server. It may take several hours for public DNS to update after a change. If you did not recently
-				make a change, you must resolve this immediately by following the instructions provided by your domain name registrar and
-				provide to them this information:""")
+				make a change, you must resolve this immediately (see below).""")
+
+	output.print_line("""Follow the instructions provided by your domain name registrar to set a DS record.
+		Registrars support different sorts of DS records. Use the first option that works:""")
+	preferred_ds_order = [(7, 1), (7, 2), (8, 4), (13, 4), (8, 1), (8, 2), (13, 1), (13, 2)] # low to high
+	def preferred_ds_order_func(ds_suggestion):
+		k = (int(ds_suggestion['alg']), int(ds_suggestion['digalg']))
+		if k in preferred_ds_order:
+			return preferred_ds_order.index(k)
+		return -1 # index before first item
 	output.print_line("")
-		output.print_line("Key Tag: " + ds_keytag + ("" if not ds_looks_valid or ds[0] == ds_keytag else " (Got '%s')" % ds[0]))
+	for i, ds_suggestion in enumerate(sorted(expected_ds_records.values(), key=preferred_ds_order_func, reverse=True)):
+		output.print_line("")
+		output.print_line("Option " + str(i+1) + ":")
+		output.print_line("----------")
+		output.print_line("Key Tag: " + ds_suggestion['keytag'])
 		output.print_line("Key Flags: KSK")
-		output.print_line(
-			  ("Algorithm: %s / %s" % (ds_alg, alg_name_map[ds_alg]))
-			+ ("" if not ds_looks_valid or ds[1] == ds_alg else " (Got '%s')" % ds[1]))
-			# see http://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml
-		output.print_line("Digest Type: 2 / SHA-256")
-			# http://www.ietf.org/assignments/ds-rr-types/ds-rr-types.xml
-		output.print_line("Digest: " + digests['2'])
-		if ds_looks_valid and ds[3] != digests.get(ds[2]):
-			output.print_line("(Got digest type %s and digest %s which do not match.)" % (ds[2], ds[3]))
+		output.print_line("Algorithm: %s / %s" % (ds_suggestion['alg'], ds_suggestion['alg_name']))
+		output.print_line("Digest Type: %s / %s" % (ds_suggestion['digalg'], ds_suggestion['digalg_name']))
+		output.print_line("Digest: " + ds_suggestion['digest'])
 		output.print_line("Public Key: ")
-		output.print_line(dnsssec_pubkey, monospace=True)
+		output.print_line(ds_suggestion['pubkey'], monospace=True)
 		output.print_line("")
 		output.print_line("Bulk/Record Format:")
-		output.print_line("" + ds_correct[0])
+		output.print_line(ds_suggestion['record'], monospace=True)
+	if len(ds) > 0:
 		output.print_line("")
+		output.print_line("The DS record is currently set to:")
+		for rr in ds:
+			output.print_line("Key Tag: {0}, Algorithm: {1}, Digest Type: {2}, Digest: {3}".format(*rr))

 def check_mail_domain(domain, env, output):
 	# Check the MX record.
@@ -619,6 +665,8 @@ def check_mail_domain(domain, env, output):

 	if mx is None:
 		mxhost = None
+	elif mx == "[timeout]":
+		mxhost = None
 	else:
 		# query_dns returns a semicolon-delimited list
 		# of priority-host pairs.
@@ -678,6 +726,8 @@ def check_mail_domain(domain, env, output):
 	dbl = query_dns(domain+'.dbl.spamhaus.org', "A", nxdomain=None)
 	if dbl is None:
 		output.print_ok("Domain is not blacklisted by dbl.spamhaus.org.")
+	elif dbl == "[timeout]":
+		output.print_warning("Connection to dbl.spamhaus.org timed out. We could not determine whether the domain {} is blacklisted. Please try again later.".format(domain))
 	else:
 		output.print_error("""This domain is listed in the Spamhaus Domain Block List (code %s),
 			which may prevent recipients from receiving your mail.
@@ -709,7 +759,7 @@ def check_web_domain(domain, rounded_time, ssl_certificates, env, output):
 	# website for also needs a signed certificate.
 	check_ssl_cert(domain, rounded_time, ssl_certificates, env, output)

-def query_dns(qname, rtype, nxdomain='[Not Set]', at=None):
+def query_dns(qname, rtype, nxdomain='[Not Set]', at=None, as_list=False):
 	# Make the qname absolute by appending a period. Without this, dns.resolver.query
 	# will fall back a failed lookup to a second query with this machine's hostname
 	# appended. This has been causing some false-positive Spamhaus reports. The
@@ -746,6 +796,9 @@ def query_dns(qname, rtype, nxdomain='[Not Set]', at=None):
 	if rtype in ("A", "AAAA"):
 		response = [normalize_ip(str(r)) for r in response]

+	if as_list:
+		return response
+
 	# There may be multiple answers; concatenate the response. Remove trailing
 	# periods from responses since that's how qnames are encoded in DNS but is
 	# confusing for us. The order of the answers doesn't matter, so sort so we
@@ -1046,3 +1099,7 @@ if __name__ == "__main__":

 	elif sys.argv[1] == "--version":
 		print(what_version_is_this(env))
+
+	elif sys.argv[1] == "--only":
+		with multiprocessing.pool.Pool(processes=10) as pool:
+			run_checks(False, env, ConsoleOutput(), pool, domains_to_check=sys.argv[2:])
--- a/management/templates/aliases.html
+++ b/management/templates/aliases.html
@@ -153,8 +153,8 @@ function show_aliases() {
    function(r) {
      $('#alias_table tbody').html("");
      for (var i = 0; i < r.length; i++) {
-        var hdr = $("<tr><td colspan='3'><h4/></td></tr>");
-        hdr.find('h4').text(r[i].domain);
+        var hdr = $("<tr><th colspan='4' style='background-color: #EEE'></th></tr>");
+        hdr.find('th').text(r[i].domain);
        $('#alias_table tbody').append(hdr);

        for (var k = 0; k < r[i].aliases.length; k++) {
--- a/management/templates/custom-dns.html
+++ b/management/templates/custom-dns.html
@@ -57,7 +57,13 @@
  </div>
 </form>

-<table id="custom-dns-current" class="table" style="width: auto; display: none">
+<div style="text-align: right; font-size; 90%; margin-top: 1em;">
+  sort by:
+  <a href="#" onclick="window.miab_custom_dns_data_sort_order='qname'; show_current_custom_dns_update_after_sort(); return false;">domain name</a>
+  |
+  <a href="#" onclick="window.miab_custom_dns_data_sort_order='created'; show_current_custom_dns_update_after_sort(); return false;">created</a>
+</div>
+<table id="custom-dns-current" class="table" style="width: auto; display: none; margin-top: 0;">
  <thead>
    <th>Domain Name</th>
    <th>Record Type</th>
@@ -192,36 +198,38 @@ function show_current_custom_dns() {
        $('#custom-dns-current').fadeIn();
      else
        $('#custom-dns-current').fadeOut();
-        
-      var reverse_fqdn = function(el) {
-        el.qname = el.qname.split('.').reverse().join('.');
-        return el;
-      }
-      var sort = function(a, b) {
-        if(a.qname === b.qname) {
-          if(a.rtype === b.rtype) {
-            return a.value > b.value ? 1 : -1;
-          }
-          return a.rtype > b.rtype ? 1 : -1;
-        }
-        return a.qname > b.qname ? 1 : -1;
+      window.miab_custom_dns_data = data;
+      show_current_custom_dns_update_after_sort();
+    });
 }

-      data = data.map(reverse_fqdn).sort(sort).map(reverse_fqdn);
+function show_current_custom_dns_update_after_sort() {
+      var data = window.miab_custom_dns_data;
+      var sort_key = window.miab_custom_dns_data_sort_order || "qname";

-      $('#custom-dns-current').find("tbody").text('');
+      data.sort(function(a, b) { return a["sort-order"][sort_key] - b["sort-order"][sort_key] });
+
+      var tbody = $('#custom-dns-current').find("tbody");
+      tbody.text('');
+      var last_zone = null;
      for (var i = 0; i < data.length; i++) {
+        if (sort_key == "qname" && data[i].zone != last_zone) {
+          var r = $("<tr><th colspan=4 style='background-color: #EEE'></th></tr>");
+          r.find("th").text(data[i].zone);
+          tbody.append(r);
+          last_zone = data[i].zone;
+        }
+
        var tr = $("<tr/>");
-        $('#custom-dns-current').find("tbody").append(tr);
+        tbody.append(tr);
        tr.attr('data-qname', data[i].qname);
        tr.attr('data-rtype', data[i].rtype);
        tr.attr('data-value', data[i].value);
        tr.append($('<td class="long"/>').text(data[i].qname));
        tr.append($('<td/>').text(data[i].rtype));
-        tr.append($('<td class="long"/>').text(data[i].value));
+        tr.append($('<td class="long" style="max-width: 40em"/>').text(data[i].value));
        tr.append($('<td>[<a href="#" onclick="return delete_custom_dns_record(this)">delete</a>]</td>'));
      }
-    });
 }

 function delete_custom_dns_record(elem) {
--- a/management/templates/external-dns.html
+++ b/management/templates/external-dns.html
@@ -42,6 +42,19 @@
 	You may need to adopt this technique when adding DomainKeys. Use a tool like <code>named-checkzone</code> to validate your zone file.
 </p>

+<h3>Download zonefile</h3>
+<p>You can download your zonefiles here or use the table of records below.</p>
+<form class="form-inline" role="form" onsubmit="do_download_zonefile(); return false;">
+    <div class="form-group">
+        <div class="form-group">
+            <label for="downloadZonefile" class="control-label sr-only">Zone</label>
+            <select id="downloadZonefile" class="form-control" style="width: auto"> </select>
+        </div>
+        <button type="submit" class="btn btn-primary">Download</button>
+    </div>
+</form>
+
+<h3>Records</h3>

 <table id="external_dns_settings" class="table">
 	<thead>
@@ -57,6 +70,18 @@

 <script>
 function show_external_dns() {
+  api(
+    "/dns/zones",
+    "GET",
+    { },
+    function(data) {
+        var zones = $('#downloadZonefile');
+        zones.text('');
+        for (var j = 0; j < data.length; j++) {
+            zones.append($('<option/>').text(data[j]));
+        }
+    });
+
  $('#external_dns_settings tbody').html("<tr><td colspan='2' class='text-muted'>Loading...</td></tr>")
  api(
    "/dns/dump",
@@ -84,4 +109,19 @@ function show_external_dns() {
      }
    })
 }
+
+function do_download_zonefile() {
+    var zone = $('#downloadZonefile').val();
+
+    api(
+        "/dns/zonefile/"+ zone,
+        "GET",
+        {},
+        function(data) {
+            show_modal_error("Download Zonefile", $("<pre/>").text(data));
+        },
+        function(err) {
+            show_modal_error("Download Zonefile (Error)", $("<pre/>").text(err));
+        });
+}
 </script>
--- a/management/templates/index.html
+++ b/management/templates/index.html
@@ -97,11 +97,14 @@
              </ul>
            </li>
            <li class="dropdown">
-              <a href="#" class="dropdown-toggle" data-toggle="dropdown">Mail <b class="caret"></b></a>
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown">Mail &amp; Users <b class="caret"></b></a>
              <ul class="dropdown-menu">
                <li><a href="#mail-guide" onclick="return show_panel(this);">Instructions</a></li>
                <li><a href="#users" onclick="return show_panel(this);">Users</a></li>
                <li><a href="#aliases" onclick="return show_panel(this);">Aliases</a></li>
+                <li class="divider"></li>
+                <li class="dropdown-header">Your Account</li>
+                <li><a href="#mfa" onclick="return show_panel(this);">Two-Factor Authentication</a></li>
              </ul>
            </li>
            <li><a href="#sync_guide" onclick="return show_panel(this);">Contacts/Calendar</a></li>
@@ -131,6 +134,10 @@
      {% include "custom-dns.html" %}
      </div>

+      <div id="panel_mfa" class="admin_panel">
+      {% include "mfa.html" %}
+      </div>
+
      <div id="panel_login" class="admin_panel">
      {% include "login.html" %}
      </div>
@@ -292,7 +299,7 @@ function ajax_with_indicator(options) {
 }

 var api_credentials = ["", ""];
-function api(url, method, data, callback, callback_error) {
+function api(url, method, data, callback, callback_error, headers) {
  // from http://www.webtoolkit.info/javascript-base64.html
  function base64encode(input) {
    _keyStr = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
@@ -330,7 +337,7 @@ function api(url, method, data, callback, callback_error) {
    method: method,
    cache: false,
    data: data,
-
+    headers: headers,
    // the custom DNS api sends raw POST/PUT bodies --- prevent URL-encoding
    processData: typeof data != "string",
    mimeType: typeof data == "string" ? "text/plain; charset=ascii" : null,
@@ -358,6 +365,16 @@ function api(url, method, data, callback, callback_error) {

 var current_panel = null;
 var switch_back_to_panel = null;
+
+function do_logout() {
+  api_credentials = ["", ""];
+  if (typeof localStorage != 'undefined')
+    localStorage.removeItem("miab-cp-credentials");
+  if (typeof sessionStorage != 'undefined')
+    sessionStorage.removeItem("miab-cp-credentials");
+  show_panel('login');
+}
+
 function show_panel(panelid) {
  if (panelid.getAttribute)
    // we might be passed an HTMLElement <a>.
--- a/management/templates/login.html
+++ b/management/templates/login.html
@@ -1,4 +1,29 @@
-<h1 style="margin: 1em; text-align: center">{{hostname}}</h1>
+<style>
+  .title {
+    margin: 1em;
+    text-align: center;
+  }
+
+  .subtitle {
+    margin: 2em;
+    text-align: center;
+  }
+
+  .login {
+    margin: 0 auto;
+    max-width: 32em;
+  }
+
+  .login #loginOtp {
+    display: none;
+  }
+
+  #loginForm.is-twofactor #loginOtp {
+    display: block
+  }
+</style>
+
+<h1 class="title">{{hostname}}</h1>

 {% if no_users_exist or no_admins_exist %}
 <div class="row">
@@ -7,23 +32,23 @@
  <p class="text-danger">There are no users on this system! To make an administrative user,
  log into this machine using SSH (like when you first set it up) and run:</p>
  <pre>cd mailinabox
-sudo tools/mail.py user add me@{{hostname}}
-sudo tools/mail.py user make-admin me@{{hostname}}</pre>
+sudo management/cli.py user add me@{{hostname}}
+sudo management/cli.py user make-admin me@{{hostname}}</pre>
  {% else %}
  <p class="text-danger">There are no administrative users on this system! To make an administrative user,
  log into this machine using SSH (like when you first set it up) and run:</p>
  <pre>cd mailinabox
-sudo tools/mail.py user make-admin me@{{hostname}}</pre>
+sudo management/cli.py user make-admin me@{{hostname}}</pre>
  {% endif %}
  <hr>
 </div>
 </div>
 {% endif %}

-<p style="margin: 2em; text-align: center;">Log in here for your Mail-in-a-Box control panel.</p>
+<p class="subtitle">Log in here for your Mail-in-a-Box control panel.</p>

-<div style="margin: 0 auto; max-width: 32em;">
-  <form class="form-horizontal" role="form" onsubmit="do_login(); return false;" method="get">
+<div class="login">
+  <form id="loginForm" class="form-horizontal" role="form" onsubmit="do_login(); return false;" method="get">
    <div class="form-group">
      <label for="inputEmail3" class="col-sm-3 control-label">Email</label>
      <div class="col-sm-9">
@@ -36,6 +61,13 @@ sudo tools/mail.py user make-admin me@{{hostname}}</pre>
        <input name="password" type="password" class="form-control" id="loginPassword" placeholder="Password">
      </div>
    </div>
+    <div class="form-group" id="loginOtp">
+      <label for="loginOtpInput" class="col-sm-3 control-label">Code</label>
+      <div class="col-sm-9">
+          <input type="text" class="form-control" id="loginOtpInput" placeholder="6-digit code">
+          <div class="help-block" style="margin-top: 5px; font-size: 90%">Enter the six-digit code generated by your two factor authentication app.</div>
+      </div>
+    </div>
    <div class="form-group">
      <div class="col-sm-offset-3 col-sm-9">
        <div class="checkbox">
@@ -53,7 +85,6 @@ sudo tools/mail.py user make-admin me@{{hostname}}</pre>
  </form>
 </div>

-
 <script>
 function do_login() {
  if ($('#loginEmail').val() == "") {
@@ -62,6 +93,7 @@ function do_login() {
    });
    return false;
  }
+
  if ($('#loginPassword').val() == "") {
    show_modal_error("Login Failed", "Enter your email password.", function() {
        $('#loginPassword').focus();
@@ -79,13 +111,25 @@ function do_login() {
  function(response) {
    // This API call always succeeds. It returns a JSON object indicating
    // whether the request was authenticated or not.
-    if (response.status != "ok") {
+    if (response.status != 'ok') {
+      if (response.status === 'missing-totp-token' || (response.status === 'invalid' && response.reason == 'invalid-totp-token')) {
+        $('#loginForm').addClass('is-twofactor');
+        if (response.reason === "invalid-totp-token") {
+          show_modal_error("Login Failed", "Incorrect two factor authentication token.");
+        } else {
+          setTimeout(() => {
+              $('#loginOtpInput').focus();
+          });
+        }
+      } else {
+        $('#loginForm').removeClass('is-twofactor');
+
        // Show why the login failed.
        show_modal_error("Login Failed", response.reason)

        // Reset any saved credentials.
        do_logout();
-
+      }
    } else if (!("api_key" in response)) {
      // Login succeeded but user might not be authorized!
      show_modal_error("Login Failed", "You are not an administrator on this system.")
@@ -102,6 +146,8 @@ function do_login() {
      // Try to wipe the username/password information.
      $('#loginEmail').val('');
      $('#loginPassword').val('');
+      $('#loginOtpInput').val('');
+      $('#loginForm').removeClass('is-twofactor');

      // Remember the credentials.
      if (typeof localStorage != 'undefined' && typeof sessionStorage != 'undefined') {
@@ -119,19 +165,16 @@ function do_login() {
      // which confuses the loading indicator.
      setTimeout(function() { show_panel(!switch_back_to_panel || switch_back_to_panel == "login" ? 'system_status' : switch_back_to_panel) }, 300);
    }
-  })
-}
-
-function do_logout() {
-  api_credentials = ["", ""];
-  if (typeof localStorage != 'undefined')
-    localStorage.removeItem("miab-cp-credentials");
-  if (typeof sessionStorage != 'undefined')
-    sessionStorage.removeItem("miab-cp-credentials");
-  show_panel('login');
+  },
+  undefined,
+  {
+    'x-auth-token': $('#loginOtpInput').val()
+  });
 }

 function show_login() {
+  $('#loginForm').removeClass('is-twofactor');
+  $('#loginOtpInput').val('');
  $('#loginEmail,#loginPassword').each(function() {
    var input = $(this);
    if (!$.trim(input.val())) {
--- a/management/templates/mail-guide.html
+++ b/management/templates/mail-guide.html
@@ -30,8 +30,8 @@
 					<tr><th>Mail server</th> <td>{{hostname}}</td>
 					<tr><th>IMAP Port</th> <td>993</td></tr>
 					<tr><th>IMAP Security</th> <td>SSL or TLS</td></tr>
-					<tr><th>SMTP Port</th> <td>587</td></tr>
-					<tr><th>SMTP Security</td> <td>STARTTLS <small>(&ldquo;always&rdquo; or &ldquo;required&rdquo;, if prompted)</small></td></tr>
+					<tr><th>SMTP Port</th> <td>465</td></tr>
+					<tr><th>SMTP Security</td> <td>SSL or TLS</td></tr>
 					<tr><th>Username:</th> <td>Your whole email address.</td></tr>
 					<tr><th>Password:</th> <td>Your mail password.</td></tr>
 					</table>
--- a/management/templates/mfa.html
+++ b/management/templates/mfa.html
@@ -0,0 +1,242 @@
+<style>
+    .twofactor #totp-setup,
+    .twofactor #disable-2fa,
+    .twofactor #output-2fa {
+        display: none;
+    }
+
+    .twofactor.loaded .loading-indicator {
+        display: none;
+    }
+
+    .twofactor.disabled #disable-2fa,
+    .twofactor.enabled #totp-setup {
+        display: none;
+    }
+
+    .twofactor.disabled #totp-setup,
+    .twofactor.enabled #disable-2fa {
+        display: block;
+    }
+
+    .twofactor #totp-setup-qr img {
+        display: block;
+        width: 256px;
+        max-width: 100%;
+        height: auto;
+    }
+
+    .twofactor #output-2fa.visible {
+        display: block;
+    }
+</style>
+
+<h2>Two-Factor Authentication</h2>
+
+<p>When two-factor authentication is enabled, you will be prompted to enter a six digit code from an
+authenticator app (usually on your phone) when you log into this control panel.</p>
+
+<div class="panel panel-danger">
+<div class="panel-heading">
+Enabling two-factor authentication does not protect access to your email
+</div>
+<div class="panel-body">
+Enabling two-factor authentication on this page only limits access to this control panel. Remember that most websites allow you to
+reset your password by checking your email, so anyone with access to your email can typically take over
+your other accounts. Additionally, if your email address or any alias that forwards to your email
+address is a typical domain control validation address (e.g admin@, administrator@, postmaster@, hostmaster@,
+webmaster@, abuse@), extra care should be taken to protect the account. <strong>Always use a strong password,
+and ensure every administrator account for this control panel does the same.</strong>
+</div>
+</div>
+
+<div class="twofactor">
+    <div class="loading-indicator">Loading...</div>
+
+    <form id="totp-setup">
+        <h3>Setup Instructions</h3>
+
+        <div class="form-group">
+            <p>1. Install <a href="https://freeotp.github.io/">FreeOTP</a> or <a href="https://www.pcworld.com/article/3225913/what-is-two-factor-authentication-and-which-2fa-apps-are-best.html">any
+            other two-factor authentication app</a> that supports TOTP.</p>
+        </div>
+
+        <div class="form-group">
+            <p style="margin-bottom: 0">2. Scan the QR code in the app or directly enter the secret into the app:</p>
+            <div id="totp-setup-qr"></div>
+        </div>
+
+        <div class="form-group">
+            <label for="otp-label" style="font-weight: normal">3. Optionally, give your device a label so that you can remember what device you set it up on:</label>
+            <input type="text" id="totp-setup-label" class="form-control" placeholder="my phone" />
+        </div>
+
+        <div class="form-group">
+            <label for="otp" style="font-weight: normal">4. Use the app to generate your first six-digit code and enter it here:</label>
+            <input type="text" id="totp-setup-token" class="form-control" placeholder="6-digit code" />
+        </div>
+
+        <input type="hidden" id="totp-setup-secret" />
+
+        <div class="form-group">
+            <p>When you click Enable Two-Factor Authentication, you will be logged out of the control panel and will have to log in
+            again, now using your two-factor authentication app.</p>
+            <button id="totp-setup-submit" disabled type="submit" class="btn">Enable Two-Factor Authentication</button>
+        </div>
+    </form>
+
+    <form id="disable-2fa">
+        <div class="form-group">
+            <p>Two-factor authentication is active for your account<span id="mfa-device-label"></span>.</p>
+            <p>You will have to log into the admin panel again after disabling two-factor authentication.</p>
+        </div>
+        <div class="form-group">
+            <button type="submit" class="btn btn-danger">Disable Two-Factor Authentication</button>
+        </div>
+    </form>
+
+    <div id="output-2fa" class="panel panel-danger">
+        <div class="panel-body"></div>
+    </div>
+</div>
+
+<script>
+    var el = {
+        disableForm: document.getElementById('disable-2fa'),
+        output: document.getElementById('output-2fa'),
+        totpSetupForm: document.getElementById('totp-setup'),
+        totpSetupToken: document.getElementById('totp-setup-token'),
+        totpSetupSecret: document.getElementById('totp-setup-secret'),
+        totpSetupLabel: document.getElementById('totp-setup-label'),
+        totpQr: document.getElementById('totp-setup-qr'),
+        totpSetupSubmit: document.querySelector('#totp-setup-submit'),
+        wrapper: document.querySelector('.twofactor')
+    }
+
+    function update_setup_disabled(evt) {
+        var val = evt.target.value.trim();
+
+        if (
+            typeof val !== 'string' ||
+            typeof el.totpSetupSecret.value !== 'string' ||
+            val.length !== 6 ||
+            el.totpSetupSecret.value.length !== 32 ||
+            !(/^\+?\d+$/.test(val))
+        ) {
+            el.totpSetupSubmit.setAttribute('disabled', '');
+        } else {
+            el.totpSetupSubmit.removeAttribute('disabled');
+        }
+    }
+
+    function render_totp_setup(provisioned_totp) {
+        var img = document.createElement('img');
+        img.src = "data:image/png;base64," + provisioned_totp.qr_code_base64;
+
+        var code = document.createElement('div');
+        code.innerHTML = `Secret: ${provisioned_totp.secret}`;
+
+        el.totpQr.appendChild(img);
+        el.totpQr.appendChild(code);
+
+        el.totpSetupToken.addEventListener('input', update_setup_disabled);
+        el.totpSetupForm.addEventListener('submit', do_enable_totp);
+
+        el.totpSetupSecret.setAttribute('value', provisioned_totp.secret);
+
+        el.wrapper.classList.add('disabled');
+    }
+
+    function render_disable(mfa) {
+        el.disableForm.addEventListener('submit', do_disable);
+        el.wrapper.classList.add('enabled');
+        if (mfa.label)
+          $("#mfa-device-label").text(" on device '" + mfa.label + "'");
+    }
+
+    function hide_error() {
+        el.output.querySelector('.panel-body').innerHTML = '';
+        el.output.classList.remove('visible');
+    }
+
+    function render_error(msg) {
+        el.output.querySelector('.panel-body').innerHTML = msg;
+        el.output.classList.add('visible');
+    }
+
+    function reset_view() {
+        el.wrapper.classList.remove('loaded', 'disabled', 'enabled');
+
+        el.disableForm.removeEventListener('submit', do_disable);
+
+        hide_error();
+
+        el.totpSetupForm.reset();
+        el.totpSetupForm.removeEventListener('submit', do_enable_totp);
+
+        el.totpSetupSecret.setAttribute('value', '');
+        el.totpSetupToken.removeEventListener('input', update_setup_disabled);
+
+        el.totpSetupSubmit.setAttribute('disabled', '');
+        el.totpQr.innerHTML = '';
+    }
+
+    function show_mfa() {
+        reset_view();
+
+        api(
+            '/mfa/status',
+            'POST',
+            {},
+            function(res) {
+                el.wrapper.classList.add('loaded');
+
+                var has_mfa = false;
+                res.enabled_mfa.forEach(function(mfa) {
+                    if (mfa.type == "totp") {
+                        render_disable(mfa);
+                        has_mfa = true;
+                    }
+                });
+
+                if (!has_mfa)
+                  render_totp_setup(res.new_mfa.totp);
+            }
+        );
+    }
+
+    function do_disable(evt) {
+        evt.preventDefault();
+        hide_error();
+
+        api(
+            '/mfa/disable',
+            'POST',
+            { type: 'totp' },
+            function() {
+                do_logout();
+            }
+        );
+
+        return false;
+    }
+
+    function do_enable_totp(evt) {
+        evt.preventDefault();
+        hide_error();
+
+        api(
+            '/mfa/totp/enable',
+            'POST',
+            {
+                token: $(el.totpSetupToken).val(),
+                secret: $(el.totpSetupSecret).val(),
+                label: $(el.totpSetupLabel).val()
+            },
+            function(res) { do_logout(); },
+            function(res) { render_error(res); }
+        );
+
+        return false;
+    }
+</script>
--- a/management/templates/system-backup.html
+++ b/management/templates/system-backup.html
@@ -5,7 +5,7 @@

 <h2>Backup Status</h2>

-<p>The box makes an incremental backup each night. By default the backup is stored on the machine itself, but you can also have it stored on Amazon S3.</p>
+<p>The box makes an incremental backup each night. By default the backup is stored on the machine itself, but you can also store in on S3-compatible services like Amazon Web Services (AWS).</p>

 <h3>Configuration</h3>

@@ -17,7 +17,8 @@
        <option value="off">Nowhere (Disable Backups)</option>
        <option value="local">{{hostname}}</option>
        <option value="rsync">rsync</option>
-        <option value="s3">Amazon S3</option>
+        <option value="s3">S3 (Amazon or compatible) </option>
+        <option value="b2">Backblaze B2</option>
      </select>
    </div>
  </div>
@@ -72,8 +73,8 @@
  <!-- S3 BACKUP -->
  <div class="form-group backup-target-s3">
    <div class="col-sm-10 col-sm-offset-2">
-      <p>Backups are stored in an Amazon Web Services S3 bucket. You must have an AWS account already.</p>
-      <p>You MUST manually copy the encryption password from <tt class="backup-encpassword-file"></tt> to a safe and secure location. You will need this file to decrypt backup files. It is NOT stored in your Amazon S3 bucket.</p>
+      <p>Backups are stored in an S3-compatible bucket. You must have an AWS or other S3 service account already.</p>
+      <p>You MUST manually copy the encryption password from <tt class="backup-encpassword-file"></tt> to a safe and secure location. You will need this file to decrypt backup files. It is <b>NOT</b> stored in your S3 bucket.</p>
    </div>
  </div>
  <div class="form-group backup-target-s3">
@@ -83,7 +84,7 @@
        {% for name, host in backup_s3_hosts %}
          <option value="{{host}}">{{name}}</option>
        {% endfor %}
-        <option value="other">Other</option>
+        <option value="other">Other (non AWS)</option>
      </select>
    </div>
  </div>
@@ -111,12 +112,37 @@
      <input type="text" class="form-control" rows="1" id="backup-target-pass">
    </div>
  </div>
+  <!-- Backblaze -->
+  <div class="form-group backup-target-b2">
+      <div class="col-sm-10 col-sm-offset-2">
+        <p>Backups are stored in a <a href="https://www.backblaze.com/" target="_blank" rel="noreferrer">Backblaze</a> B2 bucket. You must have a Backblaze account already.</p>
+        <p>You MUST manually copy the encryption password from <tt class="backup-encpassword-file"></tt> to a safe and secure location. You will need this file to decrypt backup files. It is NOT stored in your Backblaze B2 bucket.</p>
+      </div>
+  </div>
+  <div class="form-group backup-target-b2">
+      <label for="backup-target-b2-user" class="col-sm-2 control-label">B2 Application KeyID</label>
+      <div class="col-sm-8">
+        <input type="text" class="form-control" rows="1" id="backup-target-b2-user">
+      </div>
+  </div>
+  <div class="form-group backup-target-b2">
+      <label for="backup-target-b2-pass" class="col-sm-2 control-label">B2 Application Key</label>
+      <div class="col-sm-8">
+        <input type="text" class="form-control" rows="1" id="backup-target-b2-pass">
+      </div>
+  </div>
+  <div class="form-group backup-target-b2">
+      <label for="backup-target-b2-bucket" class="col-sm-2 control-label">B2 Bucket</label>
+      <div class="col-sm-8">
+        <input type="text" class="form-control" rows="1" id="backup-target-b2-bucket">
+      </div>
+  </div>
  <!-- Common -->
  <div class="form-group backup-target-local backup-target-rsync backup-target-s3">
-    <label for="min-age" class="col-sm-2 control-label">Days:</label>
+    <label for="min-age" class="col-sm-2 control-label">Retention Days:</label>
    <div class="col-sm-8">
      <input type="number" class="form-control" rows="1" id="min-age">
-      <div class="small" style="margin-top: 2px">This is the <i>minimum</i> number of days backup data is kept for. The box makes an incremental backup, so backup data is often kept much longer. An incremental backup file that is less than this number of days old requires that all previous increments back to the most recent full backup, plus that full backup, remain available.</div>
+      <div class="small" style="margin-top: 2px">This is the minimum time backup data is kept for. The box makes an incremental backup most nights, which requires that previous backups back to the most recent full backup be preserved, so backup data is often kept much longer than this setting. Full backups are made periodically when the incremental backup data size exceeds a limit.</div>
    </div>
  </div>
  <div class="form-group">
@@ -144,7 +170,7 @@

 function toggle_form() {
  var target_type = $("#backup-target-type").val();
-  $(".backup-target-local, .backup-target-rsync, .backup-target-s3").hide();
+  $(".backup-target-local, .backup-target-rsync, .backup-target-s3, .backup-target-b2").hide();
  $(".backup-target-" + target_type).show();

  init_inputs(target_type);
@@ -215,7 +241,7 @@ function show_system_backup() {
 }

 function show_custom_backup() {
-    $(".backup-target-local, .backup-target-rsync, .backup-target-s3").hide();
+    $(".backup-target-local, .backup-target-rsync, .backup-target-s3, .backup-target-b2").hide();
    api(
      "/system/backup/config",
      "GET",
@@ -245,6 +271,15 @@ function show_custom_backup() {
          var host = hostpath.shift();
          $("#backup-target-s3-host").val(host);
          $("#backup-target-s3-path").val(hostpath.join('/'));
+        } else if (r.target.substring(0, 5) == "b2://") {
+          $("#backup-target-type").val("b2");
+          var targetPath = r.target.substring(5);
+          var b2_application_keyid = targetPath.split(':')[0];
+          var b2_applicationkey = targetPath.split(':')[1].split('@')[0];
+          var b2_bucket = targetPath.split('@')[1];
+          $("#backup-target-b2-user").val(b2_application_keyid);
+          $("#backup-target-b2-pass").val(b2_applicationkey);
+          $("#backup-target-b2-bucket").val(b2_bucket);
        }
        toggle_form()
      })
@@ -264,6 +299,11 @@ function set_custom_backup() {
    target = "rsync://" + $("#backup-target-rsync-user").val() + "@" + $("#backup-target-rsync-host").val()
                                                                + "/" + $("#backup-target-rsync-path").val();
    target_user = '';
+  } else if (target_type == "b2") {
+    target = 'b2://' + $('#backup-target-b2-user').val() + ':' + $('#backup-target-b2-pass').val()
+        + '@' + $('#backup-target-b2-bucket').val()
+    target_user = '';
+    target_pass = '';
  }


--- a/management/templates/users.html
+++ b/management/templates/users.html
@@ -1,7 +1,6 @@
 <h2>Users</h2>

 <style>
-#user_table h4 { margin: 1em 0 0 0; }
 #user_table tr.account_inactive td.address { color: #888; text-decoration: line-through; }
 #user_table .actions { margin-top: .33em; font-size: 95%; }
 #user_table .account_inactive .if_active { display: none; }
@@ -31,7 +30,7 @@
  <button type="submit" class="btn btn-primary">Add User</button>
 </form>
 <ul style="margin-top: 1em; padding-left: 1.5em; font-size: 90%;">
-  <li>Passwords must be at least eight characters consisting of English lettters and numbers only. For best results, <a href="#" onclick="return generate_random_password()">generate a random password</a>.</li>
+  <li>Passwords must be at least eight characters consisting of English letters and numbers only. For best results, <a href="#" onclick="return generate_random_password()">generate a random password</a>.</li>
  <li>Use <a href="#" onclick="return show_panel('aliases')">aliases</a> to create email addresses that forward to existing accounts.</li>
  <li>Administrators get access to this control panel.</li>
  <li>User accounts cannot contain any international (non-ASCII) characters, but <a href="#" onclick="return show_panel('aliases');">aliases</a> can.</li>
@@ -134,8 +133,8 @@ function show_users() {
    function(r) {
      $('#user_table tbody').html("");
      for (var i = 0; i < r.length; i++) {
-        var hdr = $("<tr><td colspan='3'><h4/></td></tr>");
-        hdr.find('h4').text(r[i].domain);
+        var hdr = $("<tr><th colspan='2' style='background-color: #EEE'></th></tr>");
+        hdr.find('th').text(r[i].domain);
        $('#user_table tbody').append(hdr);

        for (var k = 0; k < r[i].users.length; k++) {
--- a/management/web_update.py
+++ b/management/web_update.py
@@ -9,7 +9,7 @@ from dns_update import get_custom_dns_config, get_dns_zones
 from ssl_certificates import get_ssl_certificates, get_domain_ssl_files, check_certificate
 from utils import shell, safe_domain_name, sort_domains

-def get_web_domains(env, include_www_redirects=True, exclude_dns_elsewhere=True):
+def get_web_domains(env, include_www_redirects=True, include_auto=True, exclude_dns_elsewhere=True):
 	# What domains should we serve HTTP(S) for?
 	domains = set()

@@ -18,15 +18,16 @@ def get_web_domains(env, include_www_redirects=True, exclude_dns_elsewhere=True)
 	# if the user wants to make one.
 	domains |= get_mail_domains(env)

-	if include_www_redirects:
+	if include_www_redirects and include_auto:
 		# Add 'www.' subdomains that we want to provide default redirects
 		# to the main domain for. We'll add 'www.' to any DNS zones, i.e.
 		# the topmost of each domain we serve.
 		domains |= set('www.' + zone for zone, zonefile in get_dns_zones(env))

+	if include_auto:
 		# Add Autoconfiguration domains for domains that there are user accounts at:
 		# 'autoconfig.' for Mozilla Thunderbird auto setup.
-	# 'autodiscover.' for Activesync autodiscovery.
+		# 'autodiscover.' for ActiveSync autodiscovery (Z-Push).
 		domains |= set('autoconfig.' + maildomain for maildomain in get_mail_domains(env, users_only=True))
 		domains |= set('autodiscover.' + maildomain for maildomain in get_mail_domains(env, users_only=True))

@@ -146,7 +147,7 @@ def make_domain_config(domain, templates, ssl_certificates, env):
 		finally:
 			f.close()
 		return sha1.hexdigest()
-	nginx_conf_extra += "# ssl files sha1: %s / %s\n" % (hashfile(tls_cert["private-key"]), hashfile(tls_cert["certificate"]))
+	nginx_conf_extra += "\t# ssl files sha1: %s / %s\n" % (hashfile(tls_cert["private-key"]), hashfile(tls_cert["certificate"]))

 	# Add in any user customizations in YAML format.
 	hsts = "yes"
@@ -160,17 +161,27 @@ def make_domain_config(domain, templates, ssl_certificates, env):
 			for path, url in yaml.get("proxies", {}).items():
 				# Parse some flags in the fragment of the URL.
 				pass_http_host_header = False
+				proxy_redirect_off = False
+				frame_options_header_sameorigin = False
 				m = re.search("#(.*)$", url)
 				if m:
 					for flag in m.group(1).split(","):
 						if flag == "pass-http-host":
 							pass_http_host_header = True
+						elif flag == "no-proxy-redirect":
+							proxy_redirect_off = True
+						elif flag == "frame-options-sameorigin":
+							frame_options_header_sameorigin = True
 					url = re.sub("#(.*)$", "", url)

 				nginx_conf_extra += "\tlocation %s {" % path
 				nginx_conf_extra += "\n\t\tproxy_pass %s;" % url
+				if proxy_redirect_off:
+					nginx_conf_extra += "\n\t\tproxy_redirect off;"
 				if pass_http_host_header:
 					nginx_conf_extra += "\n\t\tproxy_set_header Host $http_host;"
+				if frame_options_header_sameorigin:
+					nginx_conf_extra += "\n\t\tproxy_set_header X-Frame-Options SAMEORIGIN;"
 				nginx_conf_extra += "\n\t\tproxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;"
 				nginx_conf_extra += "\n\t\tproxy_set_header X-Forwarded-Host $http_host;"
 				nginx_conf_extra += "\n\t\tproxy_set_header X-Forwarded-Proto $scheme;"
@@ -188,9 +199,9 @@ def make_domain_config(domain, templates, ssl_certificates, env):

 	# Add the HSTS header.
 	if hsts == "yes":
-		nginx_conf_extra += "add_header Strict-Transport-Security \"max-age=15768000\" always;\n"
+		nginx_conf_extra += "\tadd_header Strict-Transport-Security \"max-age=15768000\" always;\n"
 	elif hsts == "preload":
-		nginx_conf_extra += "add_header Strict-Transport-Security \"max-age=15768000; includeSubDomains; preload\" always;\n"
+		nginx_conf_extra += "\tadd_header Strict-Transport-Security \"max-age=15768000; includeSubDomains; preload\" always;\n"

 	# Add in any user customizations in the includes/ folder.
 	nginx_conf_custom_include = os.path.join(env["STORAGE_ROOT"], "www", safe_domain_name(domain) + ".conf")
@@ -251,3 +262,4 @@ def get_web_domains_info(env):
 		}
 		for domain in get_web_domains(env)
 	]
+
--- a/security.md
+++ b/security.md
@@ -32,7 +32,7 @@ The box's administrator and its (non-administrative) mail users must sometimes c

 These services are protected by [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security):

-* SMTP Submission (port 587). Mail users submit outbound mail through SMTP with STARTTLS on port 587.
+* SMTP Submission (ports 465/587). Mail users submit outbound mail through SMTP with TLS (port 465) or STARTTLS (port 587).
 * IMAP/POP (ports 993, 995). Mail users check for incoming mail through IMAP or POP over TLS.
 * HTTPS (port 443). Webmail, the Exchange/ActiveSync protocol, the administrative control panel, and any static hosted websites are accessed over HTTPS.

@@ -44,7 +44,7 @@ The services all follow these rules:

 Additionally:

-* SMTP Submission (port 587) will not accept user credentials without STARTTLS (true also of SMTP on port 25 in case of client misconfiguration), and the submission port won't accept mail without encryption. The minimum cipher key length is 128 bits. (The box is of course configured not to be an open relay. User credentials are required to send outbound mail.) ([source](setup/mail-postfix.sh))
+* SMTP Submission on port 587 will not accept user credentials without STARTTLS (true also of SMTP on port 25 in case of client misconfiguration), and the submission port won't accept mail without encryption. The minimum cipher key length is 128 bits. (The box is of course configured not to be an open relay. User credentials are required to send outbound mail.) ([source](setup/mail-postfix.sh))
 * HTTPS (port 443): The HTTPS Strict Transport Security header is set. A redirect from HTTP to HTTPS is offered. The [Qualys SSL Labs test](https://www.ssllabs.com/ssltest) should report an A+ grade. ([source 1](conf/nginx-ssl.conf), [source 2](conf/nginx.conf))

 ### Password Storage
--- a/setup/bootstrap.sh
+++ b/setup/bootstrap.sh
@@ -18,11 +18,11 @@ if [ -z "$TAG" ]; then
 	# space, but if we put it in a comment it would confuse the status checks!)
 	# to get the latest version, so the first such line must be the one that we
 	# want to display in status checks.
-	if [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' `" == "Ubuntu 18.04 LTS" ]; then
+	if [ "$(lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' )" == "Ubuntu 18.04 LTS" ]; then
 		# This machine is running Ubuntu 18.04.
-		TAG=v0.50
+		TAG=v0.54

-	elif [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/14\.04\.[0-9]/14.04/' `" == "Ubuntu 14.04 LTS" ]; then
+	elif [ "$(lsb_release -d | sed 's/.*:\s*//' | sed 's/14\.04\.[0-9]/14.04/' )" == "Ubuntu 14.04 LTS" ]; then
 		# This machine is running Ubuntu 14.04.
 		echo "You are installing the last version of Mail-in-a-Box that will"
 		echo "support Ubuntu 14.04. If this is a new installation of Mail-in-a-Box,"
@@ -68,11 +68,11 @@ fi
 cd $HOME/mailinabox

 # Update it.
-if [ "$TAG" != `git describe` ]; then
+if [ "$TAG" != $(git describe) ]; then
 	echo Updating Mail-in-a-Box to $TAG . . .
 	git fetch --depth 1 --force --prune origin tag $TAG
 	if ! git checkout -q $TAG; then
-		echo "Update failed. Did you modify something in `pwd`?"
+		echo "Update failed. Did you modify something in $(pwd)?"
 		exit 1
 	fi
 	echo
--- a/setup/dkim.sh
+++ b/setup/dkim.sh
@@ -62,7 +62,40 @@ chmod go-rwx $STORAGE_ROOT/mail/dkim

 tools/editconf.py /etc/opendmarc.conf -s \
 	"Syslog=true" \
-	"Socket=inet:8893@[127.0.0.1]"
+	"Socket=inet:8893@[127.0.0.1]" \
+	"FailureReports=true"
+
+# SPFIgnoreResults causes the filter to ignore any SPF results in the header
+# of the message. This is useful if you want the filter to perfrom SPF checks
+# itself, or because you don't trust the arriving header. This added header is
+# used by spamassassin to evaluate the mail for spamminess.
+
+tools/editconf.py /etc/opendmarc.conf -s \
+        "SPFIgnoreResults=true"
+
+# SPFSelfValidate causes the filter to perform a fallback SPF check itself
+# when it can find no SPF results in the message header. If SPFIgnoreResults
+# is also set, it never looks for SPF results in headers and always performs
+# the SPF check itself when this is set. This added header is used by
+# spamassassin to evaluate the mail for spamminess.
+
+tools/editconf.py /etc/opendmarc.conf -s \
+        "SPFSelfValidate=true"
+
+# Enables generation of failure reports for sending domains that publish a
+# "none" policy.
+
+tools/editconf.py /etc/opendmarc.conf -s \
+        "FailureReportsOnNone=true"
+
+# AlwaysAddARHeader Adds an "Authentication-Results:" header field even to
+# unsigned messages from domains with no "signs all" policy. The reported DKIM
+# result will be  "none" in such cases. Normally unsigned mail from non-strict
+# domains does not cause the results header field to be added. This added header
+# is used by spamassassin to evaluate the mail for spamminess.
+
+tools/editconf.py /etc/opendkim.conf -s \
+        "AlwaysAddARHeader=true"

 # Add OpenDKIM and OpenDMARC as milters to postfix, which is how OpenDKIM
 # intercepts outgoing mail to perform the signing (by adding a mail header)
--- a/setup/dns.sh
+++ b/setup/dns.sh
@@ -68,27 +68,15 @@ echo "include: /etc/nsd/zones.conf" >> /etc/nsd/nsd.conf;

 mkdir -p "$STORAGE_ROOT/dns/dnssec";

-# TLDs don't all support the same algorithms, so we'll generate keys using a few
-# different algorithms. RSASHA1-NSEC3-SHA1 was possibly the first widely used
-# algorithm that supported NSEC3, which is a security best practice. However TLDs
-# will probably be moving away from it to a a SHA256-based algorithm.
-#
-# Supports `RSASHA1-NSEC3-SHA1` (didn't test with `RSASHA256`):
-#
-#  * .info
-#  * .me
-#
-# Requires `RSASHA256`
-#
-#  * .email
-#  * .guide
-#
-# Supports `RSASHA256` (and defaulting to this)
-#
-#  * .fund
-
+# TLDs, registrars, and validating nameservers don't all support the same algorithms,
+# so we'll generate keys using a few different algorithms so that dns_update.py can
+# choose which algorithm to use when generating the zonefiles. See #1953 for recent
+# discussion. File for previously used algorithms (i.e. RSASHA1-NSEC3-SHA1) may still
+# be in the output directory, and we'll continue to support signing zones with them
+# so that trust isn't broken with deployed DS records, but we won't generate those
+# keys on new systems.
 FIRST=1 #NODOC
-for algo in RSASHA1-NSEC3-SHA1 RSASHA256; do
+for algo in RSASHA256 ECDSAP256SHA256; do
 if [ ! -f "$STORAGE_ROOT/dns/dnssec/$algo.conf" ]; then
 	if [ $FIRST == 1 ]; then
 		echo "Generating DNSSEC signing keys..."
@@ -105,17 +93,22 @@ if [ ! -f "$STORAGE_ROOT/dns/dnssec/$algo.conf" ]; then
 	# ldns-keygen uses /dev/random for generating random numbers by default.
 	# This is slow and unecessary if we ensure /dev/urandom is seeded properly,
 	# so we use /dev/urandom. See system.sh for an explanation. See #596, #115.
-	KSK=$(umask 077; cd $STORAGE_ROOT/dns/dnssec; ldns-keygen -r /dev/urandom -a $algo -b 2048 -k _domain_);
+	# (This previously used -b 2048 but it's unclear if this setting makes sense
+	# for non-RSA keys, so it's removed. The RSA-based keys are not recommended
+	# anymore anyway.)
+	KSK=$(umask 077; cd $STORAGE_ROOT/dns/dnssec; ldns-keygen -r /dev/urandom -a $algo -k _domain_);

 	# Now create a Zone-Signing Key (ZSK) which is expected to be
 	# rotated more often than a KSK, although we have no plans to
 	# rotate it (and doing so would be difficult to do without
-	# disturbing DNS availability.) Omit `-k` and use a shorter key length.
-	ZSK=$(umask 077; cd $STORAGE_ROOT/dns/dnssec; ldns-keygen -r /dev/urandom -a $algo -b 1024 _domain_);
+	# disturbing DNS availability.) Omit `-k`.
+	# (This previously used -b 1024 but it's unclear if this setting makes sense
+	# for non-RSA keys, so it's removed.)
+	ZSK=$(umask 077; cd $STORAGE_ROOT/dns/dnssec; ldns-keygen -r /dev/urandom -a $algo _domain_);

 	# These generate two sets of files like:
 	#
-	# * `K_domain_.+007+08882.ds`: DS record normally provided to domain name registrar (but it's actually invalid with `_domain_`)
+	# * `K_domain_.+007+08882.ds`: DS record normally provided to domain name registrar (but it's actually invalid with `_domain_` so we don't use this file)
 	# * `K_domain_.+007+08882.key`: public key
 	# * `K_domain_.+007+08882.private`: private key (secret!)

@@ -139,7 +132,7 @@ cat > /etc/cron.daily/mailinabox-dnssec << EOF;
 #!/bin/bash
 # Mail-in-a-Box
 # Re-sign any DNS zones with DNSSEC because the signatures expire periodically.
-`pwd`/tools/dns_update
+$(pwd)/tools/dns_update
 EOF
 chmod +x /etc/cron.daily/mailinabox-dnssec

--- a/setup/firstuser.sh
+++ b/setup/firstuser.sh
@@ -1,6 +1,6 @@
 # If there aren't any mail users yet, create one.
-if [ -z "`tools/mail.py user`" ]; then
-	# The outut of "tools/mail.py user" is a list of mail users. If there
+if [ -z "$(management/cli.py user)" ]; then
+	# The outut of "management/cli.py user" is a list of mail users. If there
 	# aren't any yet, it'll be empty.

 	# If we didn't ask for an email address at the start, do so now.
@@ -10,7 +10,7 @@ if [ -z "`tools/mail.py user`" ]; then
 			input_box "Mail Account" \
 				"Let's create your first mail account.
 				\n\nWhat email address do you want?" \
-				me@`get_default_hostname` \
+				me@$(get_default_hostname) \
 				EMAIL_ADDR

 			if [ -z "$EMAIL_ADDR" ]; then
@@ -47,11 +47,11 @@ if [ -z "`tools/mail.py user`" ]; then
 	fi

 	# Create the user's mail account. This will ask for a password if none was given above.
-	tools/mail.py user add $EMAIL_ADDR ${EMAIL_PW:-}
+	management/cli.py user add $EMAIL_ADDR ${EMAIL_PW:-}

 	# Make it an admin.
-	hide_output tools/mail.py user make-admin $EMAIL_ADDR
+	hide_output management/cli.py user make-admin $EMAIL_ADDR

 	# Create an alias to which we'll direct all automatically-created administrative aliases.
-	tools/mail.py alias add administrator@$PRIMARY_HOSTNAME $EMAIL_ADDR > /dev/null
+	management/cli.py alias add administrator@$PRIMARY_HOSTNAME $EMAIL_ADDR > /dev/null
 fi
--- a/setup/functions.sh
+++ b/setup/functions.sh
@@ -9,12 +9,12 @@ function hide_output {
 	# and returns a non-zero exit code.

 	# Get a temporary file.
-	OUTPUT=$(tempfile)
+	OUTPUT=$(mktemp)

 	# Execute command, redirecting stderr/stdout to the temporary file. Since we
 	# check the return code ourselves, disable 'set -e' temporarily.
 	set +e
-	$@ &> $OUTPUT
+	"$@" &> $OUTPUT
 	E=$?
 	set -e

@@ -22,7 +22,7 @@ function hide_output {
 	if [ $E != 0 ]; then
 		# Something failed.
 		echo
-		echo FAILED: $@
+		echo FAILED: "$@"
 		echo -----------------------------------------
 		cat $OUTPUT
 		echo -----------------------------------------
@@ -53,8 +53,7 @@ function apt_install {
 	# install' for all of the packages.  Calling `dpkg` on each package is slow,
 	# and doesn't affect what we actually do, except in the messages, so let's
 	# not do that anymore.
-	PACKAGES=$@
-	apt_get_quiet install $PACKAGES
+	apt_get_quiet install "$@"
 }

 function get_default_hostname {
--- a/setup/mail-dovecot.sh
+++ b/setup/mail-dovecot.sh
@@ -45,8 +45,8 @@ apt_install \
 # - https://www.dovecot.org/list/dovecot/2012-August/137569.html
 # - https://www.dovecot.org/list/dovecot/2011-December/132455.html
 tools/editconf.py /etc/dovecot/conf.d/10-master.conf \
-	default_process_limit=$(echo "`nproc` * 250" | bc) \
-	default_vsz_limit=$(echo "`free -tm  | tail -1 | awk '{print $2}'` / 3" | bc)M \
+	default_process_limit=$(echo "$(nproc) * 250" | bc) \
+	default_vsz_limit=$(echo "$(free -tm  | tail -1 | awk '{print $2}') / 3" | bc)M \
 	log_path=/var/log/mail.log

 # The inotify `max_user_instances` default is 128, which constrains
@@ -183,6 +183,7 @@ plugin {
  sieve_after = $STORAGE_ROOT/mail/sieve/global_after
  sieve = $STORAGE_ROOT/mail/sieve/%d/%n.sieve
  sieve_dir = $STORAGE_ROOT/mail/sieve/%d/%n
+  sieve_redirect_envelope_from = recipient
 }
 EOF

--- a/setup/mail-postfix.sh
+++ b/setup/mail-postfix.sh
@@ -17,7 +17,7 @@
 # LMTP. Spamassassin then passes mail over to Dovecot for
 # storage in the user's mailbox.
 #
-# Postfix also listens on port 587 (SMTP+STARTLS) for
+# Postfix also listens on ports 465/587 (SMTPS, SMTP+STARTLS) for
 # connections from users who can authenticate and then sends
 # their email out to the outside world. Postfix queries Dovecot
 # to authenticate users.
@@ -71,7 +71,7 @@ tools/editconf.py /etc/postfix/main.cf \

 # ### Outgoing Mail

-# Enable the 'submission' port 587 smtpd server and tweak its settings.
+# Enable the 'submission' ports 465 and 587 and tweak their settings.
 #
 # * Enable authentication. It's disabled globally so that it is disabled on port 25,
 #   so we need to explicitly enable it here.
@@ -80,13 +80,19 @@ tools/editconf.py /etc/postfix/main.cf \
 #   OpenDKIM milter only. See dkim.sh.
 # * Even though we dont allow auth over non-TLS connections (smtpd_tls_auth_only below, and without auth the client cant
 #   send outbound mail), don't allow non-TLS mail submission on this port anyway to prevent accidental misconfiguration.
-#   Setting smtpd_tls_security_level=encrypt also triggers the use of the 'mandatory' settings below.
+#   Setting smtpd_tls_security_level=encrypt also triggers the use of the 'mandatory' settings below (but this is ignored with smtpd_tls_wrappermode=yes.)
 # * Give it a different name in syslog to distinguish it from the port 25 smtpd server.
 # * Add a new cleanup service specific to the submission service ('authclean')
 #   that filters out privacy-sensitive headers on mail being sent out by
 #   authenticated users.  By default Postfix also applies this to attached
 #   emails but we turn this off by setting nested_header_checks empty.
 tools/editconf.py /etc/postfix/master.cf -s -w \
+	"smtps=inet n       -       -       -       -       smtpd
+	  -o smtpd_tls_wrappermode=yes
+	  -o smtpd_sasl_auth_enable=yes
+	  -o syslog_name=postfix/submission
+	  -o smtpd_milters=inet:127.0.0.1:8891
+	  -o cleanup_service_name=authclean" \
 	"submission=inet n       -       -       -       -       smtpd
 	  -o smtpd_sasl_auth_enable=yes
 	  -o syslog_name=postfix/submission
@@ -107,7 +113,7 @@ sed -i "s/PRIMARY_HOSTNAME/$PRIMARY_HOSTNAME/" /etc/postfix/outgoing_mail_header
 sed -i "s/PUBLIC_IP/$PUBLIC_IP/" /etc/postfix/outgoing_mail_header_filters

 # Enable TLS on incoming connections. It is not required on port 25, allowing for opportunistic
-# encryption. On port 587 it is mandatory (see above). Shared and non-shared settings are
+# encryption. On ports 465 and 587 it is mandatory (see above). Shared and non-shared settings are
 # given here. Shared settings include:
 # * Require TLS before a user is allowed to authenticate.
 # * Set the path to the server TLS certificate and 2048-bit DH parameters for old DH ciphers.
@@ -117,9 +123,6 @@ sed -i "s/PUBLIC_IP/$PUBLIC_IP/" /etc/postfix/outgoing_mail_header_filters
 #   won't fall back to cleartext. So we don't disable too much. smtpd_tls_exclude_ciphers applies to
 #   both port 25 and port 587, but because we override the cipher list for both, it probably isn't used.
 #   Use Mozilla's "Old" recommendations at https://ssl-config.mozilla.org/#server=postfix&server-version=3.3.0&config=old&openssl-version=1.1.1
-# For port 587 (via the 'mandatory' settings):
-# * Use Mozilla's "Intermediate" TLS recommendations from https://ssl-config.mozilla.org/#server=postfix&server-version=3.3.0&config=intermediate&openssl-version=1.1.1
-#   using and overriding the "high" cipher list so we don't conflict with the more permissive settings for port 25.
 tools/editconf.py /etc/postfix/main.cf \
 	smtpd_tls_security_level=may\
 	smtpd_tls_auth_only=yes \
@@ -130,18 +133,23 @@ tools/editconf.py /etc/postfix/main.cf \
 	smtpd_tls_ciphers=medium \
 	tls_medium_cipherlist=ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA \
 	smtpd_tls_exclude_ciphers=aNULL,RC4 \
+	tls_preempt_cipherlist=no \
+	smtpd_tls_received_header=yes
+
+# For ports 465/587 (via the 'mandatory' settings):
+# * Use Mozilla's "Intermediate" TLS recommendations from https://ssl-config.mozilla.org/#server=postfix&server-version=3.3.0&config=intermediate&openssl-version=1.1.1
+#   using and overriding the "high" cipher list so we don't conflict with the more permissive settings for port 25.
+tools/editconf.py /etc/postfix/main.cf \
 	smtpd_tls_mandatory_protocols="!SSLv2,!SSLv3,!TLSv1,!TLSv1.1" \
 	smtpd_tls_mandatory_ciphers=high \
 	tls_high_cipherlist=ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 \
-	smtpd_tls_mandatory_exclude_ciphers=aNULL,DES,3DES,MD5,DES+MD5,RC4 \
-	tls_preempt_cipherlist=no \
-	smtpd_tls_received_header=yes
+	smtpd_tls_mandatory_exclude_ciphers=aNULL,DES,3DES,MD5,DES+MD5,RC4

 # Prevent non-authenticated users from sending mail that requires being
 # relayed elsewhere. We don't want to be an "open relay". On outbound
 # mail, require one of:
 #
-# * `permit_sasl_authenticated`: Authenticated users (i.e. on port 587).
+# * `permit_sasl_authenticated`: Authenticated users (i.e. on port 465/587).
 # * `permit_mynetworks`: Mail that originates locally.
 # * `reject_unauth_destination`: No one else. (Permits mail whose destination is local and rejects other mail.)
 tools/editconf.py /etc/postfix/main.cf \
@@ -191,7 +199,7 @@ tools/editconf.py /etc/postfix/main.cf \
 #
 # In a basic setup we would pass mail directly to Dovecot by setting
 # virtual_transport to `lmtp:unix:private/dovecot-lmtp`.
-tools/editconf.py /etc/postfix/main.cf virtual_transport=lmtp:[127.0.0.1]:10025
+tools/editconf.py /etc/postfix/main.cf "virtual_transport=lmtp:[127.0.0.1]:10025"
 # Because of a spampd bug, limit the number of recipients in each connection.
 # See https://github.com/mail-in-a-box/mailinabox/issues/1523.
 tools/editconf.py /etc/postfix/main.cf lmtp_destination_recipient_limit=1
@@ -263,6 +271,7 @@ tools/editconf.py /etc/postfix/main.cf \
 # Allow the two SMTP ports in the firewall.

 ufw_allow smtp
+ufw_allow smtps
 ufw_allow submission

 # Restart services
--- a/setup/mail-users.sh
+++ b/setup/mail-users.sh
@@ -22,6 +22,7 @@ if [ ! -f $db_path ]; then
 	echo Creating new user database: $db_path;
 	echo "CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, email TEXT NOT NULL UNIQUE, password TEXT NOT NULL, extra, privileges TEXT NOT NULL DEFAULT '');" | sqlite3 $db_path;
 	echo "CREATE TABLE aliases (id INTEGER PRIMARY KEY AUTOINCREMENT, source TEXT NOT NULL UNIQUE, destination TEXT NOT NULL, permitted_senders TEXT);" | sqlite3 $db_path;
+	echo "CREATE TABLE mfa (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, type TEXT NOT NULL, secret TEXT NOT NULL, mru_token TEXT, label TEXT, FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE);" | sqlite3 $db_path;
 fi

 # ### User Authentication
--- a/setup/management.sh
+++ b/setup/management.sh
@@ -18,11 +18,7 @@ while [ -d /usr/local/lib/python3.4/dist-packages/acme ]; do
 	pip3 uninstall -y acme;
 done

-# duplicity is used to make backups of user data. It uses boto
-# (via Python 2) to do backups to AWS S3. boto from the Ubuntu
-# package manager is too out-of-date -- it doesn't support the newer
-# S3 api used in some regions, which breaks backups to those regions.
-# See #627, #653.
+# duplicity is used to make backups of user data.
 #
 # virtualenv is used to isolate the Python 3 packages we
 # install via pip from the system-installed packages.
@@ -30,7 +26,11 @@ done
 # certbot installs EFF's certbot which we use to
 # provision free TLS certificates.
 apt_install duplicity python-pip virtualenv certbot
-hide_output pip2 install --upgrade boto
+
+# b2sdk is used for backblaze backups.
+# boto is used for amazon aws backups.
+# Both are installed outside the pipenv, so they can be used by duplicity
+hide_output pip3 install --upgrade b2sdk boto

 # Create a virtualenv for the installation of Python 3 packages
 # used by the management daemon.
@@ -50,7 +50,8 @@ hide_output $venv/bin/pip install --upgrade pip
 hide_output $venv/bin/pip install --upgrade \
 	rtyaml "email_validator>=1.0.0" "exclusiveprocess" \
 	flask dnspython python-dateutil \
-	"idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver
+  qrcode[pil] pyotp \
+	"idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver b2sdk

 # CONFIGURATION

@@ -89,8 +90,14 @@ rm -f /tmp/bootstrap.zip
 # running after a reboot.
 cat > $inst_dir/start <<EOF;
 #!/bin/bash
+# Set character encoding flags to ensure that any non-ASCII don't cause problems.
+export LANGUAGE=en_US.UTF-8
+export LC_ALL=en_US.UTF-8
+export LANG=en_US.UTF-8
+export LC_TYPE=en_US.UTF-8
+
 source $venv/bin/activate
-exec python `pwd`/management/daemon.py
+exec python $(pwd)/management/daemon.py
 EOF
 chmod +x $inst_dir/start
 cp --remove-destination conf/mailinabox.service /lib/systemd/system/mailinabox.service # target was previously a symlink so remove it first
@@ -105,7 +112,7 @@ minute=$((RANDOM % 60))  # avoid overloading mailinabox.email
 cat > /etc/cron.d/mailinabox-nightly << EOF;
 # Mail-in-a-Box --- Do not edit / will be overwritten on update.
 # Run nightly tasks: backup, status checks.
-$minute 3 * * *	root	(cd `pwd` && management/daily_tasks.sh)
+$minute 3 * * *	root	(cd $(pwd) && management/daily_tasks.sh)
 EOF

 # Start the management server.
--- a/setup/migrate.py
+++ b/setup/migrate.py
@@ -181,6 +181,12 @@ def migration_12(env):
            conn.commit()
            conn.close()

+def migration_13(env):
+	# Add the "mfa" table for configuring MFA for login to the control panel.
+	db = os.path.join(env["STORAGE_ROOT"], 'mail/users.sqlite')
+	shell("check_call", ["sqlite3", db, "CREATE TABLE mfa (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, type TEXT NOT NULL, secret TEXT NOT NULL, mru_token TEXT, label TEXT, FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE);"])
+
+###########################################################

 def get_current_migration():
 	ver = 0
--- a/setup/nextcloud.sh
+++ b/setup/nextcloud.sh
@@ -14,12 +14,18 @@ apt-get purge -qq -y owncloud* # we used to use the package manager
 apt_install php php-fpm \
 	php-cli php-sqlite3 php-gd php-imap php-curl php-pear curl \
 	php-dev php-gd php-xml php-mbstring php-zip php-apcu php-json \
-	php-intl php-imagick
+	php-intl php-imagick php-gmp php-bcmath

 InstallNextcloud() {

 	version=$1
 	hash=$2
+	version_contacts=$3
+	hash_contacts=$4
+	version_calendar=$5
+	hash_calendar=$6
+	version_user_external=${7:-}
+	hash_user_external=${8:-}

 	echo
 	echo "Upgrading to Nextcloud version $version"
@@ -40,18 +46,18 @@ InstallNextcloud() {
 	# their github repositories.
 	mkdir -p /usr/local/lib/owncloud/apps

-	wget_verify https://github.com/nextcloud/contacts/releases/download/v3.3.0/contacts.tar.gz e55d0357c6785d3b1f3b5f21780cb6d41d32443a /tmp/contacts.tgz
+	wget_verify https://github.com/nextcloud/contacts/releases/download/v$version_contacts/contacts.tar.gz $hash_contacts /tmp/contacts.tgz
 	tar xf /tmp/contacts.tgz -C /usr/local/lib/owncloud/apps/
 	rm /tmp/contacts.tgz

-	wget_verify https://github.com/nextcloud/calendar/releases/download/v2.0.3/calendar.tar.gz 9d9717b29337613b72c74e9914c69b74b346c466 /tmp/calendar.tgz
+	wget_verify https://github.com/nextcloud/calendar/releases/download/v$version_calendar/calendar.tar.gz $hash_calendar /tmp/calendar.tgz
 	tar xf /tmp/calendar.tgz -C /usr/local/lib/owncloud/apps/
 	rm /tmp/calendar.tgz

 	# Starting with Nextcloud 15, the app user_external is no longer included in Nextcloud core,
 	# we will install from their github repository.
-	if [[ $version =~ ^1[567] ]]; then
-		wget_verify https://github.com/nextcloud/user_external/releases/download/v0.7.0/user_external-0.7.0.tar.gz 555a94811daaf5bdd336c5e48a78aa8567b86437 /tmp/user_external.tgz
+	if [ -n "$version_user_external" ]; then
+		wget_verify https://github.com/nextcloud/user_external/releases/download/v$version_user_external/user_external-$version_user_external.tar.gz $hash_user_external /tmp/user_external.tgz
 		tar -xf /tmp/user_external.tgz -C /usr/local/lib/owncloud/apps/
 		rm /tmp/user_external.tgz
 	fi
@@ -91,8 +97,14 @@ InstallNextcloud() {
 }

 # Nextcloud Version to install. Checks are done down below to step through intermediate versions.
-nextcloud_ver=17.0.6
-nextcloud_hash=50b98d2c2f18510b9530e558ced9ab51eb4f11b0
+nextcloud_ver=20.0.8
+nextcloud_hash=372b0b4bb07c7984c04917aff86b280e68fbe761
+contacts_ver=3.5.1
+contacts_hash=d2ffbccd3ed89fa41da20a1dff149504c3b33b93
+calendar_ver=2.2.0
+calendar_hash=673ad72ca28adb8d0f209015ff2dca52ffad99af
+user_external_ver=1.0.0
+user_external_hash=3bf2609061d7214e7f0f69dd8883e55c4ec8f50a

 # Current Nextcloud Version, #1623
 # Checking /usr/local/lib/owncloud/version.php shows version of the Nextcloud application, not the DB
@@ -116,7 +128,7 @@ if [ ! -d /usr/local/lib/owncloud/ ] || [[ ! ${CURRENT_NEXTCLOUD_VER} =~ ^$nextc

 	# Backup the existing ownCloud/Nextcloud.
 	# Create a backup directory to store the current installation and database to
-	BACKUP_DIRECTORY=$STORAGE_ROOT/owncloud-backup/`date +"%Y-%m-%d-%T"`
+	BACKUP_DIRECTORY=$STORAGE_ROOT/owncloud-backup/$(date +"%Y-%m-%d-%T")
 	mkdir -p "$BACKUP_DIRECTORY"
 	if [ -d /usr/local/lib/owncloud/ ]; then
 		echo "Upgrading Nextcloud --- backing up existing installation, configuration, and database to directory to $BACKUP_DIRECTORY..."
@@ -141,23 +153,39 @@ if [ ! -d /usr/local/lib/owncloud/ ] || [[ ! ${CURRENT_NEXTCLOUD_VER} =~ ^$nextc
 			return 0
 		elif [[ ${CURRENT_NEXTCLOUD_VER} =~ ^13 ]]; then
 			# If we are running Nextcloud 13, upgrade to Nextcloud 14
-			InstallNextcloud 14.0.6 4e43a57340f04c2da306c8eea98e30040399ae5a
+			InstallNextcloud 14.0.6 4e43a57340f04c2da306c8eea98e30040399ae5a 3.3.0 e55d0357c6785d3b1f3b5f21780cb6d41d32443a 2.0.3 9d9717b29337613b72c74e9914c69b74b346c466
 			CURRENT_NEXTCLOUD_VER="14.0.6"
 		fi
 		if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^14 ]]; then
 			# During the upgrade from Nextcloud 14 to 15, user_external may cause the upgrade to fail.
 			# We will disable it here before the upgrade and install it again after the upgrade.
 			hide_output sudo -u www-data php /usr/local/lib/owncloud/console.php app:disable user_external
-			InstallNextcloud 15.0.8 4129d8d4021c435f2e86876225fb7f15adf764a3
+			InstallNextcloud 15.0.8 4129d8d4021c435f2e86876225fb7f15adf764a3 3.3.0 e55d0357c6785d3b1f3b5f21780cb6d41d32443a 2.0.3 9d9717b29337613b72c74e9914c69b74b346c466 0.7.0 555a94811daaf5bdd336c5e48a78aa8567b86437
 			CURRENT_NEXTCLOUD_VER="15.0.8"
 		fi
 		if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^15 ]]; then
-                        InstallNextcloud 16.0.6 0bb3098455ec89f5af77a652aad553ad40a88819
+                        InstallNextcloud 16.0.6 0bb3098455ec89f5af77a652aad553ad40a88819 3.3.0 e55d0357c6785d3b1f3b5f21780cb6d41d32443a 2.0.3 9d9717b29337613b72c74e9914c69b74b346c466 0.7.0 555a94811daaf5bdd336c5e48a78aa8567b86437
                        CURRENT_NEXTCLOUD_VER="16.0.6"
        	fi
+	        if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^16 ]]; then
+                        InstallNextcloud 17.0.6 50b98d2c2f18510b9530e558ced9ab51eb4f11b0 3.3.0 e55d0357c6785d3b1f3b5f21780cb6d41d32443a 2.0.3 9d9717b29337613b72c74e9914c69b74b346c466 0.7.0 555a94811daaf5bdd336c5e48a78aa8567b86437
+                        CURRENT_NEXTCLOUD_VER="17.0.6"
+	        fi
+	        if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^17 ]]; then
+			echo "ALTER TABLE oc_flow_operations ADD COLUMN entity VARCHAR;" | sqlite3 $STORAGE_ROOT/owncloud/owncloud.db
+                        InstallNextcloud 18.0.10 39c0021a8b8477c3f1733fddefacfa5ebf921c68 3.4.1 aee680a75e95f26d9285efd3c1e25cf7f3bfd27e 2.0.3 9d9717b29337613b72c74e9914c69b74b346c466 1.0.0 3bf2609061d7214e7f0f69dd8883e55c4ec8f50a
+                        CURRENT_NEXTCLOUD_VER="18.0.10"
+	        fi
+	        if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^18 ]]; then
+                        InstallNextcloud 19.0.4 01e98791ba12f4860d3d4047b9803f97a1b55c60 3.4.1 aee680a75e95f26d9285efd3c1e25cf7f3bfd27e 2.0.3 9d9717b29337613b72c74e9914c69b74b346c466 1.0.0 3bf2609061d7214e7f0f69dd8883e55c4ec8f50a
+                        CURRENT_NEXTCLOUD_VER="19.0.4"
+	        fi
 	fi

-	InstallNextcloud $nextcloud_ver $nextcloud_hash
+	InstallNextcloud $nextcloud_ver $nextcloud_hash $contacts_ver $contacts_hash $calendar_ver $calendar_hash $user_external_ver $user_external_hash
+
+	# Nextcloud 20 needs to have some optional columns added
+	sudo -u www-data php /usr/local/lib/owncloud/occ db:add-missing-columns
 fi

 # ### Configuring Nextcloud
@@ -283,6 +311,11 @@ hide_output sudo -u www-data php /usr/local/lib/owncloud/console.php app:enable
 sudo -u www-data php /usr/local/lib/owncloud/occ upgrade
 if [ \( $? -ne 0 \) -a \( $? -ne 3 \) ]; then exit 1; fi

+# Disable default apps that we don't support
+sudo -u www-data \
+	php /usr/local/lib/owncloud/occ app:disable photos dashboard activity \
+	| (grep -v "No such app enabled" || /bin/true)
+
 # Set PHP FPM values to support large file uploads
 # (semicolon is the comment character in this file, hashes produce deprecation warnings)
 tools/editconf.py /etc/php/7.2/fpm/php.ini -c ';' \
@@ -324,7 +357,7 @@ rm -f /etc/cron.hourly/mailinabox-owncloud
 # and there's a lot they could mess up, so we don't make any users admins of Nextcloud.
 # But if we wanted to, we would do this:
 # ```
-# for user in $(tools/mail.py user admins); do
+# for user in $(management/cli.py user admins); do
 #	 sqlite3 $STORAGE_ROOT/owncloud/owncloud.db "INSERT OR IGNORE INTO oc_group_user VALUES ('admin', '$user')"
 # done
 # ```
--- a/setup/preflight.sh
+++ b/setup/preflight.sh
@@ -8,7 +8,7 @@ if [[ $EUID -ne 0 ]]; then
 fi

 # Check that we are running on Ubuntu 18.04 LTS (or 18.04.xx).
-if [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' `" != "Ubuntu 18.04 LTS" ]; then
+if [ "$(lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' )" != "Ubuntu 18.04 LTS" ]; then
 	echo "Mail-in-a-Box only supports being installed on Ubuntu 18.04, sorry. You are running:"
 	echo
 	lsb_release -d | sed 's/.*:\s*//'
--- a/setup/spamassassin.sh
+++ b/setup/spamassassin.sh
@@ -67,6 +67,56 @@ tools/editconf.py /etc/spamassassin/local.cf -s \
 	"add_header all Report"=_REPORT_ \
 	"add_header all Score"=_SCORE_

+
+# Authentication-Results SPF/Dmarc checks
+# ---------------------------------------
+# OpenDKIM and OpenDMARC are configured to validate and add "Authentication-Results: ..."
+# headers by checking the sender's SPF & DMARC policies. Instead of blocking mail that fails
+# these checks, we can use these headers to evaluate the mail as spam.
+#
+# Our custom rules are added to their own file so that an update to the deb package config
+# does not remove our changes.
+#
+# We need to escape period's in $PRIMARY_HOSTNAME since spamassassin config uses regex.
+
+escapedprimaryhostname="${PRIMARY_HOSTNAME//./\\.}"
+
+cat > /etc/spamassassin/miab_spf_dmarc.cf << EOF
+# Evaluate DMARC Authentication-Results
+header DMARC_PASS Authentication-Results =~ /$escapedprimaryhostname; dmarc=pass/
+describe DMARC_PASS DMARC check passed
+score DMARC_PASS -0.1
+
+header DMARC_NONE Authentication-Results =~ /$escapedprimaryhostname; dmarc=none/
+describe DMARC_NONE DMARC record not found
+score DMARC_NONE 0.1
+
+header DMARC_FAIL_NONE Authentication-Results =~ /$escapedprimaryhostname; dmarc=fail \(p=none/
+describe DMARC_FAIL_NONE DMARC check failed (p=none)
+score DMARC_FAIL_NONE 2.0
+
+header DMARC_FAIL_QUARANTINE Authentication-Results =~ /$escapedprimaryhostname; dmarc=fail \(p=quarantine/
+describe DMARC_FAIL_QUARANTINE DMARC check failed (p=quarantine)
+score DMARC_FAIL_QUARANTINE 5.0
+
+header DMARC_FAIL_REJECT Authentication-Results =~ /$escapedprimaryhostname; dmarc=fail \(p=reject/
+describe DMARC_FAIL_REJECT DMARC check failed (p=reject)
+score DMARC_FAIL_REJECT 10.0
+
+# Evaluate SPF Authentication-Results
+header SPF_PASS Authentication-Results =~ /$escapedprimaryhostname; spf=pass/
+describe SPF_PASS SPF check passed
+score SPF_PASS -0.1
+
+header SPF_NONE Authentication-Results =~ /$escapedprimaryhostname; spf=none/
+describe SPF_NONE SPF record not found
+score SPF_NONE 2.0
+
+header SPF_FAIL Authentication-Results =~ /$escapedprimaryhostname; spf=fail/
+describe SPF_FAIL SPF check failed
+score SPF_FAIL 5.0
+EOF
+
 # Bayesean learning
 # -----------------
 #
--- a/setup/ssl.sh
+++ b/setup/ssl.sh
@@ -10,7 +10,7 @@
 #
 #  * DNSSEC DANE TLSA records
 #  * IMAP
-#  * SMTP (opportunistic TLS for port 25 and submission on port 587)
+#  * SMTP (opportunistic TLS for port 25 and submission on ports 465/587)
 #  * HTTPS
 #
 # The certificate is created with its CN set to the PRIMARY_HOSTNAME. It is
--- a/setup/start.sh
+++ b/setup/start.sh
@@ -46,7 +46,7 @@ fi
 # in the first dialog prompt, so we should do this before that starts.
 cat > /usr/local/bin/mailinabox << EOF;
 #!/bin/bash
-cd `pwd`
+cd $(pwd)
 source setup/start.sh
 EOF
 chmod +x /usr/local/bin/mailinabox
@@ -78,7 +78,7 @@ if [ ! -d $STORAGE_ROOT ]; then
 	mkdir -p $STORAGE_ROOT
 fi
 if [ ! -f $STORAGE_ROOT/mailinabox.version ]; then
-	echo $(setup/migrate.py --current) > $STORAGE_ROOT/mailinabox.version
+	setup/migrate.py --current > $STORAGE_ROOT/mailinabox.version
 	chown $STORAGE_USER.$STORAGE_USER $STORAGE_ROOT/mailinabox.version
 fi

@@ -94,7 +94,7 @@ PUBLIC_IP=$PUBLIC_IP
 PUBLIC_IPV6=$PUBLIC_IPV6
 PRIVATE_IP=$PRIVATE_IP
 PRIVATE_IPV6=$PRIVATE_IPV6
-MTA_STS_MODE=${MTA_STS_MODE-}
+MTA_STS_MODE=${DEFAULT_MTA_STS_MODE:-enforce}
 EOF

 # Start service configuration.
--- a/setup/system.sh
+++ b/setup/system.sh
@@ -93,6 +93,9 @@ hide_output add-apt-repository -y universe
 # Install the certbot PPA.
 hide_output add-apt-repository -y ppa:certbot/certbot

+# Install the duplicity PPA.
+hide_output add-apt-repository -y ppa:duplicity-team/duplicity-release-git
+
 # ### Update Packages

 # Update system packages to make sure we have the latest upstream versions
@@ -128,7 +131,7 @@ apt_get_quiet autoremove
 # * openssh-client: provides ssh-keygen

 echo Installing system packages...
-apt_install python3 python3-dev python3-pip \
+apt_install python3 python3-dev python3-pip python3-setuptools \
 	netcat-openbsd wget curl git sudo coreutils bc \
 	haveged pollinate openssh-client unzip \
 	unattended-upgrades cron ntp fail2ban rsyslog
@@ -317,6 +320,9 @@ fi #NODOC
 #   name server, on IPV6.
 # * The listen-on directive in named.conf.options restricts `bind9` to
 #   binding to the loopback interface instead of all interfaces.
+# * The max-recursion-queries directive increases the maximum number of iterative queries.
+#  	If more queries than specified are sent, bind9 returns SERVFAIL. After flushing the cache during system checks,
+#	we ran into the limit thus we are increasing it from 75 (default value) to 100.
 apt_install bind9
 tools/editconf.py /etc/default/bind9 \
 	"OPTIONS=\"-u bind -4\""
@@ -324,6 +330,10 @@ if ! grep -q "listen-on " /etc/bind/named.conf.options; then
 	# Add a listen-on directive if it doesn't exist inside the options block.
 	sed -i "s/^}/\n\tlisten-on { 127.0.0.1; };\n}/" /etc/bind/named.conf.options
 fi
+if ! grep -q "max-recursion-queries " /etc/bind/named.conf.options; then
+	# Add a max-recursion-queries directive if it doesn't exist inside the options block.
+	sed -i "s/^}/\n\tmax-recursion-queries 100;\n}/" /etc/bind/named.conf.options
+fi

 # First we'll disable systemd-resolved's management of resolv.conf and its stub server.
 # Breaking the symlink to /run/systemd/resolve/stub-resolv.conf means
--- a/setup/web.sh
+++ b/setup/web.sh
@@ -126,13 +126,13 @@ chmod a+r /var/lib/mailinabox/mozilla-autoconfig.xml
 # nginx configuration at /.well-known/mta-sts.txt
 # more documentation is available on: 
 # https://www.uriports.com/blog/mta-sts-explained/
-# default mode is "enforce". Change to "testing" which means
-# "Messages will be delivered as though there was no failure
-# but a report will be sent if TLS-RPT is configured" if you
-# are not sure you want this yet. Or "none".
+# default mode is "enforce". In /etc/mailinabox.conf change
+# "MTA_STS_MODE=testing" which means "Messages will be delivered
+# as though there was no failure but a report will be sent if
+# TLS-RPT is configured" if you are not sure you want this yet. Or "none".
 PUNY_PRIMARY_HOSTNAME=$(echo "$PRIMARY_HOSTNAME" | idn2)
 cat conf/mta-sts.txt \
-        | sed "s/MODE/${MTA_STS_MODE:-enforce}/" \
+        | sed "s/MODE/${MTA_STS_MODE}/" \
        | sed "s/PRIMARY_HOSTNAME/$PUNY_PRIMARY_HOSTNAME/" \
         > /var/lib/mailinabox/mta-sts.txt
 chmod a+r /var/lib/mailinabox/mta-sts.txt
--- a/setup/webmail.sh
+++ b/setup/webmail.sh
@@ -28,10 +28,11 @@ apt_install \
 # Install Roundcube from source if it is not already present or if it is out of date.
 # Combine the Roundcube version number with the commit hash of plugins to track
 # whether we have the latest version of everything.
-VERSION=1.4.8
-HASH=3a6824fd68fef2e0d24f186cfbee5c6f9d6edbe9
-PERSISTENT_LOGIN_VERSION=6b3fc450cae23ccb2f393d0ef67aa319e877e435
-HTML5_NOTIFIER_VERSION=4b370e3cd60dabd2f428a26f45b677ad1b7118d5
+
+VERSION=1.4.11
+HASH=3877f0e70f29e7d0612155632e48c3db1e626be3
+PERSISTENT_LOGIN_VERSION=6b3fc450cae23ccb2f393d0ef67aa319e877e435 # version 5.2.0
+HTML5_NOTIFIER_VERSION=68d9ca194212e15b3c7225eb6085dbcf02fd13d7 # version 0.6.4+
 CARDDAV_VERSION=3.0.3
 CARDDAV_HASH=d1e3b0d851ffa2c6bd42bf0c04f70d0e1d0d78f8

@@ -46,7 +47,7 @@ needs_update=0 #NODOC
 if [ ! -f /usr/local/lib/roundcubemail/version ]; then
 	# not installed yet #NODOC
 	needs_update=1 #NODOC
-elif [[ "$UPDATE_KEY" != `cat /usr/local/lib/roundcubemail/version` ]]; then
+elif [[ "$UPDATE_KEY" != $(cat /usr/local/lib/roundcubemail/version) ]]; then
 	# checks if the version is what we want
 	needs_update=1 #NODOC
 fi
@@ -90,8 +91,9 @@ fi

 # ### Configuring Roundcube

-# Generate a safe 24-character secret key of safe characters.
-SECRET_KEY=$(dd if=/dev/urandom bs=1 count=18 2>/dev/null | base64 | fold -w 24 | head -n 1)
+# Generate a secret key of PHP-string-safe characters appropriate
+# for the cipher algorithm selected below.
+SECRET_KEY=$(dd if=/dev/urandom bs=1 count=32 2>/dev/null | base64 | sed s/=//g)

 # Create a configuration file.
 #
@@ -125,7 +127,8 @@ cat > $RCM_CONFIG <<EOF;
 );
 \$config['support_url'] = 'https://mailinabox.email/';
 \$config['product_name'] = '$PRIMARY_HOSTNAME Webmail';
-\$config['des_key'] = '$SECRET_KEY';
+\$config['cipher_method'] = 'AES-256-CBC'; # persistent login cookie and potentially other things
+\$config['des_key'] = '$SECRET_KEY'; # 37 characters -> ~256 bits for AES-256, see above
 \$config['plugins'] = array('html5_notifier', 'archive', 'zipdownload', 'password', 'managesieve', 'jqueryui', 'persistent_login', 'carddav');
 \$config['skin'] = 'elastic';
 \$config['login_autocomplete'] = 2;
--- a/setup/zpush.sh
+++ b/setup/zpush.sh
@@ -22,23 +22,23 @@ apt_install \
 phpenmod -v php imap

 # Copy Z-Push into place.
-VERSION=2.5.2
-TARGETHASH=2dc3dbd791b96b0ba2638df0d3d1e03c7e1cbab2
+VERSION=2.6.2
+TARGETHASH=f0e8091a8030e5b851f5ba1f9f0e1a05b8762d80
 needs_update=0 #NODOC
 if [ ! -f /usr/local/lib/z-push/version ]; then
 	needs_update=1 #NODOC
-elif [[ $VERSION != `cat /usr/local/lib/z-push/version` ]]; then
+elif [[ $VERSION != $(cat /usr/local/lib/z-push/version) ]]; then
 	# checks if the version
 	needs_update=1 #NODOC
 fi
 if [ $needs_update == 1 ]; then
 	# Download
-	wget_verify "https://stash.z-hub.io/rest/api/latest/projects/ZP/repos/z-push/archive?at=refs%2Ftags%2F$VERSION&format=zip" $TARGETHASH /tmp/z-push.zip
+	wget_verify "https://github.com/Z-Hub/Z-Push/archive/refs/tags/$VERSION.zip" $TARGETHASH /tmp/z-push.zip

 	# Extract into place.
 	rm -rf /usr/local/lib/z-push /tmp/z-push
 	unzip -q /tmp/z-push.zip -d /tmp/z-push
-	mv /tmp/z-push/src /usr/local/lib/z-push
+	mv /tmp/z-push/*/src /usr/local/lib/z-push
 	rm -rf /tmp/z-push.zip /tmp/z-push

 	rm -f /usr/sbin/z-push-{admin,top}
--- a/tests/test_mail.py
+++ b/tests/test_mail.py
@@ -41,9 +41,8 @@ This is a test message. It should be automatically deleted by the test script.""
 	)

 # Connect to the server on the SMTP submission TLS port.
-server = smtplib.SMTP(host, 587)
+server = smtplib.SMTP_SSL(host)
 #server.set_debuglevel(1)
-server.starttls()

 # Verify that the EHLO name matches the server's reverse DNS.
 ipaddr = socket.gethostbyname(host) # IPv4 only!
--- a/tools/mail.py
+++ b/tools/mail.py
@@ -1,128 +1,3 @@
-#!/usr/bin/python3
-
-import sys, getpass, urllib.request, urllib.error, json, re
-
-def mgmt(cmd, data=None, is_json=False):
-	# The base URL for the management daemon. (Listens on IPv4 only.)
-	mgmt_uri = 'http://127.0.0.1:10222'
-
-	setup_key_auth(mgmt_uri)
-
-	req = urllib.request.Request(mgmt_uri + cmd, urllib.parse.urlencode(data).encode("utf8") if data else None)
-	try:
-		response = urllib.request.urlopen(req)
-	except urllib.error.HTTPError as e:
-		if e.code == 401:
-			try:
-				print(e.read().decode("utf8"))
-			except:
-				pass
-			print("The management daemon refused access. The API key file may be out of sync. Try 'service mailinabox restart'.", file=sys.stderr)
-		elif hasattr(e, 'read'):
-			print(e.read().decode('utf8'), file=sys.stderr)
-		else:
-			print(e, file=sys.stderr)
-		sys.exit(1)
-	resp = response.read().decode('utf8')
-	if is_json: resp = json.loads(resp)
-	return resp
-
-def read_password():
-    while True:
-        first = getpass.getpass('password: ')
-        if len(first) < 8:
-            print("Passwords must be at least eight characters.")
-            continue
-        second = getpass.getpass(' (again): ')
-        if first != second:
-            print("Passwords not the same. Try again.")
-            continue
-        break
-    return first
-
-def setup_key_auth(mgmt_uri):
-	key = open('/var/lib/mailinabox/api.key').read().strip()
-
-	auth_handler = urllib.request.HTTPBasicAuthHandler()
-	auth_handler.add_password(
-		realm='Mail-in-a-Box Management Server',
-		uri=mgmt_uri,
-		user=key,
-		passwd='')
-	opener = urllib.request.build_opener(auth_handler)
-	urllib.request.install_opener(opener)
-
-if len(sys.argv) < 2:
-	print("Usage: ")
-	print("  tools/mail.py user  (lists users)")
-	print("  tools/mail.py user add user@domain.com [password]")
-	print("  tools/mail.py user password user@domain.com [password]")
-	print("  tools/mail.py user remove user@domain.com")
-	print("  tools/mail.py user make-admin user@domain.com")
-	print("  tools/mail.py user remove-admin user@domain.com")
-	print("  tools/mail.py user admins (lists admins)")
-	print("  tools/mail.py alias  (lists aliases)")
-	print("  tools/mail.py alias add incoming.name@domain.com sent.to@other.domain.com")
-	print("  tools/mail.py alias add incoming.name@domain.com 'sent.to@other.domain.com, multiple.people@other.domain.com'")
-	print("  tools/mail.py alias remove incoming.name@domain.com")
-	print()
-	print("Removing a mail user does not delete their mail folders on disk. It only prevents IMAP/SMTP login.")
-	print()
-
-elif sys.argv[1] == "user" and len(sys.argv) == 2:
-	# Dump a list of users, one per line. Mark admins with an asterisk.
-	users = mgmt("/mail/users?format=json", is_json=True)
-	for domain in users:
-		for user in domain["users"]:
-			if user['status'] == 'inactive': continue
-			print(user['email'], end='')
-			if "admin" in user['privileges']:
-				print("*", end='')
-			print()
-
-elif sys.argv[1] == "user" and sys.argv[2] in ("add", "password"):
-	if len(sys.argv) < 5:
-		if len(sys.argv) < 4:
-			email = input("email: ")
-		else:
-			email = sys.argv[3]
-		pw = read_password()
-	else:
-		email, pw = sys.argv[3:5]
-
-	if sys.argv[2] == "add":
-		print(mgmt("/mail/users/add", { "email": email, "password": pw }))
-	elif sys.argv[2] == "password":
-		print(mgmt("/mail/users/password", { "email": email, "password": pw }))
-
-elif sys.argv[1] == "user" and sys.argv[2] == "remove" and len(sys.argv) == 4:
-	print(mgmt("/mail/users/remove", { "email": sys.argv[3] }))
-
-elif sys.argv[1] == "user" and sys.argv[2] in ("make-admin", "remove-admin") and len(sys.argv) == 4:
-	if sys.argv[2] == "make-admin":
-		action = "add"
-	else:
-		action = "remove"
-	print(mgmt("/mail/users/privileges/" + action, { "email": sys.argv[3], "privilege": "admin" }))
-
-elif sys.argv[1] == "user" and sys.argv[2] == "admins":
-	# Dump a list of admin users.
-	users = mgmt("/mail/users?format=json", is_json=True)
-	for domain in users:
-		for user in domain["users"]:
-			if "admin" in user['privileges']:
-				print(user['email'])
-
-elif sys.argv[1] == "alias" and len(sys.argv) == 2:
-	print(mgmt("/mail/aliases"))
-
-elif sys.argv[1] == "alias" and sys.argv[2] == "add" and len(sys.argv) == 5:
-	print(mgmt("/mail/aliases/add", { "address": sys.argv[3], "forwards_to": sys.argv[4] }))
-
-elif sys.argv[1] == "alias" and sys.argv[2] == "remove" and len(sys.argv) == 4:
-	print(mgmt("/mail/aliases/remove", { "address": sys.argv[3] }))
-
-else:
-	print("Invalid command-line arguments.")
-	sys.exit(1)
-
+#!/bin/bash
+# This script has moved.
+management/cli.py "$@"
Author	SHA1	Message	Date
Joshua Tauberer	4cb46ea465	v0.54	2021-06-20 15:50:04 -04:00
Joshua Tauberer	35fa3fe891	Changelog entries	2021-05-15 16:50:19 -04:00
Joshua Tauberer	d510c8ae2a	Enable and recommend port 465 for mail submission instead of port 587 (fixes #1849 ) Port 465 with "implicit" (i.e. always-on) TLS is a more secure approach than port 587 with explicit (i.e. optional and only on with STARTTLS). Although we reject credentials on port 587 without STARTTLS, by that point credentials have already been sent.	2021-05-15 16:42:14 -04:00
Joshua Tauberer	e283a12047	Add null SPF, DMARC, and MX records for automatically generated autoconfig, autodiscover, and mta-sts subdomains; add null MX records for custom A-record subdomains All A/AAAA-resolvable domains that don't send or receive mail should have these null records. This simplifies the handling of domains a bit by handling automatically generated subdomains more like other domains.	2021-05-15 16:42:14 -04:00
Joshua Tauberer	e421addf1c	Pre-load domain purpopses when building DNS zonefiles rather than querying mail domains at each subdomain	2021-05-09 08:16:07 -04:00
Joshua Tauberer	354a774989	Remove a debug line added in `8cda58fb`	2021-05-09 07:34:44 -04:00
Joshua Tauberer	aaa81ec879	Fix indentation issue in `bc4ae51c2d`	2021-05-08 09:06:18 -04:00
Joshua Tauberer	dbd6dae5ce	Fix exit status issue cased by `69fc2fdd`	2021-05-08 09:02:48 -04:00
John @ S4	d4c5872547	Make clear that non-AWS S3 backups are supported (#1947 ) Just a few wording changes to show that it is possible to make S3 backups to other services than AWS - prompted by a thread on MIAB discourse.	2021-05-08 08:32:58 -04:00
Thomas Urban	3701e05d92	Rewrite envelope from address in sieve forwards (#1949 ) Fixes #1946.	2021-05-08 08:30:53 -04:00
Hala Alajlan	bc4ae51c2d	Handle query dns timeout unhandled error (#1950 ) Co-authored-by: hala alajlan <halalajlan@gmail.com>	2021-05-08 08:26:40 -04:00
Jawad Seddar	12aaebfc54	`custom.yaml`: add support for X-Frame-Options header and proxy_redirect off (#1954 )	2021-05-08 08:25:33 -04:00
jvolkenant	49813534bd	Updated Nextcloud to 20.0.8, contacts to 3.5.1, calendar to 2.2.0 (#1960 )	2021-05-08 08:24:04 -04:00
jvolkenant	16e81e1439	Fix to allow for non forced "enforce" MTA_STS_MODE (#1970 )	2021-05-08 08:18:49 -04:00
Joshua Tauberer	b7b67e31b7	Merged point release branch for v0.53a Changed the Z-Push download URL.	2021-05-08 08:14:39 -04:00
Joshua Tauberer	2e7f2835e7	v0.53a	2021-05-08 08:13:37 -04:00
Joshua Tauberer	8a5f9f464a	Download Z-Push from alternate site The old server has been down for a few days. Solution from https://discourse.mailinabox.email/t/temporary-fix-for-failed-wget-o-tmp-z-push-zip-https-stash-z-hub-io/8028. Fixes #1974.	2021-05-08 07:59:53 -04:00
Joshua Tauberer	69fc2fdd3a	Hide spurrious Nextcloud setup output	2021-05-03 19:41:00 -04:00
Joshua Tauberer	9b07d86bf7	Use $(...) notation instead of legacy backtick notation for embedded shell commands shellcheck reported SC2006: Use $(...) notation instead of legacy backticked `...`. Fixed by applying shellcheck's diff output as a patch.	2021-05-03 19:28:23 -04:00
Joshua Tauberer	ae3feebd80	Fix warnings reported by shellcheck * SC2068: Double quote array expansions to avoid re-splitting elements. * SC2186: tempfile is deprecated. Use mktemp instead. * SC2124: Assigning an array to a string! Assign as array, or use * instead of @ to concatenate. * SC2102: Ranges can only match single chars (mentioned due to duplicates). * SC2005: Useless echo? Instead of 'echo $(cmd)', just use 'cmd'.	2021-05-03 19:25:09 -04:00
Joshua Tauberer	2c295bcafd	Upgrade the Roundcube persistent login cookie encryption to AES-256-CBC and increase the key length accordingly This change will force everyone to be logged out of Roundcube since the encryption key and cipher won't match anyone's already-set cookie, but this happens anyway after every Mail-in-a-Box update since we generate a new key each time already. Fixes #1968.	2021-04-23 17:04:56 -04:00
Joshua Tauberer	8cda58fb22	Speed up status checks a bit by removing a redundant check if the PRIMARY_HOSTNAME certificate is signed and valid	2021-04-12 19:42:12 -04:00
Joshua Tauberer	178c587654	Migrate to the ECDSAP256SHA256 (13) DNSSEC algorithm * Stop generating RSASHA1-NSEC3-SHA1 keys on new installs since it is no longer recommended, but preserve the key on existing installs so that we continue to sign zones with existing keys to retain the chain of trust with existing DS records. * Start generating ECDSAP256SHA256 keys during setup, the current best practice (in addition to RSASHA256 which is also ok). See https://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml#dns-sec-alg-numbers-1 and https://www.cloudflare.com/dns/dnssec/ecdsa-and-dnssec/. * Sign zones using all available keys rather than choosing just one based on the TLD to enable rotation/migration to the new key and to give the user some options since not every registrar/TLD supports every algorithm. * Allow a user to drop a key from signing specific domains using DOMAINS= in our key configuration file. Signing the zones with extraneous keys may increase the size of DNS responses, which isn't ideal, although I don't know if this is a problem in practice. (Although a user can delete the RSASHA1-NSEC3-SHA1 key file, the other keys will be re-generated on upgrade.) * When generating zonefiles, add a hash of all of the DNSSEC signing keys so that when the keys change the zone is definitely regenerated and re-signed. * In status checks, if DNSSEC is not active (or not valid), offer to use all of the keys that have been generated (for RSASHA1-NSEC3-SHA1 on existing installs, RSASHA256, and now ECDSAP256SHA256) with all digest types, since not all registers support everything, but list them in an order that guides users to the best practice. * In status checks, if the deployed DS record doesn't use a ECDSAP256SHA256 key, prompt the user to update their DS record. * In status checks, if multiple DS records are set, only fail if none are valid. If some use ECDSAP256SHA256 and some don't, remind the user to delete the DS records that don't. * Don't fail if the DS record uses the SHA384 digest (by pre-generating a DS record with that digest type) but don't recommend it because it is not in the IANA mandatory list yet (https://www.iana.org/assignments/ds-rr-types/ds-rr-types.xhtml). See #1953	2021-04-12 19:42:12 -04:00
Joshua Tauberer	34569d24a9	v0.53	2021-04-11 12:45:37 -04:00
Joshua Tauberer	6653dbb2e2	Sort the Custom DNS by zone and qname, and add an option to go back to the old sort order (creation order) Update the zone grouping style on the users and aliases page to match. Fixes #1927	2021-02-28 09:40:32 -05:00
Joshua Tauberer	5fc1162355	Other CHANGELOG entries	2021-02-28 08:22:30 -05:00
Paul	a839602cba	Enable sending DMARC failure reports (#1929 ) Configures opendmarc to send failure reports for domains that request them, including when p=none. The emails are sent as the package default of package name and user@hostname: OpenDMARC Filter <opendmarc@box.example.com> Note I have been running this for several months with a configuration I did not include in the PR to have reports BCC'd to me (FailureReportsBcc postmaster@example.com). Very low load for my personal server of rarely more than a dozen emails sent out per day. I am not familiar with editing scripts, so apologies in advance and please feel free to correct me.	2021-02-28 08:21:15 -05:00
Joshua Tauberer	f21a41dc84	Merge #1932 , with some edits	2021-02-28 08:16:50 -05:00
davDevOps	055ac07663	Update roundcube to 1.4.11 roundcube Bug Fixes: Fix for Cross-Site Scripting (XSS) via HTML messages with malicious CSS content General Improvements from roundcube's Issue Tracker	2021-02-28 08:14:17 -05:00
davDevOps	c7b295f403	Update zpush to 2.6.2	2021-02-28 08:05:40 -05:00
Joshua Tauberer	d36a2cc938	Enable Backblaze B2 backups This reverts commit `b1d703a5e7` and adds python3-setuptools per the first version of #1899 which fixes an installation error for the b2sdk Python package.	2021-02-28 08:04:14 -05:00
jeremitu	82ca54df96	Fixed #1894 log date over year change, START_DATE < END_DATE now. (#1905 ) * Fixed #1894 log date over year change, START_DATE < END_DATE now. * Corrected mail_log.py argument help and message. Co-authored-by: Jarek <jarek@box.jurasz.de>	2021-02-28 07:59:26 -05:00
jvolkenant	af62e7a99b	Fixes unbound variable when upgrading from Nextcloud 13 (#1913 )	2021-02-06 16:49:43 -05:00
Joshua Tauberer	90d63fd208	v0.52	2021-01-31 08:48:14 -05:00
Joshua Tauberer	e81963e585	Remove the instructions for checking that release tags are signed by me since I am not going to do that anymore	2021-01-31 08:47:59 -05:00
Joshua Tauberer	b1d703a5e7	Disable Backblaze B2 backups until #1899 is resolved	2021-01-31 08:33:56 -05:00
Felix Spöttel	e3d98b781e	Warn when connection to Spamhaus times out (#1817 )	2021-01-28 18:22:43 -05:00
jvolkenant	50d50ba653	Update zpush to 2.6.1 (#1908 )	2021-01-28 18:20:19 -05:00
Josh Brown	879467d358	Fix typo in users.html (#1895 ) lettters -> letters fixes #1888	2021-01-05 21:12:01 -05:00
Nicolas North	8025c41ee4	Bump TTL for NS records to 1800 (30 min) to 86400 (1 day) as some registries require this (#1892 ) Co-authored-by: Nicolas North [norðurljósahviða] <nz@tillverka.xyz>	2021-01-03 17:57:54 -05:00
Josh Brown	7a5d729a53	Fix misspelling (#1893 ) Change Blackblaze to Backblaze. Include B2 as the integration name.	2021-01-03 17:54:31 -05:00
jcm-shove-it	e2f9cd845a	Update roundcube to 1.4.10 (#1891 )	2020-12-28 08:11:33 -05:00
Joshua Tauberer	e26cf4512c	Update CHANGELOG	2020-12-25 17:28:34 -05:00
jvolkenant	c7280055a8	Implement SPF/DMARC checks, add spam weight to those mails (#1836 )	2020-12-25 17:22:24 -05:00
Hilko	003e8b7bb1	Adjust max-recursion-queries to fix alternating rdns status (#1876 )	2020-12-25 17:19:16 -05:00
Hilko	3422cc61ce	Include en_US.UTF-8 locale in daemon startup (#1883 ) Fixes #1881.	2020-12-19 19:11:58 -05:00
Hilko	8664afa997	Implement Backblaze for Backup (#1812 ) * Installing b2sdk for b2 support * Added Duplicity PPA so the most recent version is used * Implemented list_target_files for b2 * Implemented b2 in frontend * removed python2 boto package	2020-11-26 07:13:31 -05:00
Joshua Tauberer	82229ce04b	Document how to start the control panel from the command line and in debugging use a stable API key	2020-11-26 07:11:49 -05:00
Richard Willis	f66e609d3f	Api spec cleanup (#1869 ) * Fix indentation * Add parameter definition and remove unused model * Update version * Quote example string	2020-11-26 06:56:04 -05:00
Victor	b85b86e6de	Add download zonefile button to external DNS page (#1853 ) Co-authored-by: Joshua Tauberer <jt@occams.info>	2020-11-16 06:03:41 -05:00
Joshua Tauberer	7fd35bbd11	Disable default Nextcloud apps that we don't support Contacts and calendar are the only supported apps in Mail-in-a-Box. Files can't be disabled. Fixes #1864	2020-11-15 17:17:58 -05:00
gumida	7ce41e3865	Changed mta-sts.txt end of line from LF to CRLF per RFC 8461 (#1863 )	2020-11-15 07:54:34 -05:00
Joshua Tauberer	92221f9efb	v0.51	2020-11-14 10:05:20 -05:00
Joshua Tauberer	0bd3977cde	CHANGELOG updates	2020-10-31 10:36:40 -04:00
Joshua Tauberer	6a979f4f52	Add TOTP two-factor authentication to admin panel login (#1814 ) * add user interface for managing 2fa * update user schema with 2fa columns * implement two factor check during login * Use pyotp for validating TOTP codes * also implements resynchronisation support via `pyotp`'s `valid_window option * Update API route naming, update setup page * Rename /two-factor-auth/ => /2fa/ * Nest totp routes under /2fa/totp/ * Update ids and methods in panel to allow for different setup types * Autofocus otp input when logging in, update layout * Extract TOTPStrategy class to totp.py * this decouples `TOTP` validation and storage logic from `auth` and moves it to `totp` * reduce `pyotp.validate#valid_window` from `2` to `1` * Update OpenApi docs, rename /2fa/ => /mfa/ * Decouple totp from users table by moving to totp_credentials table * this allows implementation of other mfa schemes in the future (webauthn) * also makes key management easier and enforces one totp credentials per user on db-level * Add sqlite migration * Rename internal validate_two_factor_secret => validate_two_factor_secret * conn.close() if mru_token update can't .commit() * Address review feedback, thanks @hija * Use hmac.compare_digest() to compare mru_token * Safeguard against empty mru_token column * hmac.compare_digest() expects arguments of type string, make sure we don't pass None * Currently, this cannot happen but we might not want to store `mru_token` during setup * Do not log failed login attempts for MissingToken errors * Due to the way that the /login UI works, this persists at least one failed login each time a user logs into the admin panel. This in turn triggers fail2ban at some point. * Add TOTP secret to user_key hash thanks @downtownallday * this invalidates all user_keys after TOTP status is changed for user * after changing TOTP state, a login is required * due to the forced login, we can't and don't need to store the code used for setup in `mru_code` * Typo * Reorganize the MFA backend methods * Reorganize MFA front-end and add label column * Fix handling of bad input when enabling mfa * Update openAPI docs * Remove unique key constraint on foreign key user_id in mfa table * Don't expose mru_token and secret for enabled mfas over HTTP * Only update mru_token for matched mfa row * Exclude mru_token in user key hash * Rename tools/mail.py to management/cli.py * Add MFA list/disable to the management CLI so admins can restore access if MFA device is lost Co-authored-by: Joshua Tauberer <jt@occams.info>	2020-10-31 10:27:38 -04:00
Joshua Tauberer	545e7a52e4	Add MFA list/disable to the management CLI so admins can restore access if MFA device is lost	2020-10-31 10:23:43 -04:00
David Duque	48c233ebe5	Update Roundcube to version 1.4.9 (#1830 )	2020-10-31 10:01:14 -04:00
Michael Kroes	9a588de754	Upgrade Nextcloud to version 20.0.1 (#1848 )	2020-10-31 09:58:26 -04:00
Joshua Tauberer	ac9ecc3bd3	Rename tools/mail.py to management/cli.py	2020-10-29 15:41:54 -04:00
David Duque	8b166f3041	Display certificate expiry dates in ISO format (#1841 )	2020-10-16 16:22:36 -04:00
Joshua Tauberer	5509420637	s/Days/Retention Days/ on the backup settings page	2020-10-15 14:11:43 -04:00
Felix Spöttel	7d6c7b6610	Increase mta-sts max_age to one week (#1829 ) This aligns the policy with the example policy found in the spec see https://tools.ietf.org/html/rfc8461#section-3.2	2020-10-02 21:27:21 -04:00
Felix Spöttel	1f0e493b8c	Exclude mru_token in user key hash	2020-09-30 12:34:26 +02:00
Felix Spöttel	ada2167d08	Only update mru_token for matched mfa row	2020-09-29 20:05:58 +02:00
Felix Spöttel	be5032ffbe	Don't expose mru_token and secret for enabled mfas over HTTP	2020-09-29 19:46:02 +02:00
Felix Spöttel	00b3a3b0a9	Remove unique key constraint on foreign key user_id in mfa table	2020-09-29 19:39:40 +02:00
Felix Spöttel	6d82c0035a	Update openAPI docs	2020-09-28 21:27:24 +02:00
Felix Spöttel	4dced10a3f	Fix handling of bad input when enabling mfa	2020-09-28 21:06:59 +02:00
Joshua Tauberer	b80f225691	Reorganize MFA front-end and add label column	2020-09-27 08:31:23 -04:00
0pis	7f0f28f8e3	Use tabs instead of spaces in nginx conf (#1827 ) * conf/nginx-primaryonly.conf: Use tabs instead of spaces * management/web_update.py: Includes the tabs so they display with the correct indentation when added to the local.conf Co-authored-by: 0pis <0pis>	2020-09-27 07:13:33 -04:00
Joshua Tauberer	a8ea456b49	Reorganize the MFA backend methods	2020-09-26 09:58:25 -04:00
Felix Spöttel	7d6427904f	Typo	2020-09-12 16:38:44 +02:00
Felix Spöttel	dcb93d071c	Add TOTP secret to user_key hash thanks @downtownallday * this invalidates all user_keys after TOTP status is changed for user * after changing TOTP state, a login is required * due to the forced login, we can't and don't need to store the code used for setup in `mru_code`	2020-09-12 16:34:06 +02:00
Felix Spöttel	2ea97f0643	Do not log failed login attempts for MissingToken errors * Due to the way that the /login UI works, this persists at least one failed login each time a user logs into the admin panel. This in turn triggers fail2ban at some point.	2020-09-06 13:08:44 +02:00
Felix Spöttel	4791c2fc62	Safeguard against empty mru_token column * hmac.compare_digest() expects arguments of type string, make sure we don't pass None * Currently, this cannot happen but we might not want to store `mru_token` during setup	2020-09-06 13:03:54 +02:00
Felix Spöttel	49c333221a	Use hmac.compare_digest() to compare mru_token	2020-09-06 12:54:45 +02:00
Felix Spöttel	481a333dc0	Address review feedback, thanks @hija	2020-09-04 20:28:15 +02:00
Felix Spöttel	b0df35eba0	conn.close() if mru_token update can't .commit()	2020-09-03 20:39:03 +02:00
Felix Spöttel	08ae3d2b7f	Rename internal validate_two_factor_secret => validate_two_factor_secret	2020-09-03 19:48:54 +02:00
Felix Spöttel	7c4eb0fb70	Add sqlite migration	2020-09-03 19:39:29 +02:00
Felix Spöttel	ee01eae55e	Decouple totp from users table by moving to totp_credentials table * this allows implementation of other mfa schemes in the future (webauthn) * also makes key management easier and enforces one totp credentials per user on db-level	2020-09-03 19:07:21 +02:00
Felix Spöttel	89b301afc7	Update OpenApi docs, rename /2fa/ => /mfa/	2020-09-03 13:54:28 +02:00
Felix Spöttel	ce70f44c58	Extract TOTPStrategy class to totp.py * this decouples `TOTP` validation and storage logic from `auth` and moves it to `totp` * reduce `pyotp.validate#valid_window` from `2` to `1`	2020-09-03 11:19:19 +02:00
Felix Spöttel	6594e19a1f	Autofocus otp input when logging in, update layout	2020-09-02 20:30:08 +02:00
Felix Spöttel	8597646a12	Update API route naming, update setup page * Rename /two-factor-auth/ => /2fa/ * Nest totp routes under /2fa/totp/ * Update ids and methods in panel to allow for different setup types	2020-09-02 19:41:06 +02:00
Felix Spöttel	f205c48564	Use pyotp for validating TOTP codes * also implements resynchronisation support via `pyotp`'s `valid_window option	2020-09-02 19:12:15 +02:00
Felix Spöttel	3c3683429b	implement two factor check during login	2020-09-02 17:23:32 +02:00
Felix Spöttel	a7a66929aa	add user interface for managing 2fa * update user schema with 2fa columns	2020-09-02 16:48:23 +02:00