WebAuthn MFA for the control panel

By not advertising SMTPUTF8 support at the start, senders may opt to transmit recipient internationalized domain names in IDNA form instead, which will be deliverable.

Incoming mail with internationalized domains was probably working prior to our move to Ubuntu 18.04 when postfix's SMTPUTF8 support became enabled by default.

The previous commit is retained because Mail-in-a-Box users might prefer to keep SMTPUTF8 on for outbound mail, if they are not using internationalized domains for email, in which case the previous commit fixes the 'relay access denied' error even if the emails aren't deliverable.

CHANGELOG.md

@@ -1,6 +1,35 @@
 CHANGELOG
 =========
 
+In Development
+--------------
+
+Mail:
+
+* "SMTPUTF8" is now disabled in Postfix. Because Dovecot still does not support SMTPUTF8, incoming mail to internationalized addresses was bouncing. This fixes incoming mail to internationalized domains (which was probably working prior to v0.40), but it will prevent sending outbound mail to addresses with internationalized local-parts.
+* Upgraded to Roundcube 1.5 Release Candidate.
+
+Firewall:
+
+* Fail2ban's IPv6 support is enabled.
+
+Control panel:
+
+* The control panel menus are now hidden before login, but now non-admins can log in to access the mail and contacts/calendar instruction pages.
+* The login form now disables browser autocomplete in the two-factor authentication code field.
+* After logging in, the default page is now a fast-loading welcome page rather than the slow-loading system status checks page.
+* The backup retention period option now displays for B2 backup targets.
+* The DNSSEC DS record recommendations are cleaned up and now recommend changing records that use SHA1.
+* The Munin monitoring pages no longer require a separate HTTP basic authentication login and can be used if two-factor authentication is turned on.
+* Control panel logins are now tied to a session backend that allows true logouts (rather than an encrypted cookie).
+* Failed logins no longer directly reveal whether the email address corresponds to a user account.
+* Browser dark mode now inverts the color scheme.
+
+Other:
+
+* The mail log tool now doesn't crash if there are email addresess in log messages with invalid UTF-8 characters.
+* Additional nsd.conf files can be placed in /etc/nsd.conf.d.
+
 v0.54 (June 20, 2021)
 ---------------------
 

README.md

@@ -42,6 +42,8 @@ It also includes system management tools:
 * A control panel for adding/removing mail users, aliases, custom DNS records, configuring backups, etc.
 * An API for all of the actions on the control panel
 
+Internationalized domain names are supported and configured easily (but SMTPUTF8 is not supported, unfortunately).
+
 It also supports static website hosting since the box is serving HTTPS anyway. (To serve a website for your domains elsewhere, just add a custom DNS "A" record in you Mail-in-a-Box's control panel to point domains to another server.)
 
 For more information on how Mail-in-a-Box handles your privacy, see the [security details page](security.md).

api/mailinabox.yml

@@ -54,24 +54,24 @@ tags:
       System operations, which include system status checks, new version checks
       and reboot status.
 paths:
-  /me:
-    get:
+  /login:
+    post:
       tags:
         - User
-      summary: Get user information
+      summary: Exchange a username and password for a session API key.
       description: |
-        Returns user information. Used for user authentication.
+        Returns user information and a session API key.
 
         Authenticate a user by supplying the auth token as a base64 encoded string in
         format `email:password` using basic authentication headers.
 
         If successful, a long-lived `api_key` is returned which can be used for subsequent
-        requests to the API.
-      operationId: getMe
+        requests to the API in place of the password.
+      operationId: login
       x-codeSamples:
         - lang: curl
           source: |
-            curl -X GET "https://{host}/admin/me" \
+            curl -X GET "https://{host}/admin/login" \
               -u "<email>:<password>"
       responses:
         200:
@@ -92,6 +92,24 @@ paths:
                     privileges:
                       - admin
                     status: ok
+  /logout:
+    post:
+      tags:
+        - User
+      summary: Invalidates a session API key.
+      description: |
+        Invalidates a session API key so that it cannot be used after this API call.
+      operationId: logout
+      x-codeSamples:
+        - lang: curl
+          source: |
+            curl -X GET "https://{host}/admin/logout" \
+              -u "<email>:<session_key>"
+      responses:
+        200:
+          description: Successful operation
+          content:
+            application/json:
   /system/status:
     post:
       tags:
@@ -1722,7 +1740,7 @@ paths:
             text/html:
               schema:
                 type: string
-  /mfa/totp/enable:
+  /mfa/enable/totp:
     post:
       tags:
         - MFA
@@ -1803,7 +1821,7 @@ components:
 
         The `access-token` is comprised of the Base64 encoding of `username:password`.
         The `username` is the mail user's email address, and `password` can either be the mail user's
-        password, or the `api_key` returned from the `getMe` operation.
+        password, or the `api_key` returned from the `login` operation.
 
         When using `curl`, you can supply user credentials using the `-u` or `--user` parameter.
   requestBodies:

management/auth.py

@@ -1,6 +1,7 @@
-import base64, os, os.path, hmac, json
+import base64, os, os.path, hmac, json, secrets
+from datetime import timedelta
 
-from flask import make_response
+from expiringdict import ExpiringDict
 
 import utils
 from mailconfig import get_mail_password, get_mail_user_privileges
@@ -9,25 +10,18 @@ from mfa import get_hash_mfa_state, validate_auth_mfa
 DEFAULT_KEY_PATH   = '/var/lib/mailinabox/api.key'
 DEFAULT_AUTH_REALM = 'Mail-in-a-Box Management Server'
 
-class KeyAuthService:
-	"""Generate an API key for authenticating clients
-
-	Clients must read the key from the key file and send the key with all HTTP
-	requests. The key is passed as the username field in the standard HTTP
-	Basic Auth header.
-	"""
+class AuthService:
 	def __init__(self):
 		self.auth_realm = DEFAULT_AUTH_REALM
-		self.key = self._generate_key()
 		self.key_path = DEFAULT_KEY_PATH
+		self.max_session_duration = timedelta(days=2)
 
-	def write_key(self):
-		"""Write key to file so authorized clients can get the key
+		self.init_system_api_key()
+		self.sessions = ExpiringDict(max_len=64, max_age_seconds=self.max_session_duration.total_seconds())
+
+	def init_system_api_key(self):
+		"""Write an API key to a local file so local processes can use the API"""
 
-		The key file is created with mode 0640 so that additional users can be
-		authorized to access the API by granting group/ACL read permissions on
-		the key file.
-		"""
 		def create_file_with_mode(path, mode):
 			# Based on answer by A-B-B: http://stackoverflow.com/a/15015748
 			old_umask = os.umask(0)
@@ -36,123 +30,137 @@ class KeyAuthService:
 			finally:
 				os.umask(old_umask)
 
+		self.key = secrets.token_hex(32)
+
 		os.makedirs(os.path.dirname(self.key_path), exist_ok=True)
 
 		with create_file_with_mode(self.key_path, 0o640) as key_file:
 			key_file.write(self.key + '\n')
 
-	def authenticate(self, request, env):
-		"""Test if the client key passed in HTTP Authorization header matches the service key
-		or if the or username/password passed in the header matches an administrator user.
+	def authenticate(self, request, env, login_only=False, logout=False):
+		"""Test if the HTTP Authorization header's username matches the system key, a session key,
+		or if the username/password passed in the header matches a local user.
 		Returns a tuple of the user's email address and list of user privileges (e.g.
 		('my@email', []) or ('my@email', ['admin']); raises a ValueError on login failure.
-		If the user used an API key, the user's email is returned as None."""
+		If the user used the system API key, the user's email is returned as None since
+		this key is not associated with a user."""
 
-		def decode(s):
-			return base64.b64decode(s.encode('ascii')).decode('ascii')
-
-		def parse_basic_auth(header):
+		def parse_http_authorization_basic(header):
+			def decode(s):
+				return base64.b64decode(s.encode('ascii')).decode('ascii')
 			if " " not in header:
 				return None, None
 			scheme, credentials = header.split(maxsplit=1)
 			if scheme != 'Basic':
 				return None, None
-
 			credentials = decode(credentials)
 			if ":" not in credentials:
 				return None, None
 			username, password = credentials.split(':', maxsplit=1)
 			return username, password
 
-		header = request.headers.get('Authorization')
-		if not header:
-			raise ValueError("No authorization header provided.")
-
-		username, password = parse_basic_auth(header)
-
+		username, password = parse_http_authorization_basic(request.headers.get('Authorization', ''))
 		if username in (None, ""):
 			raise ValueError("Authorization header invalid.")
-		elif username == self.key:
-			# The user passed the master API key which grants administrative privs.
+
+		if username.strip() == "" and password.strip() == "":
+			raise ValueError("No email address, password, session key, or API key provided.")
+
+		# If user passed the system API key, grant administrative privs. This key
+		# is not associated with a user.
+		if username == self.key and not login_only:
 			return (None, ["admin"])
+
+		# If the password corresponds with a session token for the user, grant access for that user.
+		if self.get_session(username, password, "login", env) and not login_only:
+			sessionid = password
+			session = self.sessions[sessionid]
+			if logout:
+				# Clear the session.
+				del self.sessions[sessionid]
+			else:
+				# Re-up the session so that it does not expire.
+				self.sessions[sessionid] = session
+
+		# If no password was given, but a username was given, we're missing some information.
+		elif password.strip() == "":
+			raise ValueError("Enter a password.")
+
 		else:
-			# The user is trying to log in with a username and either a password
-			# (and possibly a MFA token) or a user-specific API key.
-			return (username, self.check_user_auth(username, password, request, env))
+			# The user is trying to log in with a username and a password
+			# (and possibly a MFA token). On failure, an exception is raised.
+			self.check_user_auth(username, password, request, env)
+
+		# Get privileges for authorization. This call should never fail because by this
+		# point we know the email address is a valid user --- unless the user has been
+		# deleted after the session was granted. On error the call will return a tuple
+		# of an error message and an HTTP status code.
+		privs = get_mail_user_privileges(username, env)
+		if isinstance(privs, tuple): raise ValueError(privs[0])
+
+		# Return the authorization information.
+		return (username, privs)
 
 	def check_user_auth(self, email, pw, request, env):
 		# Validate a user's login email address and password. If MFA is enabled,
 		# check the MFA token in the X-Auth-Token header.
 		#
-		# On success returns a list of privileges (e.g. [] or ['admin']). On login
-		# failure, raises a ValueError with a login error message.
+		# On login failure, raises a ValueError with a login error message. On
+		# success, nothing is returned.
 
-		# Sanity check.
-		if email == "" or pw == "":
-			raise ValueError("Enter an email address and password.")
-
-		# The password might be a user-specific API key. create_user_key raises
-		# a ValueError if the user does not exist.
-		if hmac.compare_digest(self.create_user_key(email, env), pw):
-			# OK.
-			pass
-		else:
+		# Authenticate.
+		try:
 			# Get the hashed password of the user. Raise a ValueError if the
-			# email address does not correspond to a user.
+			# email address does not correspond to a user. But wrap it in the
+			# same exception as if a password fails so we don't easily reveal
+			# if an email address is valid.
 			pw_hash = get_mail_password(email, env)
 
-			# Authenticate.
-			try:
-				# Use 'doveadm pw' to check credentials. doveadm will return
-				# a non-zero exit status if the credentials are no good,
-				# and check_call will raise an exception in that case.
-				utils.shell('check_call', [
-					"/usr/bin/doveadm", "pw",
-					"-p", pw,
-					"-t", pw_hash,
-					])
-			except:
-				# Login failed.
-				raise ValueError("Invalid password.")
+			# Use 'doveadm pw' to check credentials. doveadm will return
+			# a non-zero exit status if the credentials are no good,
+			# and check_call will raise an exception in that case.
+			utils.shell('check_call', [
+				"/usr/bin/doveadm", "pw",
+				"-p", pw,
+				"-t", pw_hash,
+				])
+		except:
+			# Login failed.
+			raise ValueError("Incorrect email address or password.")
 
-			# If MFA is enabled, check that MFA passes.
-			status, hints = validate_auth_mfa(email, request, env)
-			if not status:
-				# Login valid. Hints may have more info.
-				raise ValueError(",".join(hints))
+		# If MFA is enabled, check that MFA passes.
+		status, hints = validate_auth_mfa(email, request, env)
+		if not status:
+			# Login valid. Hints may have more info.
+			raise ValueError(",".join(hints))
 
-		# Get privileges for authorization. This call should never fail because by this
-		# point we know the email address is a valid user. But on error the call will
-		# return a tuple of an error message and an HTTP status code.
-		privs = get_mail_user_privileges(email, env)
-		if isinstance(privs, tuple): raise ValueError(privs[0])
-
-		# Return a list of privileges.
-		return privs
-
-	def create_user_key(self, email, env):
-		# Create a user API key, which is a shared secret that we can re-generate from
-		# static information in our database. The shared secret contains the user's
-		# email address, current hashed password, and current MFA state, so that the
-		# key becomes invalid if any of that information changes.
-		#
-		# Use an HMAC to generate the API key using our master API key as a key,
-		# which also means that the API key becomes invalid when our master API key
-		# changes --- i.e. when this process is restarted.
-		#
-		# Raises ValueError via get_mail_password if the user doesn't exist.
-
-		# Construct the HMAC message from the user's email address and current password.
-		msg = b"AUTH:" + email.encode("utf8") + b" " + get_mail_password(email, env).encode("utf8")
+	def create_user_password_state_token(self, email, env):
+		# Create a token that changes if the user's password or MFA options change
+		# so that sessions become invalid if any of that information changes.
+		msg = get_mail_password(email, env).encode("utf8")
 
 		# Add to the message the current MFA state, which is a list of MFA information.
 		# Turn it into a string stably.
 		msg += b" " + json.dumps(get_hash_mfa_state(email, env), sort_keys=True).encode("utf8")
 
-		# Make the HMAC.
+		# Make a HMAC using the system API key as a hash key.
 		hash_key = self.key.encode('ascii')
 		return hmac.new(hash_key, msg, digestmod="sha256").hexdigest()
 
-	def _generate_key(self):
-		raw_key = os.urandom(32)
-		return base64.b64encode(raw_key).decode('ascii')
+	def create_session_key(self, username, env, type=None):
+		# Create a new session.
+		token = secrets.token_hex(32)
+		self.sessions[token] = {
+			"email": username,
+			"password_token": self.create_user_password_state_token(username, env),
+			"type": type,
+		}
+		return token
+
+	def get_session(self, user_email, session_key, session_type, env):
+		if session_key not in self.sessions: return None
+		session = self.sessions[session_key]
+		if session_type == "login" and session["email"] != user_email: return None
+		if session["type"] != session_type: return None
+		if session["password_token"] != self.create_user_password_state_token(session["email"], env): return None
+		return session

management/daemon.py

@@ -1,5 +1,8 @@
 #!/usr/local/lib/mailinabox/env/bin/python3
 #
+# The API can be accessed on the command line, e.g. use `curl` like so:
+#    curl --user $(</var/lib/mailinabox/api.key): http://localhost:10222/mail/users
+#
 # During development, you can start the Mail-in-a-Box control panel
 # by running this script, e.g.:
 #
@@ -18,11 +21,11 @@ import auth, utils
 from mailconfig import get_mail_users, get_mail_users_ex, get_admins, add_mail_user, set_mail_password, remove_mail_user
 from mailconfig import get_mail_user_privileges, add_remove_mail_user_privilege
 from mailconfig import get_mail_aliases, get_mail_aliases_ex, get_mail_domains, add_mail_alias, remove_mail_alias
-from mfa import get_public_mfa_state, provision_totp, validate_totp_secret, enable_mfa, disable_mfa
+from mfa import get_public_mfa_state, provision_totp, provision_webauthn, validate_totp_secret, enable_mfa, disable_mfa
 
 env = utils.load_environment()
 
-auth_service = auth.KeyAuthService()
+auth_service = auth.AuthService()
 
 # We may deploy via a symbolic link, which confuses flask's template finding.
 me = __file__
@@ -53,8 +56,10 @@ def authorized_personnel_only(viewfunc):
 		try:
 			email, privs = auth_service.authenticate(request, env)
 		except ValueError as e:
-			# Write a line in the log recording the failed login
-			log_failed_login(request)
+			# Write a line in the log recording the failed login, unless no authorization header
+			# was given which can happen on an initial request before a 403 response.
+			if "Authorization" in request.headers:
+				log_failed_login(request)
 
 			# Authentication failed.
 			error = str(e)
@@ -131,11 +136,12 @@ def index():
 		csr_country_codes=csr_country_codes,
 	)
 
-@app.route('/me')
-def me():
+# Create a session key by checking the username/password in the Authorization header.
+@app.route('/login', methods=["POST"])
+def login():
 	# Is the caller authorized?
 	try:
-		email, privs = auth_service.authenticate(request, env)
+		email, privs = auth_service.authenticate(request, env, login_only=True)
 	except ValueError as e:
 		if "missing-totp-token" in str(e):
 			return json_response({
@@ -150,19 +156,29 @@ def me():
 				"reason": str(e),
 			})
 
+	# Return a new session for the user.
 	resp = {
 		"status": "ok",
 		"email": email,
 		"privileges": privs,
+		"api_key": auth_service.create_session_key(email, env, type='login'),
 	}
 
-	# Is authorized as admin? Return an API key for future use.
-	if "admin" in privs:
-		resp["api_key"] = auth_service.create_user_key(email, env)
+	app.logger.info("New login session created for {}".format(email))
 
 	# Return.
 	return json_response(resp)
 
+@app.route('/logout', methods=["POST"])
+def logout():
+	try:
+		email, _ = auth_service.authenticate(request, env, logout=True)
+		app.logger.info("{} logged out".format(email))
+	except ValueError as e:
+		pass
+	finally:
+		return json_response({ "status": "ok" })
+
 # MAIL
 
 @app.route('/mail/users')
@@ -219,7 +235,7 @@ def mail_aliases():
 	if request.args.get("format", "") == "json":
 		return json_response(get_mail_aliases_ex(env))
 	else:
-		return "".join(address+"\t"+receivers+"\t"+(senders or "")+"\n" for address, receivers, senders in get_mail_aliases(env))
+		return "".join(address+"\t"+receivers+"\t"+(senders or "")+"\n" for address, receivers, senders, auto in get_mail_aliases(env))
 
 @app.route('/mail/aliases/add', methods=['POST'])
 @authorized_personnel_only
@@ -452,7 +468,7 @@ def ssl_provision_certs():
 def mfa_get_status():
 	# Anyone accessing this route is an admin, and we permit them to
 	# see the MFA status for any user if they submit a 'user' form
-	# field. But we don't include provisioning info since a user can
+	# field. But we don't always include provisioning info since a user can
 	# only provision for themselves.
 	email = request.form.get('user', request.user_email) # user field if given, otherwise the user making the request
 	try:
@@ -462,14 +478,15 @@ def mfa_get_status():
 		if email == request.user_email:
 			resp.update({
 				"new_mfa": {
-					"totp": provision_totp(email, env)
+					"totp": provision_totp(email, env),
+					"webauthn": provision_webauthn(email, env)
 				}
 			})
 	except ValueError as e:
 		return (str(e), 400)
 	return json_response(resp)
 
-@app.route('/mfa/totp/enable', methods=['POST'])
+@app.route('/mfa/enable/totp', methods=['POST'])
 @authorized_personnel_only
 def totp_post_enable():
 	secret = request.form.get('secret')
@@ -479,7 +496,20 @@ def totp_post_enable():
 		return ("Bad Input", 400)
 	try:
 		validate_totp_secret(secret)
-		enable_mfa(request.user_email, "totp", secret, token, label, env)
+		enable_mfa(request.user_email, "totp", env, secret, token, label)
+	except ValueError as e:
+		return (str(e), 400)
+	return "OK"
+
+@app.route('/mfa/enable/webauthn', methods=['POST'])
+@authorized_personnel_only
+def webauthn_post_enable():
+	attestationObject = request.form.get('attestationObject')
+	clientDataJSON = request.form.get('clientDataJSON')
+	if type(attestationObject) != str or type(clientDataJSON) != str:
+		return ("Bad Input", 400)
+	try:
+		enable_mfa(request.user_email, "webauthn", env, attestationObject, clientDataJSON)
 	except ValueError as e:
 		return (str(e), 400)
 	return "OK"
@@ -638,16 +668,42 @@ def privacy_status_set():
 # MUNIN
 
 @app.route('/munin/')
-@app.route('/munin/<path:filename>')
 @authorized_personnel_only
-def munin(filename=""):
-	# Checks administrative access (@authorized_personnel_only) and then just proxies
-	# the request to static files.
+def munin_start():
+	# Munin pages, static images, and dynamically generated images are served
+	# outside of the AJAX API. We'll start with a 'start' API that sets a cookie
+	# that subsequent requests will read for authorization. (We don't use cookies
+	# for the API to avoid CSRF vulnerabilities.)
+	response = make_response("OK")
+	response.set_cookie("session", auth_service.create_session_key(request.user_email, env, type='cookie'),
+	    max_age=60*30, secure=True, httponly=True, samesite="Strict") # 30 minute duration
+	return response
+
+def check_request_cookie_for_admin_access():
+	session = auth_service.get_session(None, request.cookies.get("session", ""), "cookie", env)
+	if not session: return False
+	privs = get_mail_user_privileges(session["email"], env)
+	if not isinstance(privs, list): return False
+	if "admin" not in privs: return False
+	return True
+
+def authorized_personnel_only_via_cookie(f):
+	@wraps(f)
+	def g(*args, **kwargs):
+		if not check_request_cookie_for_admin_access():
+			return Response("Unauthorized", status=403, mimetype='text/plain', headers={})
+		return f(*args, **kwargs)
+	return g
+
+@app.route('/munin/<path:filename>')
+@authorized_personnel_only_via_cookie
+def munin_static_file(filename=""):
+	# Proxy the request to static files.
 	if filename == "": filename = "index.html"
 	return send_from_directory("/var/cache/munin/www", filename)
 
 @app.route('/munin/cgi-graph/<path:filename>')
-@authorized_personnel_only
+@authorized_personnel_only_via_cookie
 def munin_cgi(filename):
 	""" Relay munin cgi dynazoom requests
 	/usr/lib/munin/cgi/munin-cgi-graph is a perl cgi script in the munin package
@@ -724,30 +780,10 @@ if __name__ == '__main__':
 		# Turn on Flask debugging.
 		app.debug = True
 
-		# Use a stable-ish master API key so that login sessions don't restart on each run.
-		# Use /etc/machine-id to seed the key with a stable secret, but add something
-		# and hash it to prevent possibly exposing the machine id, using the time so that
-		# the key is not valid indefinitely.
-		import hashlib
-		with open("/etc/machine-id") as f:
-			api_key = f.read()
-		api_key += "|" + str(int(time.time() / (60*60*2)))
-		hasher = hashlib.sha1()
-		hasher.update(api_key.encode("ascii"))
-		auth_service.key = hasher.hexdigest()
-
-	if "APIKEY" in os.environ: auth_service.key = os.environ["APIKEY"]
-
 	if not app.debug:
 		app.logger.addHandler(utils.create_syslog_handler())
 
-	# For testing on the command line, you can use `curl` like so:
-	#    curl --user $(</var/lib/mailinabox/api.key): http://localhost:10222/mail/users
-	auth_service.write_key()
-
-	# For testing in the browser, you can copy the API key that's output to the
-	# debug console and enter that as the username
-	app.logger.info('API key: ' + auth_service.key)
+	#app.logger.info('API key: ' + auth_service.key)
 
 	# Start the application server. Listens on 127.0.0.1 (IPv4 only).
 	app.run(port=10222)

management/dns_update.py

@@ -604,7 +604,7 @@ def get_dns_zonefile(zone, env):
 
 def write_nsd_conf(zonefiles, additional_records, env):
 	# Write the list of zones to a configuration file.
-	nsd_conf_file = "/etc/nsd/zones.conf"
+	nsd_conf_file = "/etc/nsd/nsd.conf.d/zones.conf"
 	nsdconf = ""
 
 	# Append the zones.

management/mail_log.py

@@ -586,7 +586,7 @@ def scan_postfix_submission_line(date, log, collector):
 def readline(filename):
     """ A generator that returns the lines of a file
     """
-    with open(filename) as file:
+    with open(filename, errors='replace') as file:
         while True:
           line = file.readline()
           if not line:

management/mailconfig.py

@@ -16,8 +16,8 @@ import idna
 
 def validate_email(email, mode=None):
 	# Checks that an email address is syntactically valid. Returns True/False.
-	# Until Postfix supports SMTPUTF8, an email address may contain ASCII
-	# characters only; IDNs must be IDNA-encoded.
+	# An email address may contain ASCII characters only because Dovecot's
+	# authentication mechanism gets confused with other character encodings.
 	#
 	# When mode=="user", we're checking that this can be a user account name.
 	# Dovecot has tighter restrictions - letters, numbers, underscore, and
@@ -186,9 +186,9 @@ def get_admins(env):
 	return users
 
 def get_mail_aliases(env):
-	# Returns a sorted list of tuples of (address, forward-tos, permitted-senders).
+	# Returns a sorted list of tuples of (address, forward-tos, permitted-senders, auto).
 	c = open_database(env)
-	c.execute('SELECT source, destination, permitted_senders FROM aliases')
+	c.execute('SELECT source, destination, permitted_senders, 0 as auto FROM aliases UNION SELECT source, destination, permitted_senders, 1 as auto FROM auto_aliases')
 	aliases = { row[0]: row for row in c.fetchall() } # make dict
 
 	# put in a canonical order: sort by domain, then by email address lexicographically
@@ -208,7 +208,7 @@ def get_mail_aliases_ex(env):
 	#         address_display: "name@domain.tld", # full Unicode
 	#         forwards_to: ["user1@domain.com", "receiver-only1@domain.com", ...],
 	#         permitted_senders: ["user1@domain.com", "sender-only1@domain.com", ...] OR null,
-	#         required: True|False
+	#         auto: True|False
 	#       },
 	#       ...
 	#     ]
@@ -216,12 +216,13 @@ def get_mail_aliases_ex(env):
 	#   ...
 	# ]
 
-	required_aliases = get_required_aliases(env)
 	domains = {}
-	for address, forwards_to, permitted_senders in get_mail_aliases(env):
+	for address, forwards_to, permitted_senders, auto in get_mail_aliases(env):
+		# skip auto domain maps since these are not informative in the control panel's aliases list
+		if auto and address.startswith("@"): continue
+
 		# get alias info
 		domain = get_domain(address)
-		required = (address in required_aliases)
 
 		# add to list
 		if not domain in domains:
@@ -234,7 +235,7 @@ def get_mail_aliases_ex(env):
 			"address_display": prettify_idn_email_address(address),
 			"forwards_to": [prettify_idn_email_address(r.strip()) for r in forwards_to.split(",")],
 			"permitted_senders": [prettify_idn_email_address(s.strip()) for s in permitted_senders.split(",")] if permitted_senders is not None else None,
-			"required": required,
+			"auto": bool(auto),
 		})
 
 	# Sort domains.
@@ -242,7 +243,7 @@ def get_mail_aliases_ex(env):
 
 	# Sort aliases within each domain first by required-ness then lexicographically by address.
 	for domain in domains:
-		domain["aliases"].sort(key = lambda alias : (alias["required"], alias["address"]))
+		domain["aliases"].sort(key = lambda alias : (alias["auto"], alias["address"]))
 	return domains
 
 def get_domain(emailaddr, as_unicode=True):
@@ -261,11 +262,12 @@ def get_domain(emailaddr, as_unicode=True):
 def get_mail_domains(env, filter_aliases=lambda alias : True, users_only=False):
 	# Returns the domain names (IDNA-encoded) of all of the email addresses
 	# configured on the system. If users_only is True, only return domains
-	# with email addresses that correspond to user accounts.
+	# with email addresses that correspond to user accounts. Exclude Unicode
+	# forms of domain names listed in the automatic aliases table.
 	domains = []
 	domains.extend([get_domain(login, as_unicode=False) for login in get_mail_users(env)])
 	if not users_only:
-		domains.extend([get_domain(address, as_unicode=False) for address, *_ in get_mail_aliases(env) if filter_aliases(address) ])
+		domains.extend([get_domain(address, as_unicode=False) for address, _, _, auto in get_mail_aliases(env) if filter_aliases(address) and not auto ])
 	return set(domains)
 
 def add_mail_user(email, pw, privs, env):
@@ -512,6 +514,13 @@ def remove_mail_alias(address, env, do_kick=True):
 		# Update things in case any domains are removed.
 		return kick(env, "alias removed")
 
+def add_auto_aliases(aliases, env):
+	conn, c = open_database(env, with_connection=True)
+	c.execute("DELETE FROM auto_aliases");
+	for source, destination in aliases.items():
+		c.execute("INSERT INTO auto_aliases (source, destination) VALUES (?, ?)", (source, destination))
+	conn.commit()
+
 def get_system_administrator(env):
 	return "administrator@" + env['PRIMARY_HOSTNAME']
 
@@ -555,39 +564,34 @@ def kick(env, mail_result=None):
 	if mail_result is not None:
 		results.append(mail_result + "\n")
 
-	# Ensure every required alias exists.
+	auto_aliases = { }
 
-	existing_users = get_mail_users(env)
-	existing_alias_records = get_mail_aliases(env)
-	existing_aliases = set(a for a, *_ in existing_alias_records) # just first entry in tuple
+	# Mape required aliases to the administrator alias (which should be created manually).
+	administrator = get_system_administrator(env)
 	required_aliases = get_required_aliases(env)
+	for alias in required_aliases:
+		if alias == administrator: continue # don't make an alias from the administrator to itself --- this alias must be created manually
+		auto_aliases[alias] = administrator
 
-	def ensure_admin_alias_exists(address):
-		# If a user account exists with that address, we're good.
-		if address in existing_users:
-			return
+	# Add domain maps from Unicode forms of IDNA domains to the ASCII forms stored in the alias table.
+	for domain in get_mail_domains(env):
+		try:
+			domain_unicode = idna.decode(domain.encode("ascii"))
+			if domain == domain_unicode: continue # not an IDNA/Unicode domain
+			auto_aliases["@" + domain_unicode] = "@" + domain
+		except (ValueError, UnicodeError, idna.IDNAError):
+			continue
 
-		# If the alias already exists, we're good.
-		if address in existing_aliases:
-			return
+	add_auto_aliases(auto_aliases, env)
 
-		# Doesn't exist.
-		administrator = get_system_administrator(env)
-		if address == administrator: return # don't make an alias from the administrator to itself --- this alias must be created manually
-		add_mail_alias(address, administrator, "", env, do_kick=False)
-		if administrator not in existing_aliases: return # don't report the alias in output if the administrator alias isn't in yet -- this is a hack to supress confusing output on initial setup
-		results.append("added alias %s (=> %s)\n" % (address, administrator))
-
-	for address in required_aliases:
-		ensure_admin_alias_exists(address)
-
-	# Remove auto-generated postmaster/admin on domains we no
-	# longer have any other email addresses for.
-	for address, forwards_to, *_ in existing_alias_records:
+	# Remove auto-generated postmaster/admin/abuse alises from the main aliases table.
+	# They are now stored in the auto_aliases table.
+	for address, forwards_to, permitted_senders, auto in get_mail_aliases(env):
 		user, domain = address.split("@")
 		if user in ("postmaster", "admin", "abuse") \
 			and address not in required_aliases \
-			and forwards_to == get_system_administrator(env):
+			and forwards_to == get_system_administrator(env) \
+			and not auto:
 			remove_mail_alias(address, env, do_kick=False)
 			results.append("removed alias %s (was to %s; domain no longer used for email)\n" % (address, forwards_to))
 

management/mfa.py

@@ -1,9 +1,12 @@
 import base64
 import hmac
 import io
+import json
 import os
 import pyotp
 import qrcode
+import pywarp
+import pywarp.backends
 
 from mailconfig import open_database
 
@@ -29,25 +32,44 @@ def get_public_mfa_state(email, env):
 	]
 
 def get_hash_mfa_state(email, env):
-	mfa_state = get_mfa_state(email, env)
-	return [
-		{ "id": s["id"], "type": s["type"], "secret": s["secret"] }
-		for s in mfa_state
-	]
+	# Get the current MFA credential secrets from which we form a hash
+	# so that we can reset user logins when any authentication information
+	# changes.
+	mfa_state = []
+	for s in get_mfa_state(email, env):
+		# Add TOTP id and secret to the state.
+		# Skip WebAuthn state if it's just a challenge.
+		if s["type"] == "webauthn":
+			try:
+				# Get the credential and only include it (not challenges) in the state.
+				s["secret"] = json.loads(s["secret"])["cred_pub_key"]
+			except:
+				# Skip this one --- there is no cred_pub_key.
+				continue
+		mfa_state.append({ "id": s["id"], "type": s["type"], "secret": s["secret"] })
+	return mfa_state
 
-def enable_mfa(email, type, secret, token, label, env):
+def enable_mfa(email, type, env, *args):
 	if type == "totp":
+		secret, token, label = args
 		validate_totp_secret(secret)
 		# Sanity check with the provide current token.
 		totp = pyotp.TOTP(secret)
 		if not totp.verify(token, valid_window=1):
 			raise ValueError("Invalid token.")
+		conn, c = open_database(env, with_connection=True)
+		c.execute('INSERT INTO mfa (user_id, type, secret, label) VALUES (?, ?, ?, ?)', (get_user_id(email, c), type, secret, label))
+		conn.commit()
+	elif type == "webauthn":
+		attestationObject, clientDataJSON = args
+		rp = pywarp.RelyingPartyManager(
+			get_relying_party_name(env),
+			rp_id=env["PRIMARY_HOSTNAME"], # must match hostname the control panel is served from
+			credential_storage_backend=WebauthnStorageBackend(env))
+		rp.register(attestation_object=base64.b64decode(attestationObject), client_data_json=base64.b64decode(clientDataJSON), email=email.encode("utf8")) # encoding of email is a little funky here, pywarp calls .decode() with no args?
 	else:
 		raise ValueError("Invalid MFA type.")
 
-	conn, c = open_database(env, with_connection=True)
-	c.execute('INSERT INTO mfa (user_id, type, secret, label) VALUES (?, ?, ?, ?)', (get_user_id(email, c), type, secret, label))
-	conn.commit()
 
 def set_mru_token(email, mfa_id, token, env):
 	conn, c = open_database(env, with_connection=True)
@@ -71,6 +93,9 @@ def validate_totp_secret(secret):
 	if len(secret) != 32:
 		raise ValueError("Secret should be a 32 characters base32 string")
 
+def get_relying_party_name(env):
+	return env["PRIMARY_HOSTNAME"] + " Mail-in-a-Box Control Panel"
+
 def provision_totp(email, env):
 	# Make a new secret.
 	secret = base64.b32encode(os.urandom(20)).decode('utf-8')
@@ -79,7 +104,7 @@ def provision_totp(email, env):
 	# Make a URI that we encode within a QR code.
 	uri = pyotp.TOTP(secret).provisioning_uri(
 		name=email,
-		issuer_name=env["PRIMARY_HOSTNAME"] + " Mail-in-a-Box Control Panel"
+		issuer_name=get_relying_party_name(env)
 	)
 
 	# Generate a QR code as a base64-encode PNG image.
@@ -94,6 +119,55 @@ def provision_totp(email, env):
 		"qr_code_base64": png_b64
 	}
 
+class WebauthnStorageBackend(pywarp.backends.CredentialStorageBackend):
+	def __init__(self, env):
+		self.env = env
+	def get_record(self, email, conn=None, c=None):
+		# Get an existing record and parse the 'secret' column as JSON.
+		if conn is None: conn, c = open_database(self.env, with_connection=True)
+		c.execute('SELECT secret FROM mfa WHERE user_id=? AND type="webauthn"', (get_user_id(email, c),))
+		config = c.fetchone()
+		if config:
+			try:
+				return json.loads(config[0])
+			except:
+				pass
+		return { }
+	def update_record(self, email, fields):
+		# Update the webauthn record in the database for this user by
+		# merging the fields with the existing fields in the database.
+		conn, c = open_database(self.env, with_connection=True)
+		config = self.get_record(email, conn=conn, c=c)
+		if config:
+			# Merge and update.
+			config.update(fields)
+			config = json.dumps(config)
+			c.execute('UPDATE mfa SET secret=? WHERE user_id=? AND type="webauthn"', (config, get_user_id(email, c),))
+			conn.commit()
+			return
+
+		# Either there's no existing webauthn record or it's corrupted. Delete any existing record.
+		# Then add a new record.
+		c.execute('DELETE FROM mfa WHERE user_id=? AND type="webauthn"', (get_user_id(email, c),))
+		c.execute('INSERT INTO mfa (user_id, type, secret, label) VALUES (?, ?, ?, ?)', (
+			get_user_id(email, c), "webauthn",
+			json.dumps(fields),
+			"WebAuthn"))
+		conn.commit()
+	def save_challenge_for_user(self, email, challenge, type):
+		self.update_record(email, { type + "challenge": base64.b64encode(challenge).decode("ascii") })
+	def get_challenge_for_user(self, email, type):
+		challenge = self.get_record(email).get(type + "challenge")
+		if challenge: challenge = base64.b64decode(challenge.encode("ascii"))
+		return challenge
+
+def provision_webauthn(email, env):
+	rp = pywarp.RelyingPartyManager(
+		get_relying_party_name(env),
+		rp_id=env["PRIMARY_HOSTNAME"], # must match hostname the control panel is served from
+		credential_storage_backend=WebauthnStorageBackend(env))
+	return rp.get_registration_options(email=email)
+
 def validate_auth_mfa(email, request, env):
 	# Validates that a login request satisfies any MFA modes
 	# that have been enabled for the user's account. Returns

management/status_checks.py

@@ -612,14 +612,16 @@ def check_dnssec(domain, env, output, dns_zonefiles, is_checking_primary=False):
 			#
 			# But it may not be preferred. Only algorithm 13 is preferred. Warn if any of the
 			# matched zones uses a different algorithm.
-			if set(r[1] for r in matched_ds) == { '13' }: # all are alg 13
+			if set(r[1] for r in matched_ds) == { '13' } and set(r[2] for r in matched_ds) <= { '2', '4' }: # all are alg 13 and digest type 2 or 4
 				output.print_ok("DNSSEC 'DS' record is set correctly at registrar.")
 				return
-			elif '13' in set(r[1] for r in matched_ds): # some but not all are alg 13
-				output.print_ok("DNSSEC 'DS' record is set correctly at registrar. (Records using algorithm other than ECDSAP256SHA256 should be removed.)")
+			elif len([r for r in matched_ds if r[1] == '13' and r[2] in ( '2', '4' )]) > 0: # some but not all are alg 13
+				output.print_ok("DNSSEC 'DS' record is set correctly at registrar. (Records using algorithm other than ECDSAP256SHA256 and digest types other than SHA-256/384 should be removed.)")
 				return
 			else: # no record uses alg 13
-				output.print_warning("DNSSEC 'DS' record set at registrar is valid but should be updated to ECDSAP256SHA256 (see below).")
+				output.print_warning("""DNSSEC 'DS' record set at registrar is valid but should be updated to ECDSAP256SHA256 and SHA-256 (see below).
+				IMPORTANT: Do not delete existing DNSSEC 'DS' records for this domain until confirmation that the new DNSSEC 'DS' record
+				for this domain is valid.""")
 		else:
 			if is_checking_primary:
 				output.print_error("""The DNSSEC 'DS' record for %s is incorrect. See further details below.""" % domain)
@@ -630,7 +632,8 @@ def check_dnssec(domain, env, output, dns_zonefiles, is_checking_primary=False):
 
 	output.print_line("""Follow the instructions provided by your domain name registrar to set a DS record.
 		Registrars support different sorts of DS records. Use the first option that works:""")
-	preferred_ds_order = [(7, 1), (7, 2), (8, 4), (13, 4), (8, 1), (8, 2), (13, 1), (13, 2)] # low to high
+	preferred_ds_order = [(7, 2), (8, 4), (13, 4), (8, 2), (13, 2)] # low to high, see https://github.com/mail-in-a-box/mailinabox/issues/1998
+
 	def preferred_ds_order_func(ds_suggestion):
 		k = (int(ds_suggestion['alg']), int(ds_suggestion['digalg']))
 		if k in preferred_ds_order:
@@ -638,11 +641,12 @@ def check_dnssec(domain, env, output, dns_zonefiles, is_checking_primary=False):
 		return -1 # index before first item
 	output.print_line("")
 	for i, ds_suggestion in enumerate(sorted(expected_ds_records.values(), key=preferred_ds_order_func, reverse=True)):
+		if preferred_ds_order_func(ds_suggestion) == -1: continue # don't offer record types that the RFC says we must not offer
 		output.print_line("")
 		output.print_line("Option " + str(i+1) + ":")
 		output.print_line("----------")
 		output.print_line("Key Tag: " + ds_suggestion['keytag'])
-		output.print_line("Key Flags: KSK")
+		output.print_line("Key Flags: KSK (256)")
 		output.print_line("Algorithm: %s / %s" % (ds_suggestion['alg'], ds_suggestion['alg_name']))
 		output.print_line("Digest Type: %s / %s" % (ds_suggestion['digalg'], ds_suggestion['digalg_name']))
 		output.print_line("Digest: " + ds_suggestion['digest'])

management/templates/aliases.html

@@ -1,6 +1,6 @@
 <style>
 #alias_table .actions > * { padding-right: 3px; }
-#alias_table .alias-required .remove { display: none }
+#alias_table .alias-auto .actions > * { display: none }
 </style>
 
 <h2>Aliases</h2>
@@ -163,7 +163,7 @@ function show_aliases() {
           var n = $("#alias-template").clone();
           n.attr('id', '');
 
-          if (alias.required) n.addClass('alias-required');
+          if (alias.auto) n.addClass('alias-auto');
           n.attr('data-address', alias.address_display); // this is decoded from IDNA, but will get re-coded to IDNA on the backend
           n.find('td.address').text(alias.address_display)
           for (var j = 0; j < alias.forwards_to.length; j++)

management/templates/index.html

@@ -62,6 +62,37 @@
 	    ol li {
 	       margin-bottom: 1em;
 	    }
+          
+            .if-logged-in { display: none; }
+            .if-logged-in-admin { display: none; }
+
+      /* The below only gets used if it is supported */
+      @media (prefers-color-scheme: dark) {
+        /* Invert invert lightness but not hue */
+        html {
+          filter: invert(100%) hue-rotate(180deg);
+        }
+
+        /* Set explicit background color (necessary for Firefox) */
+        html {
+          background-color: #111;
+        }
+          
+        /* Override Boostrap theme here to give more contrast. The black turns to white by the filter. */
+        .form-control {
+          color: black !important;
+        }          
+
+        /* Revert the invert for the navbar */
+        button, div.navbar {
+          filter: invert(100%) hue-rotate(180deg);
+        }
+          
+        /* Revert the revert for the dropdowns */
+        ul.dropdown-menu {
+          filter: invert(100%) hue-rotate(180deg);
+        }
+      }
         </style>
         <link rel="stylesheet" href="/admin/assets/bootstrap/css/bootstrap-theme.min.css">
     </head>
@@ -83,7 +114,7 @@
         </div>
         <div class="navbar-collapse collapse">
           <ul class="nav navbar-nav">
-            <li class="dropdown">
+            <li class="dropdown if-logged-in-admin">
               <a href="#" class="dropdown-toggle" data-toggle="dropdown">System <b class="caret"></b></a>
               <ul class="dropdown-menu">
                 <li><a href="#system_status" onclick="return show_panel(this);">Status Checks</a></li>
@@ -93,10 +124,11 @@
                 <li class="dropdown-header">Advanced Pages</li>
                 <li><a href="#custom_dns" onclick="return show_panel(this);">Custom DNS</a></li>
                 <li><a href="#external_dns" onclick="return show_panel(this);">External DNS</a></li>
-                <li><a href="/admin/munin" target="_blank">Munin Monitoring</a></li>
+                <li><a href="#munin" onclick="return show_panel(this);">Munin Monitoring</a></li>
               </ul>
             </li>
-            <li class="dropdown">
+            <li><a href="#mail-guide" onclick="return show_panel(this);" class="if-logged-in-not-admin">Mail</a></li>
+            <li class="dropdown if-logged-in-admin">
               <a href="#" class="dropdown-toggle" data-toggle="dropdown">Mail &amp; Users <b class="caret"></b></a>
               <ul class="dropdown-menu">
                 <li><a href="#mail-guide" onclick="return show_panel(this);">Instructions</a></li>
@@ -107,17 +139,21 @@
                 <li><a href="#mfa" onclick="return show_panel(this);">Two-Factor Authentication</a></li>
               </ul>
             </li>
-            <li><a href="#sync_guide" onclick="return show_panel(this);">Contacts/Calendar</a></li>
-            <li><a href="#web" onclick="return show_panel(this);">Web</a></li>
+            <li><a href="#sync_guide" onclick="return show_panel(this);" class="if-logged-in">Contacts/Calendar</a></li>
+            <li><a href="#web" onclick="return show_panel(this);" class="if-logged-in-admin">Web</a></li>
           </ul>
           <ul class="nav navbar-nav navbar-right">
-            <li><a href="#" onclick="do_logout(); return false;" style="color: white">Log out</a></li>
+            <li class="if-logged-in"><a href="#" onclick="do_logout(); return false;" style="color: white">Log out</a></li>
           </ul>
         </div><!--/.navbar-collapse -->
       </div>
     </div>
 
     <div class="container">
+      <div id="panel_welcome" class="admin_panel">
+      {% include "welcome.html" %}
+      </div>
+
       <div id="panel_system_status" class="admin_panel">
       {% include "system-status.html" %}
       </div>
@@ -166,6 +202,10 @@
       {% include "ssl.html" %}
       </div>
 
+      <div id="panel_munin" class="admin_panel">
+      {% include "munin.html" %}
+      </div>
+
       <hr>
 
       <footer>
@@ -298,7 +338,7 @@ function ajax_with_indicator(options) {
   return false; // handy when called from onclick
 }
 
-var api_credentials = ["", ""];
+var api_credentials = null;
 function api(url, method, data, callback, callback_error, headers) {
   // from http://www.webtoolkit.info/javascript-base64.html
   function base64encode(input) {
@@ -346,9 +386,10 @@ function api(url, method, data, callback, callback_error, headers) {
       // We don't store user credentials in a cookie to avoid the hassle of CSRF
       // attacks. The Authorization header only gets set in our AJAX calls triggered
       // by user actions.
-      xhr.setRequestHeader(
-        'Authorization',
-        'Basic ' + base64encode(api_credentials[0] + ':' + api_credentials[1]));
+      if (api_credentials)
+        xhr.setRequestHeader(
+          'Authorization',
+          'Basic ' + base64encode(api_credentials.username + ':' + api_credentials.session_key));
     },
     success: callback,
     error: callback_error || default_error,
@@ -367,12 +408,21 @@ var current_panel = null;
 var switch_back_to_panel = null;
 
 function do_logout() {
-  api_credentials = ["", ""];
+  // Clear the session from the backend.
+  api("/logout", "POST");
+
+  // Forget the token.
+  api_credentials = null;
   if (typeof localStorage != 'undefined')
     localStorage.removeItem("miab-cp-credentials");
   if (typeof sessionStorage != 'undefined')
     sessionStorage.removeItem("miab-cp-credentials");
+
+  // Return to the start.
   show_panel('login');
+
+  // Reset menus.
+  show_hide_menus();
 }
 
 function show_panel(panelid) {
@@ -395,14 +445,22 @@ function show_panel(panelid) {
 
 $(function() {
   // Recall saved user credentials.
-  if (typeof sessionStorage != 'undefined' && sessionStorage.getItem("miab-cp-credentials"))
-    api_credentials = sessionStorage.getItem("miab-cp-credentials").split(":");
-  else if (typeof localStorage != 'undefined' && localStorage.getItem("miab-cp-credentials"))
-    api_credentials = localStorage.getItem("miab-cp-credentials").split(":");
+  try {
+    if (typeof sessionStorage != 'undefined' && sessionStorage.getItem("miab-cp-credentials"))
+      api_credentials = JSON.parse(sessionStorage.getItem("miab-cp-credentials"));
+    else if (typeof localStorage != 'undefined' && localStorage.getItem("miab-cp-credentials"))
+      api_credentials = JSON.parse(localStorage.getItem("miab-cp-credentials"));
+  } catch (_) {
+  }
+
+  // Toggle menu state.
+  show_hide_menus();
 
   // Recall what the user was last looking at.
-  if (typeof localStorage != 'undefined' && localStorage.getItem("miab-cp-lastpanel")) {
+  if (api_credentials != null && typeof localStorage != 'undefined' && localStorage.getItem("miab-cp-lastpanel")) {
     show_panel(localStorage.getItem("miab-cp-lastpanel"));
+  } else if (api_credentials != null) {
+    show_panel('welcome');
   } else {
     show_panel('login');
   }

management/templates/login.html

@@ -64,7 +64,7 @@ sudo management/cli.py user make-admin me@{{hostname}}</pre>
     <div class="form-group" id="loginOtp">
       <label for="loginOtpInput" class="col-sm-3 control-label">Code</label>
       <div class="col-sm-9">
-          <input type="text" class="form-control" id="loginOtpInput" placeholder="6-digit code">
+          <input type="text" class="form-control" id="loginOtpInput" placeholder="6-digit code" autocomplete="off">
           <div class="help-block" style="margin-top: 5px; font-size: 90%">Enter the six-digit code generated by your two factor authentication app.</div>
       </div>
     </div>
@@ -102,11 +102,11 @@ function do_login() {
   }
 
   // Exchange the email address & password for an API key.
-  api_credentials = [$('#loginEmail').val(), $('#loginPassword').val()]
+  api_credentials = { username: $('#loginEmail').val(), session_key: $('#loginPassword').val() }
 
   api(
-  "/me",
-  "GET",
+  "/login",
+  "POST",
   {},
   function(response) {
     // This API call always succeeds. It returns a JSON object indicating
@@ -141,7 +141,9 @@ function do_login() {
       // Login succeeded.
 
       // Save the new credentials.
-      api_credentials = [response.email, response.api_key];
+      api_credentials = { username: response.email,
+                          session_key: response.api_key,
+                          privileges: response.privileges };
 
       // Try to wipe the username/password information.
       $('#loginEmail').val('');
@@ -152,18 +154,21 @@ function do_login() {
       // Remember the credentials.
       if (typeof localStorage != 'undefined' && typeof sessionStorage != 'undefined') {
         if ($('#loginRemember').val()) {
-          localStorage.setItem("miab-cp-credentials", api_credentials.join(":"));
+          localStorage.setItem("miab-cp-credentials", JSON.stringify(api_credentials));
           sessionStorage.removeItem("miab-cp-credentials");
         } else {
           localStorage.removeItem("miab-cp-credentials");
-          sessionStorage.setItem("miab-cp-credentials", api_credentials.join(":"));
+          sessionStorage.setItem("miab-cp-credentials", JSON.stringify(api_credentials));
         }
       }
 
+      // Toggle menus.
+      show_hide_menus();
+
       // Open the next panel the user wants to go to. Do this after the XHR response
       // is over so that we don't start a new XHR request while this one is finishing,
       // which confuses the loading indicator.
-      setTimeout(function() { show_panel(!switch_back_to_panel || switch_back_to_panel == "login" ? 'system_status' : switch_back_to_panel) }, 300);
+      setTimeout(function() { show_panel(!switch_back_to_panel || switch_back_to_panel == "login" ? 'welcome' : switch_back_to_panel) }, 300);
     }
   },
   undefined,
@@ -183,4 +188,19 @@ function show_login() {
     }
   });
 }
+
+function show_hide_menus() {
+  var is_logged_in = (api_credentials != null);
+  var privs = api_credentials ? api_credentials.privileges : [];
+  $('.if-logged-in').toggle(is_logged_in);
+  $('.if-logged-in-admin, .if-logged-in-not-admin').toggle(false);
+  if (is_logged_in) {
+    $('.if-logged-in-not-admin').toggle(true);
+    privs.forEach(function(priv) {
+      $('.if-logged-in-' + priv).toggle(true);
+      $('.if-logged-in-not-' + priv).toggle(false);
+    });
+  }
+  $('.if-not-logged-in').toggle(!is_logged_in);
+}
 </script>

management/templates/mfa.html

@@ -1,34 +1,10 @@
 <style>
-    .twofactor #totp-setup,
-    .twofactor #disable-2fa,
-    .twofactor #output-2fa {
-        display: none;
-    }
-
-    .twofactor.loaded .loading-indicator {
-        display: none;
-    }
-
-    .twofactor.disabled #disable-2fa,
-    .twofactor.enabled #totp-setup {
-        display: none;
-    }
-
-    .twofactor.disabled #totp-setup,
-    .twofactor.enabled #disable-2fa {
-        display: block;
-    }
-
     .twofactor #totp-setup-qr img {
         display: block;
         width: 256px;
         max-width: 100%;
         height: auto;
     }
-
-    .twofactor #output-2fa.visible {
-        display: block;
-    }
 </style>
 
 <h2>Two-Factor Authentication</h2>
@@ -51,10 +27,11 @@ and ensure every administrator account for this control panel does the same.</st
 </div>
 
 <div class="twofactor">
-    <div class="loading-indicator">Loading...</div>
+    <div id="mfa-devices">
+    </div>
 
-    <form id="totp-setup">
-        <h3>Setup Instructions</h3>
+    <form id="totp-setup" style="display: none">
+        <h3>Add a TOTP Device</h3>
 
         <div class="form-group">
             <p>1. Install <a href="https://freeotp.github.io/">FreeOTP</a> or <a href="https://www.pcworld.com/article/3225913/what-is-two-factor-authentication-and-which-2fa-apps-are-best.html">any
@@ -85,24 +62,32 @@ and ensure every administrator account for this control panel does the same.</st
         </div>
     </form>
 
-    <form id="disable-2fa">
-        <div class="form-group">
-            <p>Two-factor authentication is active for your account<span id="mfa-device-label"></span>.</p>
-            <p>You will have to log into the admin panel again after disabling two-factor authentication.</p>
-        </div>
-        <div class="form-group">
-            <button type="submit" class="btn btn-danger">Disable Two-Factor Authentication</button>
-        </div>
+    <form id="webauthn-setup" style="display: none">
+        <h3>Add a WebAuthn Device</h3>
+        <p>If you have a WebAuthn device such as a YubiKey, plug it in and click Add WebAuthn Device.</p>
+        <button type="submit" class="btn" onclick="return do_enable_webauthn()">Add WebAuthn Device</button>
     </form>
 
-    <div id="output-2fa" class="panel panel-danger">
+    <div id="webauthn-setup" style="display: none">
+    </div>
+
+    <div id="output-2fa" class="panel panel-danger hidden">
         <div class="panel-body"></div>
     </div>
+
+    <div id="mfa-device-templates" style="display: none">
+      <form class="totp" style="margin: 1em 0; border: 1px solid #AAA; padding: 10px;">
+          <p>Two-factor authentication is active for your account<span class="mfa-device-label"></span>.</p>
+          <div class="form-group">
+              <button type="submit" class="btn btn-danger">Disable TOTP Device</button>
+          </div>
+          <p style="margin-bottom: 0">You will have to log into the admin panel again after disabling two-factor authentication.</p>
+      </form>
+    </div>
 </div>
 
 <script>
     var el = {
-        disableForm: document.getElementById('disable-2fa'),
         output: document.getElementById('output-2fa'),
         totpSetupForm: document.getElementById('totp-setup'),
         totpSetupToken: document.getElementById('totp-setup-token'),
@@ -110,6 +95,7 @@ and ensure every administrator account for this control panel does the same.</st
         totpSetupLabel: document.getElementById('totp-setup-label'),
         totpQr: document.getElementById('totp-setup-qr'),
         totpSetupSubmit: document.querySelector('#totp-setup-submit'),
+        webauthnSetupForm: document.getElementById('webauthn-setup'),
         wrapper: document.querySelector('.twofactor')
     }
 
@@ -130,6 +116,8 @@ and ensure every administrator account for this control panel does the same.</st
     }
 
     function render_totp_setup(provisioned_totp) {
+        $(el.totpSetupForm).show();
+
         var img = document.createElement('img');
         img.src = "data:image/png;base64," + provisioned_totp.qr_code_base64;
 
@@ -147,38 +135,50 @@ and ensure every administrator account for this control panel does the same.</st
         el.wrapper.classList.add('disabled');
     }
 
+    function arrayBufferToBase64(a) { return btoa(String.fromCharCode(...new Uint8Array(a))); }
+    function base64ToArrayBuffer(b) { return Uint8Array.from(atob(b), c => c.charCodeAt(0)); }
+
+    function render_webauthn_setup(provisioning) {
+        $(el.webauthnSetupForm).show();
+        provisioning.challenge = base64ToArrayBuffer(provisioning.challenge);
+        provisioning.user.id = new TextEncoder().encode(provisioning.user.name);
+        window.mailinabix_mfa_webauthn_provision = provisioning;
+    }
+
     function render_disable(mfa) {
-        el.disableForm.addEventListener('submit', do_disable);
-        el.wrapper.classList.add('enabled');
+        var panel = $('#mfa-device-templates .' + mfa.type).clone();
+        $('#mfa-devices').append(panel);
+        panel.attr('data-mfa-id', mfa.id);
+        panel.on('submit', do_disable);
         if (mfa.label)
-          $("#mfa-device-label").text(" on device '" + mfa.label + "'");
+          panel.find(".mfa-device-label").text(" on device '" + mfa.label + "'");
     }
 
     function hide_error() {
         el.output.querySelector('.panel-body').innerHTML = '';
-        el.output.classList.remove('visible');
+        el.output.classList.add('hidden');
     }
 
     function render_error(msg) {
         el.output.querySelector('.panel-body').innerHTML = msg;
-        el.output.classList.add('visible');
+        el.output.classList.remove('hidden');
     }
 
     function reset_view() {
         el.wrapper.classList.remove('loaded', 'disabled', 'enabled');
-
-        el.disableForm.removeEventListener('submit', do_disable);
+        $('#mfa-devices > *').remove();
 
         hide_error();
 
+        $(el.totpSetupForm).hide();
         el.totpSetupForm.reset();
         el.totpSetupForm.removeEventListener('submit', do_enable_totp);
-
         el.totpSetupSecret.setAttribute('value', '');
         el.totpSetupToken.removeEventListener('input', update_setup_disabled);
-
         el.totpSetupSubmit.setAttribute('disabled', '');
         el.totpQr.innerHTML = '';
+
+        $(el.webauthnSetupForm).hide();
     }
 
     function show_mfa() {
@@ -191,16 +191,16 @@ and ensure every administrator account for this control panel does the same.</st
             function(res) {
                 el.wrapper.classList.add('loaded');
 
-                var has_mfa = false;
                 res.enabled_mfa.forEach(function(mfa) {
                     if (mfa.type == "totp") {
                         render_disable(mfa);
-                        has_mfa = true;
                     }
                 });
 
-                if (!has_mfa)
-                  render_totp_setup(res.new_mfa.totp);
+                if (res.new_mfa.totp)
+                    render_totp_setup(res.new_mfa.totp);
+                if (res.new_mfa.webauthn && 'credentials' in navigator)
+                    render_webauthn_setup(res.new_mfa.webauthn);
             }
         );
     }
@@ -212,7 +212,7 @@ and ensure every administrator account for this control panel does the same.</st
         api(
             '/mfa/disable',
             'POST',
-            { type: 'totp' },
+            { id: $(this).attr('data-mfa-id') },
             function() {
                 do_logout();
             }
@@ -226,7 +226,7 @@ and ensure every administrator account for this control panel does the same.</st
         hide_error();
 
         api(
-            '/mfa/totp/enable',
+            '/mfa/enable/totp',
             'POST',
             {
                 token: $(el.totpSetupToken).val(),
@@ -239,4 +239,22 @@ and ensure every administrator account for this control panel does the same.</st
 
         return false;
     }
+
+    function do_enable_webauthn() {
+        navigator.credentials.create({ publicKey: window.mailinabix_mfa_webauthn_provision })
+        .then(function(creds) {
+            api(
+                '/mfa/enable/webauthn',
+                'POST',
+                {
+                    attestationObject: arrayBufferToBase64(creds.response['attestationObject']),
+                    clientDataJSON: arrayBufferToBase64(creds.response['clientDataJSON'])
+                },
+                function(res) { do_logout(); },
+                function(res) { render_error(res); }
+            );
+        });
+        return false;
+    }
+
 </script>

management/templates/munin.html

@@ -0,0 +1,20 @@
+<h2>Munin Monitoring</h2>
+
+<style>
+</style>
+
+<p>Opening munin in a new tab... You may need to allow pop-ups for this site.</p>
+
+<script>
+function show_munin() {
+  // Set the cookie.
+  api(
+    "/munin",
+    "GET",
+    { },
+    function(r) {
+      // Redirect.
+      window.open("/admin/munin/index.html", "_blank");
+    });
+}
+</script>

management/templates/system-backup.html

@@ -138,7 +138,7 @@
       </div>
   </div>
   <!-- Common -->
-  <div class="form-group backup-target-local backup-target-rsync backup-target-s3">
+  <div class="form-group backup-target-local backup-target-rsync backup-target-s3 backup-target-b2">
     <label for="min-age" class="col-sm-2 control-label">Retention Days:</label>
     <div class="col-sm-8">
       <input type="number" class="form-control" rows="1" id="min-age">

management/templates/users.html

@@ -203,7 +203,7 @@ function users_set_password(elem) {
   var email = $(elem).parents('tr').attr('data-email');
 
   var yourpw = "";
-  if (api_credentials != null && email == api_credentials[0])
+  if (api_credentials != null && email == api_credentials.username)
     yourpw = "<p class='text-danger'>If you change your own password, you will be logged out of this control panel and will need to log in again.</p>";
 
   show_modal_confirm(
@@ -232,7 +232,7 @@ function users_remove(elem) {
   var email = $(elem).parents('tr').attr('data-email');
 
   // can't remove yourself
-  if (api_credentials != null && email == api_credentials[0]) {
+  if (api_credentials != null && email == api_credentials.username) {
     show_modal_error("Archive User", "You cannot archive your own account.");
     return;
   }
@@ -264,7 +264,7 @@ function mod_priv(elem, add_remove) {
   var priv = $(elem).parents('td').find('.name').text();
 
   // can't remove your own admin access
-  if (priv == "admin" && add_remove == "remove" && api_credentials != null && email == api_credentials[0]) {
+  if (priv == "admin" && add_remove == "remove" && api_credentials != null && email == api_credentials.username) {
     show_modal_error("Modify Privileges", "You cannot remove the admin privilege from yourself.");
     return;
   }

management/templates/welcome.html

@@ -0,0 +1,16 @@
+<style>
+  .title {
+    margin: 1em;
+    text-align: center;
+  }
+
+  .subtitle {
+    margin: 2em;
+    text-align: center;
+  }
+</style>
+
+<h1 class="title">{{hostname}}</h1>
+
+<p class="subtitle">Welcome to your Mail-in-a-Box control panel.</p>
+

security.md

@@ -69,8 +69,6 @@ The following services are protected: SSH, IMAP (dovecot), SMTP submission (post
 
 Some other services running on the box may be missing fail2ban filters.
 
-`fail2ban` only blocks IPv4 addresses, however. If the box has a public IPv6 address, it is not protected from these attacks.
-
 Outbound Mail
 -------------
 

setup/dns.sh

@@ -62,7 +62,13 @@ for ip in $PRIVATE_IP $PRIVATE_IPV6; do
 	echo "  ip-address: $ip" >> /etc/nsd/nsd.conf;
 done
 
-echo "include: /etc/nsd/zones.conf" >> /etc/nsd/nsd.conf;
+# Create a directory for additional configuration directives, including
+# the zones.conf file written out by our management daemon.
+echo "include: /etc/nsd/nsd.conf.d/*.conf" >> /etc/nsd/nsd.conf;
+
+# Remove the old location of zones.conf that we generate. It will
+# now be stored in /etc/nsd/nsd.conf.d.
+rm -f /etc/nsd/zones.conf
 
 # Create DNSSEC signing keys.
 

setup/mail-users.sh

@@ -23,6 +23,7 @@ if [ ! -f $db_path ]; then
 	echo "CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, email TEXT NOT NULL UNIQUE, password TEXT NOT NULL, extra, privileges TEXT NOT NULL DEFAULT '');" | sqlite3 $db_path;
 	echo "CREATE TABLE aliases (id INTEGER PRIMARY KEY AUTOINCREMENT, source TEXT NOT NULL UNIQUE, destination TEXT NOT NULL, permitted_senders TEXT);" | sqlite3 $db_path;
 	echo "CREATE TABLE mfa (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, type TEXT NOT NULL, secret TEXT NOT NULL, mru_token TEXT, label TEXT, FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE);" | sqlite3 $db_path;
+	echo "CREATE TABLE auto_aliases (id INTEGER PRIMARY KEY AUTOINCREMENT, source TEXT NOT NULL UNIQUE, destination TEXT NOT NULL, permitted_senders TEXT);" | sqlite3 $db_path;
 fi
 
 # ### User Authentication
@@ -100,8 +101,12 @@ EOF
 # ### Destination Validation
 
 # Use a Sqlite3 database to check whether a destination email address exists,
-# and to perform any email alias rewrites in Postfix.
+# and to perform any email alias rewrites in Postfix. Additionally, we disable
+# SMTPUTF8 because Dovecot's LMTP server that delivers mail to inboxes does
+# not support it, and if a message is received with the SMTPUTF8 flag it will
+# bounce.
 tools/editconf.py /etc/postfix/main.cf \
+	smtputf8_enable=no \
 	virtual_mailbox_domains=sqlite:/etc/postfix/virtual-mailbox-domains.cf \
 	virtual_mailbox_maps=sqlite:/etc/postfix/virtual-mailbox-maps.cf \
 	virtual_alias_maps=sqlite:/etc/postfix/virtual-alias-maps.cf \
@@ -110,7 +115,7 @@ tools/editconf.py /etc/postfix/main.cf \
 # SQL statement to check if we handle incoming mail for a domain, either for users or aliases.
 cat > /etc/postfix/virtual-mailbox-domains.cf << EOF;
 dbpath=$db_path
-query = SELECT 1 FROM users WHERE email LIKE '%%@%s' UNION SELECT 1 FROM aliases WHERE source LIKE '%%@%s'
+query = SELECT 1 FROM users WHERE email LIKE '%%@%s' UNION SELECT 1 FROM aliases WHERE source LIKE '%%@%s' UNION SELECT 1 FROM auto_aliases WHERE source LIKE '%%@%s'
 EOF
 
 # SQL statement to check if we handle incoming mail for a user.
@@ -145,7 +150,7 @@ EOF
 # empty destination here so that other lower priority rules might match.
 cat > /etc/postfix/virtual-alias-maps.cf << EOF;
 dbpath=$db_path
-query = SELECT destination from (SELECT destination, 0 as priority FROM aliases WHERE source='%s' AND destination<>'' UNION SELECT email as destination, 1 as priority FROM users WHERE email='%s') ORDER BY priority LIMIT 1;
+query = SELECT destination from (SELECT destination, 0 as priority FROM aliases WHERE source='%s' AND destination<>'' UNION SELECT email as destination, 1 as priority FROM users WHERE email='%s' UNION SELECT destination, 2 as priority FROM auto_aliases WHERE source='%s' AND destination<>'') ORDER BY priority LIMIT 1;
 EOF
 
 # Restart Services

setup/management.sh

@@ -49,8 +49,8 @@ hide_output $venv/bin/pip install --upgrade pip
 # NOTE: email_validator is repeated in setup/questions.sh, so please keep the versions synced.
 hide_output $venv/bin/pip install --upgrade \
 	rtyaml "email_validator>=1.0.0" "exclusiveprocess" \
-	flask dnspython python-dateutil \
-  qrcode[pil] pyotp \
+	flask dnspython python-dateutil expiringdict \
+	qrcode[pil] pyotp pywarp \
 	"idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver b2sdk
 
 # CONFIGURATION

setup/migrate.py

@@ -186,6 +186,11 @@ def migration_13(env):
 	db = os.path.join(env["STORAGE_ROOT"], 'mail/users.sqlite')
 	shell("check_call", ["sqlite3", db, "CREATE TABLE mfa (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, type TEXT NOT NULL, secret TEXT NOT NULL, mru_token TEXT, label TEXT, FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE);"])
 
+def migration_14(env):
+	# Add the "auto_aliases" table.
+	db = os.path.join(env["STORAGE_ROOT"], 'mail/users.sqlite')
+	shell("check_call", ["sqlite3", db, "CREATE TABLE auto_aliases (id INTEGER PRIMARY KEY AUTOINCREMENT, source TEXT NOT NULL UNIQUE, destination TEXT NOT NULL, permitted_senders TEXT);"])
+
 ###########################################################
 
 def get_current_migration():

setup/webmail.sh

@@ -29,8 +29,8 @@ apt_install \
 # Combine the Roundcube version number with the commit hash of plugins to track
 # whether we have the latest version of everything.
 
-VERSION=1.4.11
-HASH=3877f0e70f29e7d0612155632e48c3db1e626be3
+VERSION=1.5-rc
+HASH=a7cb2a39702536d769c7ff93f716e27f0b93f9d9
 PERSISTENT_LOGIN_VERSION=6b3fc450cae23ccb2f393d0ef67aa319e877e435 # version 5.2.0
 HTML5_NOTIFIER_VERSION=68d9ca194212e15b3c7225eb6085dbcf02fd13d7 # version 0.6.4+
 CARDDAV_VERSION=3.0.3
@@ -132,6 +132,7 @@ cat > $RCM_CONFIG <<EOF;
 \$config['plugins'] = array('html5_notifier', 'archive', 'zipdownload', 'password', 'managesieve', 'jqueryui', 'persistent_login', 'carddav');
 \$config['skin'] = 'elastic';
 \$config['login_autocomplete'] = 2;
+\$config['login_username_filter'] = 'email';
 \$config['password_charset'] = 'UTF-8';
 \$config['junk_mbox'] = 'Spam';
 ?>

tests/fail2ban.py

@@ -232,7 +232,7 @@ if __name__ == "__main__":
 	run_test(managesieve_test, [], 20, 30, 4)
 
 	# Mail-in-a-Box control panel
-	run_test(http_test, ["/admin/me", 200], 20, 30, 1)
+	run_test(http_test, ["/admin/login", 200], 20, 30, 1)
 
 	# Munin via the Mail-in-a-Box control panel
 	run_test(http_test, ["/admin/munin/", 401], 20, 30, 1)