TOTP two-factor authentication

CHANGELOG.md

@@ -1,46 +1,24 @@
 CHANGELOG
 =========
 
-v0.07 (February 28, 2015)
--------------------------
-
-Mail:
-
-* If the box manages mail for a domain and a subdomain of that domain, outbound mail from the subdomain was not DKIM-signed and would therefore fail DMARC tests on the receiving end, possibly result in the mail heading into spam folders.
-* Auto-configuration for Mozilla Thunderbird, Evolution, KMail, and Kontact is now available.
-* Domains that only have a catch-all alias or domain alias no longer automatically create/require admin@ and postmaster@ addresses since they'll forward anyway.
-* Roundcube is updated to version 1.1.0.
-* Authentication-Results headers for DMARC are now added to incoming mail.
-
-DNS:
-
-* If a custom CNAME record is set on a 'www' subdomain, the default A/AAAA records were preventing the CNAME from working.
-* If a custom DNS A record overrides one provided by the box, the a corresponding default IPv6 record by the box is removed since it will probably be incorrect.
-* Internationalized domain names (IDNs) are now supported for DNS and web, but email is not yet tested.
-
-Web:
-
-* Static websites now deny access to certain dot (.) files and directories which typically have sensitive info: .ht*, .svn*, .git*, .hg*, .bzr*.
-* The nginx server no longer reports its version and OS for better privacy.
-* The HTTP->HTTPS redirect is now more efficient.
-* When serving a 'www.' domain, reuse the SSL certificate for the parent domain if it covers the 'www' subdomain too
-* If a custom DNS CNAME record is set on a domain, don't offer to put a website on that domain. (Same logic already applies to custom A/AAAA records.)
+Development
+-----------
 
 Control panel:
 
 * Status checks now check that system services are actually running by pinging each port that should have something running on it.
-* The status checks are now parallelized so they may be a little faster.
-* The status check for MX records now allow any priority, in case an unusual setup is required.
-* The interface for setting website domain-specific directories is simplified.
-* The mail guide now says that to use Outlook, Outlook 2007 or later on Windows 7 and later is required.
-* External DNS settings now skip the special "_secondary_nameserver" key which is used for storing secondary NS information.
+* If a custom CNAME record is set on a 'www' subdomain, the default A/AAAA records were preventing the CNAME from working.
 
 Setup:
 
 * Install cron if it isn't already installed.
 * Fix a units problem in the minimum memory check.
-* If you override the STORAGE_ROOT, your setting will now persist if you re-run setup.
-* Hangs due to apt wanting the user to resolve a conflict should now be fixed (apt will just clobber the problematic file now).
+
+Miscellaneous:
+
+* Internationalized domain names (IDNs) are now supported for DNS and web, but email is not yet tested.
+* Domains that only have a catch-all alias or domain alias no longer automatically create/require admin@ and postmaster@ addresses since they'll forward anyway.
+
 
 v0.06 (January 4, 2015)
 -----------------------

conf/dovecot-checkpassword.py

@@ -0,0 +1,53 @@
+#!/usr/bin/python3
+#
+# This script implement's Dovecot's checkpassword authentication mechanism:
+# http://wiki2.dovecot.org/AuthDatabase/CheckPassword?action=show&redirect=PasswordDatabase%2FCheckPassword
+#
+# This allows us to perform our own password validation, such as for two-factor authentication,
+# which Dovecot does not have any native support for.
+#
+# We will issue an HTTP request to our management server to perform authentication.
+
+import sys, os, urllib.request, base64, json, traceback
+
+try:
+	# Read fd 3 which provides the username and password separated
+	# by NULLs and two other undocumented/empty fields.
+	creds = b''
+	while True:
+		b = os.read(3, 1024)
+		if len(b) == 0: break
+		creds += b
+	email, pw, dummy, dummy = creds.split(b'\x00')
+
+	# Call the management server's "/me" method with the
+	# provided credentials
+	req = urllib.request.Request('http://127.0.0.1:10222/me')
+	req.add_header(b'Authorization', b'Basic ' + base64.b64encode(email + b':' + pw))
+	response = urllib.request.urlopen(req)
+
+	# The response is always success and always a JSON object
+	# indicating the authentication result.
+	resp = response.read().decode('utf8')
+	resp = json.loads(resp)
+	if not isinstance(resp, dict): raise ValueError("Response is not a JSON object.")
+
+except:
+	# Handle all exceptions. Print what happens (ends up in syslog, thanks
+	# to dovecot) and return an exit status that indicates temporary failure,
+	# which is passed on to the authenticating client.
+	traceback.print_exc()
+	print(json.dumps(dict(os.environ), indent=2), file=sys.stderr)
+	sys.exit(111)
+
+if resp.get('status') != 'authorized':
+	# Indicates login failure.
+	# (sys.exit should not be inside the try block.)
+	sys.exit(1)
+
+# Signal ok by executing the indicated process, per the Dovecot
+# protocol. (Note that the second parameter is the 0th argument
+# to the called process, which is required and is typically the
+# file itself.)
+os.execl(sys.argv[1], sys.argv[1])
+

conf/mozilla-autoconfig.xml

@@ -1,44 +0,0 @@
-<?xml version="1.0"?>
-<clientConfig version="1.1">
-    <emailProvider id="PRIMARY_HOSTNAME">
-      <domain>PRIMARY_HOSTNAME</domain>
-
-      <displayName>PRIMARY_HOSTNAME (Mail-in-a-Box)</displayName>
-      <displayShortName>PRIMARY_HOSTNAME</displayShortName>
-
-      <incomingServer type="imap">
-         <hostname>PRIMARY_HOSTNAME</hostname>
-         <port>993</port>
-         <socketType>SSL</socketType>
-         <username>%EMAILADDRESS%</username>
-         <authentication>password-cleartext</authentication>
-      </incomingServer>
-
-      <outgoingServer type="smtp">
-         <hostname>PRIMARY_HOSTNAME</hostname>
-         <port>587</port>
-         <socketType>STARTTLS</socketType>
-         <username>%EMAILADDRESS%</username>
-         <authentication>password-cleartext</authentication>
-         <addThisServer>true</addThisServer>
-         <useGlobalPreferredServer>true</useGlobalPreferredServer>
-      </outgoingServer>
-
-      <documentation url="https://PRIMARY_HOSTNAME/">
-         <descr lang="en">PRIMARY_HOSTNAME website.</descr>
-      </documentation>
-    </emailProvider>
-
-    <webMail>
-      <loginPage url="https://PRIMARY_HOSTNAME/mail/" />
-      <loginPageInfo url="https://PRIMARY_HOSTNAME/mail/" >
-        <username>%EMAILADDRESS%</username>
-        <usernameField id="rcmloginuser" name="_user" />
-        <passwordField id="rcmloginpwd" name="_pass" />
-        <loginButton id="rcmloginsubmit" />
-      </loginPageInfo>
-    </webMail>
-
-    <clientConfigUpdate url="https://PRIMARY_HOSTNAME/.well-known/autoconfig/mail/config-v1.1.xml" />
-
-</clientConfig>

conf/nginx.conf

@@ -7,15 +7,7 @@ server {
 
 	server_name $HOSTNAME;
 	root /tmp/invalid-path-nothing-here;
-
-	# Improve privacy: Hide version an OS information on
-	# error pages and in the "Server" HTTP-Header.
-	server_tokens off;
-
-	# Redirect using the 'return' directive and the built-in
-	# variable '$request_uri' to avoid any capturing, matching
-	# or evaluation of regular expressions.
-	return 301 https://$HOSTNAME$request_uri;
+	rewrite ^/(.*)$ https://$HOSTNAME/$1 permanent;
 }
 
 # The secure HTTPS server.
@@ -25,10 +17,6 @@ server {
 
 	server_name $HOSTNAME;
 
-	# Improve privacy: Hide version an OS information on
-	# error pages and in the "Server" HTTP-Header.
-	server_tokens off;
-
 	ssl_certificate $SSL_CERTIFICATE;
 	ssl_certificate_key $SSL_KEY;
 	include /etc/nginx/nginx-ssl.conf;
@@ -50,16 +38,6 @@ server {
 	location = /mailinabox.mobileconfig {
 		alias /var/lib/mailinabox/mobileconfig.xml;
 	}
-	location = /.well-known/autoconfig/mail/config-v1.1.xml {
-		alias /var/lib/mailinabox/mozilla-autoconfig.xml;
-	}
-
-	# Disable viewing dotfiles (.htaccess, .svn, .git, etc.)
-	location ~ /\.(ht|svn|git|hg|bzr) {
-		log_not_found off;
-		access_log off;
-		deny all;
-	}
 
 	# Roundcube Webmail configuration.
 	rewrite ^/mail$ /mail/ redirect;

management/auth.py

@@ -1,4 +1,4 @@
-import base64, os, os.path, hmac
+import base64, os, os.path, hmac, json
 
 from flask import make_response
 
@@ -97,6 +97,17 @@ class KeyAuthService:
 			# email address does not correspond to a user.
 			pw_hash = get_mail_password(email, env)
 
+			# If 2FA is set up, get the first factor and authenticate against
+			# that first.
+			twofa = None
+			if pw_hash.startswith("{TOTP}"):
+				twofa = json.loads(pw_hash[6:])
+				pw_hash = twofa["first_factor"]
+				try:
+					pw, twofa_code = pw.split(" ", 1)
+				except:
+					twofa_code = ""
+
 			# Authenticate.
 			try:
 				# Use 'doveadm pw' to check credentials. doveadm will return
@@ -111,6 +122,14 @@ class KeyAuthService:
 				# Login failed.
 				raise ValueError("Invalid password.")
 
+			# Check second factor.
+			if twofa:
+				import oath
+				ok, drift = oath.accept_totp(twofa["secret"], twofa_code, drift=twofa["drift"])
+				if not ok:
+					raise ValueError("Invalid 2FA code.")
+
+
 		# Get privileges for authorization.
 
 		# (This call should never fail on a valid user. But if it did fail, it would

management/buy_certificate.py

@@ -0,0 +1,155 @@
+#!/usr/bin/python3
+
+# Helps you purchase a SSL certificate from Gandi.net using
+# their API.
+#
+# Before you begin:
+#  1) Create an account on Gandi.net.
+#  2) Pre-pay $16 into your account at https://www.gandi.net/prepaid/operations. Wait until the payment goes through.
+#  3) Activate your API key first on the test platform (wait a while, refresh the page) and then activate the production API at https://www.gandi.net/admin/api_key.
+
+import sys, re,  os.path, urllib.request
+import xmlrpc.client
+import rtyaml
+
+from utils import load_environment, shell
+from web_update import get_web_domains, get_domain_ssl_files, get_web_root
+from status_checks import check_certificate
+
+def buy_ssl_certificate(api_key, domain, command, env):
+	if domain != env['PRIMARY_HOSTNAME'] \
+		and domain not in get_web_domains(env):
+		raise ValueError("Domain is not %s or a domain we're serving a website for." % env['PRIMARY_HOSTNAME'])
+
+	# Initialize.
+
+	gandi = xmlrpc.client.ServerProxy('https://rpc.gandi.net/xmlrpc/')
+
+	try:
+		existing_certs = gandi.cert.list(api_key)
+	except Exception as e:
+		if "Invalid API key" in str(e):
+			print("Invalid API key. Check that you copied the API Key correctly from https://www.gandi.net/admin/api_key.")
+			sys.exit(1)
+		else:
+			raise
+
+	# Where is the SSL cert stored?
+
+	ssl_key, ssl_certificate = get_domain_ssl_files(domain, env)
+
+	# Have we already created a cert for this domain?
+
+	for cert in existing_certs:
+		if cert['cn'] == domain:
+			break
+	else:
+		# No existing cert found. Purchase one.
+		if command != 'purchase':
+			print("No certificate or order found yet. If you haven't yet purchased a certificate, run ths script again with the 'purchase' command. Otherwise wait a moment and try again.")
+			sys.exit(1)
+		else:
+			# Start an order for a single standard SSL certificate.
+			# Use DNS validation. Web-based validation won't work because they
+			# require a file on HTTP but not HTTPS w/o redirects and we don't
+			# serve anything plainly over HTTP. Email might be another way but
+			# DNS is easier to automate.
+			op = gandi.cert.create(api_key, {
+				"csr": open(ssl_csr_path).read(),
+				"dcv_method": "dns",
+				"duration": 1, # year?
+				"package": "cert_std_1_0_0",
+				})
+			print("An SSL certificate has been ordered.")
+			print()
+			print(op)
+			print()
+			print("In a moment please run this script again with the 'setup' command.")
+
+	if cert['status'] == 'pending':
+		# Get the information we need to update our DNS with a code so that
+		# Gandi can verify that we own the domain.
+
+		dcv = gandi.cert.get_dcv_params(api_key, {
+				"csr": open(ssl_csr_path).read(),
+				"cert_id": cert['id'],
+				"dcv_method": "dns",
+				"duration": 1, # year?
+				"package": "cert_std_1_0_0",
+				})
+		if dcv["dcv_method"] != "dns":
+			raise Exception("Certificate ordered with an unknown validation method.")
+
+		# Update our DNS data.
+
+		dns_config = env['STORAGE_ROOT'] + '/dns/custom.yaml'
+		if os.path.exists(dns_config):
+			dns_records = rtyaml.load(open(dns_config))
+		else:
+			dns_records = { }
+
+		qname = dcv['md5'] + '.' + domain
+		value = dcv['sha1'] + '.comodoca.com.'
+		dns_records[qname] = { "CNAME": value }
+
+		with open(dns_config, 'w') as f:
+			f.write(rtyaml.dump(dns_records))
+
+		shell('check_call', ['tools/dns_update'])
+
+		# Okay, done with this step.
+
+		print("DNS has been updated. Gandi will check within 60 minutes.")
+		print()
+		print("See https://www.gandi.net/admin/ssl/%d/details for the status of this order." % cert['id'])
+
+	elif cert['status'] == 'valid':
+		# The certificate is ready.
+
+		# Check before we overwrite something we shouldn't.
+		if os.path.exists(ssl_certificate):
+			cert_status, cert_status_details = check_certificate(None, ssl_certificate, None)
+			if cert_status != "SELF-SIGNED":
+				print("Please back up and delete the file %s so I can save your new certificate." % ssl_certificate)
+				sys.exit(1)
+
+		# Form the certificate.
+
+		# The certificate comes as a long base64-encoded string. Break in
+		# into lines in the usual way.
+		pem = "-----BEGIN CERTIFICATE-----\n"
+		pem += "\n".join(chunk for chunk in re.split(r"(.{64})", cert['cert']) if chunk != "")
+		pem += "\n-----END CERTIFICATE-----\n\n"
+
+		# Append intermediary certificates.
+		pem += urllib.request.urlopen("https://www.gandi.net/static/CAs/GandiStandardSSLCA.pem").read().decode("ascii")
+
+		# Write out.
+
+		with open(ssl_certificate, "w") as f:
+			f.write(pem)
+
+		print("The certificate has been installed in %s. Restarting services..." % ssl_certificate)
+
+		# Restart dovecot and if this is for PRIMARY_HOSTNAME.
+
+		if domain == env['PRIMARY_HOSTNAME']:
+			shell('check_call', ["/usr/sbin/service", "dovecot", "restart"])
+			shell('check_call', ["/usr/sbin/service", "postfix", "restart"])
+
+		# Restart nginx in all cases.
+
+		shell('check_call', ["/usr/sbin/service", "nginx", "restart"])
+
+	else:
+		print("The certificate has an unknown status. Please check https://www.gandi.net/admin/ssl/%d/details for the status of this order." % cert['id'])
+
+if __name__ == "__main__":
+	if len(sys.argv) < 4:
+		print("Usage: python management/buy_certificate.py gandi_api_key domain_name {purchase, setup}")
+		sys.exit(1)
+	api_key = sys.argv[1]
+	domain_name = sys.argv[2]
+	cmd = sys.argv[3]
+	buy_ssl_certificate(api_key, domain_name, cmd, load_environment())
+

management/daemon.py

@@ -7,16 +7,10 @@ from functools import wraps
 from flask import Flask, request, render_template, abort, Response
 
 import auth, utils
-from mailconfig import get_mail_users, get_mail_users_ex, get_admins, add_mail_user, set_mail_password, remove_mail_user
+from mailconfig import get_mail_users, get_mail_users_ex, get_admins, add_mail_user, set_mail_password, remove_mail_user, get_mail_password
 from mailconfig import get_mail_user_privileges, add_remove_mail_user_privilege
 from mailconfig import get_mail_aliases, get_mail_aliases_ex, get_mail_domains, add_mail_alias, remove_mail_alias
 
-# Create a worker pool for the status checks. The pool should
-# live across http requests so we don't baloon the system with
-# processes.
-import multiprocessing.pool
-pool = multiprocessing.pool.Pool(processes=10)
-
 env = utils.load_environment()
 
 auth_service = auth.KeyAuthService()
@@ -46,6 +40,7 @@ def authorized_personnel_only(viewfunc):
 		# Authorized to access an API view?
 		if "admin" in privs:
 			# Call view func.	
+			request.user_email = email
 			return viewfunc(*args, **kwargs)
 		elif not error:
 			error = "You are not an administrator."
@@ -121,6 +116,81 @@ def me():
 	# Return.
 	return json_response(resp)
 
+# ME
+
+@app.route('/me/2fa')
+@authorized_personnel_only
+def twofa_status():
+	pw = get_mail_password(request.user_email, env)
+	if pw.startswith("{SHA512-CRYPT}"):
+		method = "password-only"
+	elif pw.startswith("{TOTP}"):
+		method = "TOTP 2FA"
+	else:
+		method = "unknown"
+
+	return json_response({
+		"method": method
+		})
+
+@app.route('/me/2fa/totp/initialize', methods=['POST'])
+@authorized_personnel_only
+def twofa_initialize():
+	# Generate a Google Authenticator URI that encodes TOTP info.
+	import urllib.parse, base64, qrcode, io, binascii
+
+	secret = os.urandom(32)
+	uri = "otpauth://totp/%s:%s?secret=%s&issuer=%s&digits=%d&algorithm=%s" % (
+		urllib.parse.quote(env['PRIMARY_HOSTNAME']),
+		urllib.parse.quote(request.user_email),
+		base64.b32encode(secret).decode("ascii").lower().replace("=", ""),
+		urllib.parse.quote(env['PRIMARY_HOSTNAME']),
+		6,
+		"sha1"
+		)
+
+	image_buffer = io.BytesIO()
+	im = qrcode.make(uri)
+	im.save(image_buffer, 'png')
+
+	return json_response({
+		"uri": uri,
+		"secret": binascii.hexlify(secret).decode('ascii'),
+		"qr": base64.b64encode(image_buffer.getvalue()).decode('ascii')
+		})
+
+@app.route('/me/2fa/totp/activate', methods=['POST'])
+@authorized_personnel_only
+def twofa_activate():
+	import oath
+	ok, drift = oath.accept_totp(request.form['secret'], request.form['code'])
+	if ok:
+		# use the user's current plain password as the first_factor
+		# of 2FA.
+		existing_pw = get_mail_password(request.user_email, env)
+		if existing_pw.startswith("{TOTP}"):
+			existing_pw = json.loads(existing_pw)["first_factor"]
+
+		pw = "{TOTP}" + json.dumps({
+			"secret": request.form['secret'],
+			"drift": drift,
+			"first_factor": existing_pw,
+		})
+
+		set_mail_password(request.user_email, pw, env, already_hashed=True)
+
+		return json_response({
+			"status": "ok",
+			"message": "TOTP 2FA installed."
+		})
+
+	else:
+		return json_response({
+			"status": "fail",
+			"message": "The activation code was not right. Try again?"
+		})
+
+
 # MAIL
 
 @app.route('/mail/users')
@@ -278,7 +348,7 @@ def dns_get_dump():
 @authorized_personnel_only
 def ssl_get_csr(domain):
 	from web_update import get_domain_ssl_files, create_csr
-	ssl_key, ssl_certificate, ssl_via = get_domain_ssl_files(domain, env)
+	ssl_key, ssl_certificate = get_domain_ssl_files(domain, env)
 	return create_csr(domain, ssl_key, env)
 
 @app.route('/ssl/install', methods=['POST'])
@@ -324,7 +394,7 @@ def system_status():
 		def print_line(self, message, monospace=False):
 			self.items[-1]["extra"].append({ "text": message, "monospace": monospace })
 	output = WebOutput()
-	run_checks(env, output, pool)
+	run_checks(env, output)
 	return json_response(output.items)
 
 @app.route('/system/updates')

management/dns_update.py

@@ -122,7 +122,7 @@ def do_dns_update(env, force=False):
 		shell('check_call', ["/usr/sbin/service", "nsd", "restart"])
 
 	# Write the OpenDKIM configuration tables.
-	if write_opendkim_tables(domains, env):
+	if write_opendkim_tables(zonefiles, env):
 		# Settings changed. Kick opendkim.
 		shell('check_call', ["/usr/sbin/service", "opendkim", "restart"])
 		if len(updated_domains) == 0:
@@ -222,11 +222,7 @@ def build_zone(domain, all_domains, additional_records, env, is_zone=True):
 	for qname, rtype, value, explanation in defaults:
 		if value is None or value.strip() == "": continue # skip IPV6 if not set
 		if not is_zone and qname == "www": continue # don't create any default 'www' subdomains on what are themselves subdomains
-		# Set the default record, but not if:
-		# (1) there is not a user-set record of the same type already
-		# (2) there is not a CNAME record already, since you can't set both and who knows what takes precedence
-		# (2) there is not an A record already (if this is an A record this is a dup of (1), and if this is an AAAA record then don't set a default AAAA record if the user sets a custom A record, since the default wouldn't make sense and it should not resolve if the user doesn't provide a new AAAA record)
-		if not has_rec(qname, rtype) and not has_rec(qname, "CNAME") and not has_rec(qname, "A"):
+		if not has_rec(qname, rtype) and not has_rec(qname, "CNAME"):
 			records.append((qname, rtype, value, explanation))
 
 	# Append the DKIM TXT record to the zone as generated by OpenDKIM.
@@ -258,10 +254,6 @@ def build_zone(domain, all_domains, additional_records, env, is_zone=True):
 
 def get_custom_records(domain, additional_records, env):
 	for qname, value in additional_records.items():
-		# We don't count the secondary nameserver config (if present) as a record - that would just be
-		# confusing to users. Instead it is accessed/manipulated directly via (get/set)_custom_dns_config.
-		if qname == "_secondary_nameserver": continue
-
 		# Is this record for the domain or one of its subdomains?
 		# If `domain` is None, return records for all domains.
 		if domain is not None and qname != domain and not qname.endswith("." + domain): continue
@@ -620,9 +612,8 @@ def sign_zone(domain, zonefile, env):
 
 ########################################################################
 
-def write_opendkim_tables(domains, env):
-	# Append a record to OpenDKIM's KeyTable and SigningTable for each domain
-	# that we send mail from (zones and all subdomains).
+def write_opendkim_tables(zonefiles, env):
+	# Append a record to OpenDKIM's KeyTable and SigningTable for each domain.
 
 	opendkim_key_file = os.path.join(env['STORAGE_ROOT'], 'mail/dkim/mail.private')
 
@@ -641,7 +632,7 @@ def write_opendkim_tables(domains, env):
 		"SigningTable":
 			"".join(
 				"*@{domain} {domain}\n".format(domain=domain)
-				for domain in domains
+				for domain, zonefile in zonefiles
 			),
 
 		# The KeyTable specifies the signing domain, the DKIM selector, and the
@@ -650,7 +641,7 @@ def write_opendkim_tables(domains, env):
 		"KeyTable":
 			"".join(
 				"{domain} {domain}:mail:{key_file}\n".format(domain=domain, key_file=opendkim_key_file)
-				for domain in domains
+				for domain, zonefile in zonefiles
 			),
 	}
 

management/mailconfig.py

@@ -311,15 +311,16 @@ def add_mail_user(email, pw, privs, env):
 	# Update things in case any new domains are added.
 	return kick(env, "mail user added")
 
-def set_mail_password(email, pw, env):
+def set_mail_password(email, pw, env, already_hashed=False):
 	# accept IDNA domain names but normalize to Unicode before going into database
 	email = sanitize_idn_email_address(email)
 
 	# validate that password is acceptable
-	validate_password(pw)
-	
-	# hash the password
-	pw = hash_password(pw)
+	if not already_hashed:
+		# Validate and hash the password. Skip if we're providing
+		# a raw hashed password value.
+		validate_password(pw)
+		pw = hash_password(pw)
 
 	# update the database
 	conn, c = open_database(env, with_connection=True)

management/status_checks.py

@@ -17,12 +17,12 @@ from mailconfig import get_mail_domains, get_mail_aliases
 
 from utils import shell, sort_domains, load_env_vars_from_file
 
-def run_checks(env, output, pool):
+def run_checks(env, output):
 	# run systems checks
 	output.add_heading("System")
 
 	# check that services are running
-	if not run_services_checks(env, output, pool):
+	if not run_services_checks(env, output):
 		# If critical services are not running, stop. If bind9 isn't running,
 		# all later DNS checks will timeout and that will take forever to
 		# go through, and if running over the web will cause a fastcgi timeout.
@@ -37,21 +37,13 @@ def run_checks(env, output, pool):
 
 	# perform other checks asynchronously
 
-	run_network_checks(env, output)
-	run_domain_checks(env, output, pool)
+	pool = multiprocessing.pool.Pool(processes=1)
+	r1 = pool.apply_async(run_network_checks, [env])
+	r2 = run_domain_checks(env)
+	r1.get().playback(output)
+	r2.playback(output)
 
-def get_ssh_port():
-    # Returns ssh port
-    output = shell('check_output', ['sshd', '-T'])
-    returnNext = False
-
-    for e in output.split():
-        if returnNext:
-            return int(e)
-        if e == "port":
-            returnNext = True
-
-def run_services_checks(env, output, pool):
+def run_services_checks(env, output):
 	# Check that system services are running.
 
 	services = [
@@ -62,12 +54,11 @@ def run_services_checks(env, output, pool):
 		{ "name": "Postgrey", "port": 10023, "public": False, },
 		{ "name": "Spamassassin", "port": 10025, "public": False, },
 		{ "name": "OpenDKIM", "port": 8891, "public": False, },
-		{ "name": "OpenDMARC", "port": 8893, "public": False, },
 		{ "name": "Memcached", "port": 11211, "public": False, },
 		{ "name": "Sieve (dovecot)", "port": 4190, "public": True, },
 		{ "name": "Mail-in-a-Box Management Daemon", "port": 10222, "public": False, },
 
-		{ "name": "SSH Login (ssh)", "port": get_ssh_port(), "public": True, },
+		{ "name": "SSH Login (ssh)", "port": 22, "public": True, },
 		{ "name": "Public DNS (nsd4)", "port": 53, "public": True, },
 		{ "name": "Incoming Mail (SMTP/postfix)", "port": 25, "public": True, },
 		{ "name": "Outgoing Mail (SMTP 587/postfix)", "port": 587, "public": True, },
@@ -79,6 +70,7 @@ def run_services_checks(env, output, pool):
 
 	all_running = True
 	fatal = False
+	pool = multiprocessing.pool.Pool(processes=10)
 	ret = pool.starmap(check_service, ((i, service, env) for i, service in enumerate(services)), chunksize=1)
 	for i, running, fatal2, output2 in sorted(ret):
 		all_running = all_running and running
@@ -98,28 +90,13 @@ def check_service(i, service, env):
 	s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 	s.settimeout(1)
 	try:
-		try:
-			s.connect((
-				"127.0.0.1" if not service["public"] else env['PUBLIC_IP'],
-				service["port"]))
-			running = True
-		except OSError as e1:
-			if service["public"] and service["port"] != 53:
-				# For public services (except DNS), try the private IP as a fallback.
-				s1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-				s1.settimeout(1)
-				try:
-					s1.connect(("127.0.0.1", service["port"]))
-					output.print_error("%s is running but is not publicly accessible at %s:%d (%s)." % (service['name'], env['PUBLIC_IP'], service['port'], str(e1)))
-				except:
-					raise e1
-				finally:
-					s1.close()
-			else:
-				raise
-
+		s.connect((
+			"127.0.0.1" if not service["public"] else env['PUBLIC_IP'],
+			service["port"]))
+		running = True
+	
 	except OSError as e:
-		output.print_error("%s is not running (%s; port %d)." % (service['name'], str(e), service['port']))
+		output.print_error("%s is not running (%s)." % (service['name'], str(e)))
 
 		# Why is nginx not running?
 		if service["port"] in (80, 443):
@@ -185,9 +162,10 @@ def check_free_disk_space(env, output):
 	else:
 		output.print_error(disk_msg)
 
-def run_network_checks(env, output):
+def run_network_checks(env):
 	# Also see setup/network-checks.sh.
 
+	output = BufferedOutput()
 	output.add_heading("Network")
 
 	# Stop if we cannot make an outbound connection on port 25. Many residential
@@ -215,7 +193,9 @@ def run_network_checks(env, output):
 			which may prevent recipients from receiving your email. See http://www.spamhaus.org/query/ip/%s."""
 			% (env['PUBLIC_IP'], zen, env['PUBLIC_IP']))
 
-def run_domain_checks(env, output, pool):
+	return output
+
+def run_domain_checks(env):
 	# Get the list of domains we handle mail for.
 	mail_domains = get_mail_domains(env)
 
@@ -235,10 +215,13 @@ def run_domain_checks(env, output, pool):
 	# Parallelize the checks across a worker pool.
 	args = ((domain, env, dns_domains, dns_zonefiles, mail_domains, web_domains)
 		for domain in domains_to_check)
+	pool = multiprocessing.pool.Pool(processes=10)
 	ret = pool.starmap(run_domain_checks_on_domain, args, chunksize=1)
 	ret = dict(ret) # (domain, output) => { domain: output }
+	output = BufferedOutput()
 	for domain in sort_domains(ret, env):
 		ret[domain].playback(output)
+	return output
 
 def run_domain_checks_on_domain(domain, env, dns_domains, dns_zonefiles, mail_domains, web_domains):
 	output = BufferedOutput()
@@ -416,17 +399,13 @@ def check_dnssec(domain, env, output, dns_zonefiles, is_checking_primary=False):
 def check_mail_domain(domain, env, output):
 	# Check the MX record.
 
-	recommended_mx = "10 " + env['PRIMARY_HOSTNAME']
 	mx = query_dns(domain, "MX", nxdomain=None)
+	expected_mx = "10 " + env['PRIMARY_HOSTNAME']
 
-	if mx is None:
-		mxhost = None
-	else:
-		# query_dns returns a semicolon-delimited list
-		# of priority-host pairs.
-		mxhost = mx.split('; ')[0].split(' ')[1]
+	if mx == expected_mx:
+		output.print_ok("Domain's email is directed to this domain. [%s => %s]" % (domain, mx))
 
-	if mxhost == None:
+	elif mx == None:
 		# A missing MX record is okay on the primary hostname because
 		# the primary hostname's A record (the MX fallback) is... itself,
 		# which is what we want the MX to be.
@@ -444,22 +423,15 @@ def check_mail_domain(domain, env, output):
 			else:
 				output.print_error("""This domain's DNS MX record is not set. It should be '%s'. Mail will not
 					be delivered to this box. It may take several hours for public DNS to update after a
-					change. This problem may result from other issues listed here.""" % (recommended_mx,))
+					change. This problem may result from other issues listed here.""" % (expected_mx,))
 
-	elif mxhost == env['PRIMARY_HOSTNAME']:
-		good_news = "Domain's email is directed to this domain. [%s => %s]" % (domain, mx)
-		if mx != recommended_mx:
-			good_news += "  This configuration is non-standard.  The recommended configuration is '%s'." % (recommended_mx,)
-		output.print_ok(good_news)
 	else:
 		output.print_error("""This domain's DNS MX record is incorrect. It is currently set to '%s' but should be '%s'. Mail will not
 			be delivered to this box. It may take several hours for public DNS to update after a change. This problem may result from
-			other issues listed here.""" % (mx, recommended_mx))
+			other issues listed here.""" % (mx, expected_mx))
 
-	# Check that the postmaster@ email address exists. Not required if the domain has a
-	# catch-all address or domain alias.
-	if "@" + domain not in dict(get_mail_aliases(env)):
-		check_alias_exists("postmaster@" + domain, env, output)
+	# Check that the postmaster@ email address exists.
+	check_alias_exists("postmaster@" + domain, env, output)
 
 	# Stop if the domain is listed in the Spamhaus Domain Block List.
 	# The user might have chosen a domain that was previously in use by a spammer
@@ -522,7 +494,7 @@ def check_ssl_cert(domain, env, output):
 	if query_dns(domain, "A", None) not in (env['PUBLIC_IP'], None): return
 
 	# Where is the SSL stored?
-	ssl_key, ssl_certificate, ssl_via = get_domain_ssl_files(domain, env)
+	ssl_key, ssl_certificate = get_domain_ssl_files(domain, env)
 
 	if not os.path.exists(ssl_certificate):
 		output.print_error("The SSL certificate file for this domain is missing.")
@@ -534,7 +506,7 @@ def check_ssl_cert(domain, env, output):
 
 	if cert_status == "OK":
 		# The certificate is ok. The details has expiry info.
-		output.print_ok("SSL certificate is signed & valid. %s %s" % (ssl_via if ssl_via else "", cert_status_details))
+		output.print_ok("SSL certificate is signed & valid. " + cert_status_details)
 
 	elif cert_status == "SELF-SIGNED":
 		# Offer instructions for purchasing a signed certificate.
@@ -776,25 +748,21 @@ class BufferedOutput:
 		for attr, args, kwargs in self.buf:
 			getattr(output, attr)(*args, **kwargs)
 
-
 if __name__ == "__main__":
 	import sys
 	from utils import load_environment
 	env = load_environment()
 	if len(sys.argv) == 1:
-		pool = multiprocessing.pool.Pool(processes=10)
-		run_checks(env, ConsoleOutput(), pool)
+		run_checks(env, ConsoleOutput())
 	elif sys.argv[1] == "--check-primary-hostname":
 		# See if the primary hostname appears resolvable and has a signed certificate.
 		domain = env['PRIMARY_HOSTNAME']
 		if query_dns(domain, "A") != env['PUBLIC_IP']:
 			sys.exit(1)
-		ssl_key, ssl_certificate, ssl_via = get_domain_ssl_files(domain, env)
+		ssl_key, ssl_certificate = get_domain_ssl_files(domain, env)
 		if not os.path.exists(ssl_certificate):
 			sys.exit(1)
 		cert_status, cert_status_details = check_certificate(domain, ssl_certificate, ssl_key)
 		if cert_status != "OK":
 			sys.exit(1)
 		sys.exit(0)
-
-

management/templates/2fa.html

@@ -0,0 +1,79 @@
+<style>
+</style>
+
+<h2>Two-Factor Authentication</h2>
+
+<p>Two-factor authentication (2FA) is <i>something you know</i> and <i>something you have</i>.</p>
+
+<p>Regular password-based logins are one-factor (something you know). 2FA makes an account more secure by guarding against a lost or guessed password, since you also need a special device to access your account. You can turn on 2FA for your account here.</p>
+
+<p>Your authentication method is currently: <strong id="2fa_current"> </strong></p>
+
+<h3>TOTP</h3>
+
+<p>TOTP is a time-based one-time password method of two-factor authentication.</p>
+
+<p>You will need a TOTP-compatible device, such as any Android device with the <a href="https://play.google.com/store/apps/details?id=org.fedorahosted.freeotp">FreeOTP Authenticator</a> app. We&rsquo;ll generate a QR code that you import into your device or app. After you generate the QR code, you&rsquo;ll activate 2FA by entering your first activation code provided by your device or app.</p>
+
+  <p><button onclick="totp_initialize()">Generate QR Code</button></p>
+  <div id="totp-form" class="row" style="display: none">
+    <div class="col-sm-6">
+      <center>QR Code</center>
+      <img id="totp_qr_code" src="" class="img-responsive">
+    </div>
+    <div class="col-sm-6">
+      <form class="form" role="form" onsubmit="totp_activate(); return false;">
+        <div class="form-group">
+          <label for="inputTOTP" class="control-label">Activation Code</label>
+          <p><input class="form-control" id="inputTOTP" placeholder="enter 6-digit code"></p>
+          <p><input type="submit" class="btn btn-primary" value="Activate"></p>
+        </div>
+      </form>
+    </div>
+  </div>
+
+<p>When using TOTP 2FA, your password becomes your previous plain password plus a space plus the code generated by your TOTP device.</p>
+
+<script>
+function show_2fa() {
+ $('#2fa_current').text('loading...');
+ api(
+    "/me/2fa",
+    "GET",
+    {
+    },
+    function(response) {
+      $('#2fa_current').text(response.method);
+    });
+}
+
+var secret = null;
+
+function totp_initialize() {
+ api(
+    "/me/2fa/totp/initialize",
+    "POST",
+    {
+    },
+    function(response) {
+      $('#totp_qr_code').attr('src', 'data:image/png;base64,' + response.qr);
+      $('#totp-form').fadeIn();
+      secret = response.secret;
+    });
+}
+
+function totp_activate() {
+ api(
+    "/me/2fa/totp/activate",
+    "POST",
+    {
+      "secret": secret,
+      "code": $('#inputTOTP').val()
+    },
+    function(response) {
+      show_modal_error("Two-Factor Authentication", $("<pre/>").text(response.message));
+      if (response.status == "OK")
+        $('#totp-form').fadeOut();
+    });
+}
+</script>

management/templates/index.html

@@ -115,6 +115,12 @@
             </li>
             <li><a href="#sync_guide" onclick="return show_panel(this);">Contacts/Calendar</a></li>
             <li><a href="#web" onclick="return show_panel(this);">Web</a></li>
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown">You <b class="caret"></b></a>
+              <ul class="dropdown-menu">
+                <li><a href="#2fa" onclick="return show_panel(this);">Two-Factor Authentication</a></li>
+              </ul>
+            </li>
           </ul>
           <ul class="nav navbar-nav navbar-right">
             <li><a href="#" onclick="do_logout(); return false;" style="color: white">Log out?</a></li>
@@ -168,6 +174,10 @@
       {% include "ssl.html" %}
       </div>
 
+      <div id="panel_2fa" class="admin_panel">
+      {% include "2fa.html" %}
+      </div>
+
       <hr>
 
       <footer>

management/templates/mail-guide.html

@@ -40,7 +40,7 @@
 
 				<h4>Exchange/ActiveSync settings</h4>
 
-					<p>On iOS devices, devices on this <a href="http://z-push.org/compatibility/">compatibility list</a>, or using Outlook 2007 or later on Windows 7 and later, you may set up your mail as an Exchange or ActiveSync server. However, we&rsquo;ve found this to be more buggy than using IMAP as described above. If you encounter any problems, please use the manual settings above.</p>
+					<p>On iOS devices and devices on this <a href="http://z-push.org/compatibility/">compatibility list</a>, you may set up your mail as an Exchange or ActiveSync server. However, we&rsquo;ve found this to be more buggy than using IMAP. If you encounter any problems, please use the manual settings above.</p>
 
 					<table class="table">
 					<tr><th>Server</th> <td>{{hostname}}</td></tr>

management/templates/web.html

@@ -7,7 +7,7 @@
 
 <h3>Uploading web files</h3>
 
-<p>You can replace the default website with your own HTML pages and other static files. This control panel won&rsquo;t help you design a website, but once you have <tt>.html</tt> files you can upload them following these instructions:</p>
+<p>You can replace the default website with your own HTML pages and other static files. This control panel won&rsquo;t help you design a website, but once you have <tt>.html</tt> files you can upload it following these instructions:</p>
 
 <ol>
 <li>Ensure that any domains you are publishing a website for have no problems on the <a href="#system_status" onclick="return show_panel(this);">Status Checks</a> page.</li>
@@ -18,24 +18,41 @@
 
 <li>Upload your <tt>.html</tt> or other files to the directory <tt>{{storage_root}}/www/default</tt> on this machine. They will appear directly and immediately on the web.</li>
 
-<li>The websites set up on this machine are listed in the table below with where to put the files for each website.</li>
+<li>The websites set up on this machine are listed in the table below with where to put the files for each website (if you have customized that, see next section).</li>
 
-<table id="web_domains_existing" class="table" style="margin-bottom: 1em; width: auto;">
+<table id="web_domains_existing" class="table" style="margin-bottom: 2em; width: auto;">
         <thead>
                 <tr>
                         <th>Site</th>
                         <th>Directory for Files</th>
-			<th/>
                 </tr>
         </thead>
         <tbody>
         </tbody>
 </table>
 
-<p>To add a domain to this table, create a dummy <a href="#users" onclick="return show_panel(this);">mail user</a> or <a href="#aliases" onclick="return show_panel(this);">alias</a> on the domain first and see the <a href="https://mailinabox.email/guide.html#domain-name-configuration">setup guide</a> for adding nameserver records to the new domain at your registrar (but <i>not</i> glue records).</p>
+<li>If you want to have this box host a static website on a domain that is not listed in the table, create a dummy <a href="#users" onclick="return show_panel(this);">mail user</a> or <a href="#aliases" onclick="return show_panel(this);">alias</a> on the domain first.</li>
 
 </ol>
 
+<h3>Different sites for different domains</h3>
+
+<p>Create one of the directories shown in the table below to create a space for different files for one of the websites.</p>
+
+<p>After you create one of these directories, click <button id="web_update" class="btn btn-primary" onclick="do_web_update()">Web Update</button> to restart the box&rsquo;s web server so that it sees the new website file location.</p>
+
+<table id="web_domains_custom" class="table" style="margin-bottom: 2em; width: auto;">
+        <thead>
+                <tr>
+                        <th>Site</th>
+                        <th>Create Directory</th>
+                </tr>
+        </thead>
+        <tbody>
+        </tbody>
+</table>
+
+
 <script>
 function show_web() {
  api(
@@ -47,18 +64,24 @@ function show_web() {
 	var tb = $('#web_domains_existing tbody');
 	tb.text('');
 	for (var i = 0; i < domains.length; i++) {
-		if (!domains[i].static_enabled) continue;
-		var row = $("<tr><th class='domain'><a href=''></a></th><td class='directory'><tt/></td> <td class='change-root hidden'><button class='btn btn-default btn-xs' onclick='show_change_web_root(this)'>Change</button></td></tr>");
+		var row = $("<tr><th class='domain'><a href=''></a></th><td class='directory'><tt/></td></tr>");
 		tb.append(row);
-		row.attr('data-domain', domains[i].domain);
-		row.attr('data-custom-web-root', domains[i].custom_root);
 		row.find('.domain a').text('https://' + domains[i].domain);
 		row.find('.domain a').attr('href', 'https://' + domains[i].domain);
 		row.find('.directory tt').text(domains[i].root);
-		if (domains[i].root != domains[i].custom_root)
-			row.find('.change-root').removeClass('hidden');
 	}
 
+	tb = $('#web_domains_custom tbody');
+	tb.text('');
+	for (var i = 0; i < domains.length; i++) {
+		if (domains[i].root != domains[i].custom_root) {
+			var row = $("<tr><th class='domain'><a href=''></a></th><td class='directory'><tt></td></tr>");
+			tb.append(row);
+			row.find('.domain a').text('https://' + domains[i].domain);
+			row.find('.domain a').attr('href', 'https://' + domains[i].domain);
+			row.find('.directory tt').text(domains[i].custom_root);
+		}
+	}
     });
 }
 
@@ -76,14 +99,4 @@ function do_web_update() {
       show_modal_error("Web Update", data, function() { show_web() });
     });
 }
-
-function show_change_web_root(elem) {
-  var domain = $(elem).parents('tr').attr('data-domain');
-  var root = $(elem).parents('tr').attr('data-custom-web-root');
-  show_modal_confirm(
-    'Change Root Directory for ' + domain,
-    '<p>You can change the static directory for <tt>' + domain + '</tt> to:</p> <p><tt>' + root + '</tt></p> <p>First create this directory on the server. Then click Update to scan for the directory and update web settings.',
-    'Update',
-    function() { do_web_update(); });
-}
 </script>

management/web_update.py

@@ -17,8 +17,9 @@ def get_web_domains(env):
 	domains.add(env['PRIMARY_HOSTNAME'])
 
 	# Also serve web for all mail domains so that we might at least
-	# provide auto-discover of email settings, and also a static website
-	# if the user wants to make one. These will require an SSL cert.
+	# provide Webfinger and ActiveSync auto-discover of email settings
+	# (though the latter isn't really working). These will require that
+	# an SSL cert be installed.
 	domains |= get_mail_domains(env)
 
 	# ...Unless the domain has an A/AAAA record that maps it to a different
@@ -27,7 +28,6 @@ def get_web_domains(env):
 	for domain, value in dns.items():
 		if domain not in domains: continue
 		if (isinstance(value, str) and (value != "local")) \
-		  or (isinstance(value, dict) and ("CNAME" in value)) \
 		  or (isinstance(value, dict) and ("A" in value) and (value["A"] != "local")) \
 		  or (isinstance(value, dict) and ("AAAA" in value) and (value["AAAA"] != "local")):
 			domains.remove(domain)
@@ -75,7 +75,7 @@ def make_domain_config(domain, template, template_for_primaryhost, env):
 	root = get_web_root(domain, env)
 
 	# What private key and SSL certificate will we use for this domain?
-	ssl_key, ssl_certificate, ssl_via = get_domain_ssl_files(domain, env)
+	ssl_key, ssl_certificate = get_domain_ssl_files(domain, env)
 
 	# For hostnames created after the initial setup, ensure we have an SSL certificate
 	# available. Make a self-signed one now if one doesn't exist.
@@ -149,7 +149,6 @@ def get_domain_ssl_files(domain, env, allow_shared_cert=True):
 
 	# What SSL certificate will we use?
 	ssl_certificate_primary = os.path.join(env["STORAGE_ROOT"], 'ssl/ssl_certificate.pem')
-	ssl_via = None
 	if domain == env['PRIMARY_HOSTNAME']:
 		# For PRIMARY_HOSTNAME, use the one we generated at set-up time.
 		ssl_certificate = ssl_certificate_primary
@@ -164,16 +163,8 @@ def get_domain_ssl_files(domain, env, allow_shared_cert=True):
 			from status_checks import check_certificate
 			if check_certificate(domain, ssl_certificate_primary, None)[0] == "OK":
 				ssl_certificate = ssl_certificate_primary
-				ssl_via = "Using multi/wildcard certificate of %s." % env['PRIMARY_HOSTNAME']
 
-			# For a 'www.' domain, see if we can reuse the cert of the parent.
-			elif domain.startswith('www.'):
-				ssl_certificate_parent = os.path.join(env["STORAGE_ROOT"], 'ssl/%s/ssl_certificate.pem' % safe_domain_name(domain[4:]))
-				if os.path.exists(ssl_certificate_parent) and check_certificate(domain, ssl_certificate_parent, None)[0] == "OK":
-					ssl_certificate = ssl_certificate_parent
-					ssl_via = "Using multi/wildcard certificate of %s." % domain[4:]
-
-	return ssl_key, ssl_certificate, ssl_via
+	return ssl_key, ssl_certificate
 
 def ensure_ssl_certificate_exists(domain, ssl_key, ssl_certificate, env):
 	# For domains besides PRIMARY_HOSTNAME, generate a self-signed certificate if
@@ -228,7 +219,7 @@ def install_cert(domain, ssl_cert, ssl_chain, env):
 
 	# Do validation on the certificate before installing it.
 	from status_checks import check_certificate
-	ssl_key, ssl_certificate, ssl_via = get_domain_ssl_files(domain, env, allow_shared_cert=False)
+	ssl_key, ssl_certificate = get_domain_ssl_files(domain, env, allow_shared_cert=False)
 	cert_status, cert_status_details = check_certificate(domain, fn, ssl_key)
 	if cert_status != "OK":
 		if cert_status == "SELF-SIGNED":
@@ -259,28 +250,18 @@ def install_cert(domain, ssl_cert, ssl_chain, env):
 	return "\n".join(r for r in ret if r.strip() != "")
 
 def get_web_domains_info(env):
-	# load custom settings so we can tell what domains have a redirect or proxy set up on '/',
-	# which means static hosting is not happening
-	custom_settings = { }
-	nginx_conf_custom_fn = os.path.join(env["STORAGE_ROOT"], "www/custom.yaml")
-	if os.path.exists(nginx_conf_custom_fn):
-		custom_settings = rtyaml.load(open(nginx_conf_custom_fn))
-	def has_root_proxy_or_redirect(domain):
-		return custom_settings.get(domain, {}).get('redirects', {}).get('/') or custom_settings.get(domain, {}).get('proxies', {}).get('/')
-
-	# for the SSL config panel, get cert status
 	def check_cert(domain):
 		from status_checks import check_certificate
-		ssl_key, ssl_certificate, ssl_via = get_domain_ssl_files(domain, env)
+		ssl_key, ssl_certificate = get_domain_ssl_files(domain, env)
 		if not os.path.exists(ssl_certificate):
 			return ("danger", "No Certificate Installed")
 		cert_status, cert_status_details = check_certificate(domain, ssl_certificate, ssl_key)
 		if cert_status == "OK":
-			if not ssl_via:
+			if domain == env['PRIMARY_HOSTNAME'] or ssl_certificate != get_domain_ssl_files(env['PRIMARY_HOSTNAME'], env)[1]:
 				return ("success", "Signed & valid. " + cert_status_details)
 			else:
 				# This is an alternate domain but using the same cert as the primary domain.
-				return ("success", "Signed & valid. " + ssl_via)
+				return ("success", "Signed & valid. Using multi/wildcard certificate of %s." % env['PRIMARY_HOSTNAME'])
 		elif cert_status == "SELF-SIGNED":
 			return ("warning", "Self-signed. Get a signed certificate to stop warnings.")
 		else:
@@ -292,7 +273,6 @@ def get_web_domains_info(env):
 			"root": get_web_root(domain, env),
 			"custom_root": get_web_root(domain, env, test_exists=False),
 			"ssl_certificate": check_cert(domain),
-			"static_enabled": not has_root_proxy_or_redirect(domain),
 		}
 		for domain in get_web_domains(env)
 	]

setup/bootstrap.sh

@@ -7,7 +7,7 @@
 #########################################################
 
 if [ -z "$TAG" ]; then
-	TAG=v0.07
+	TAG=v0.06
 fi
 
 # Are we running as root?

setup/dkim.sh

@@ -10,7 +10,7 @@ source setup/functions.sh # load our functions
 source /etc/mailinabox.conf # load global vars
 
 # Install DKIM...
-apt_install opendkim opendkim-tools opendmarc
+apt_install opendkim opendkim-tools
 
 # Make sure configuration directories exist.
 mkdir -p /etc/opendkim;
@@ -48,25 +48,15 @@ fi
 chown -R opendkim:opendkim $STORAGE_ROOT/mail/dkim
 chmod go-rwx $STORAGE_ROOT/mail/dkim
 
-tools/editconf.py /etc/opendmarc.conf -s \
-	"Syslog=true" \
-	"Socket=inet:8893@[127.0.0.1]"
-
-# Add OpenDKIM and OpenDMARC as milters to postfix, which is how OpenDKIM
-# intercepts outgoing mail to perform the signing (by adding a mail header)
-# and how they both intercept incoming mail to add Authentication-Results
-# headers. The order possibly/probably matters: OpenDMARC relies on the
-# OpenDKIM Authentication-Results header already being present.
-#
-# Be careful. If we add other milters later, this needs to be concatenated
-# on the smtpd_milters line.
+# Add OpenDKIM as a milter to postfix, which is how it intercepts outgoing
+# mail to perform the signing (by adding a mail header).
+# Be careful. If we add other milters later, it needs to be concatenated on the smtpd_milters line. #NODOC
 tools/editconf.py /etc/postfix/main.cf \
-	"smtpd_milters=inet:127.0.0.1:8891 inet:127.0.0.1:8893"\
+	smtpd_milters=inet:127.0.0.1:8891 \
 	non_smtpd_milters=\$smtpd_milters \
 	milter_default_action=accept
 
 # Restart services.
 restart_service opendkim
-restart_service opendmarc
 restart_service postfix
 

setup/functions.sh

@@ -22,20 +22,6 @@ function hide_output {
 	rm -f $OUTPUT
 }
 
-function apt_get_quiet {
-	# Run apt-get in a totally non-interactive mode.
-	#
-	# Somehow all of these options are needed to get it to not ask the user
-	# questions about a) whether to proceed (-y), b) package options (noninteractive),
-	# and c) what to do about files changed locally (we don't cause that to happen but
-	# some VM providers muck with their images; -o).
-	#
-	# Although we could pass -qq to apt-get to make output quieter, many packages write to stdout
-	# and stderr things that aren't really important. Use our hide_output function to capture
-	# all of that and only show it if there is a problem (i.e. if apt_get returns a failure exit status).
-	DEBIAN_FRONTEND=noninteractive hide_output apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confnew" "$@"
-}
-
 function apt_install {
 	# Report any packages already installed.
 	PACKAGES=$@
@@ -60,10 +46,18 @@ function apt_install {
 		echo installing $TO_INSTALL...
 	fi
 
-	# We still include the whole original package list in the apt-get command in
+	# 'DEBIAN_FRONTEND=noninteractive' is to prevent dbconfig-common from asking you questions.
+	#
+	# Although we could pass -qq to apt-get to make output quieter, many packages write to stdout
+	# and stderr things that aren't really important. Use our hide_output function to capture
+	# all of that and only show it if there is a problem (i.e. if apt_get returns a failure exit status).
+	#
+	# Also note that we still include the whole original package list in the apt-get command in
 	# case it wants to upgrade anything, I guess? Maybe we can remove it. Doesn't normally make
 	# a difference.
-	apt_get_quiet install $PACKAGES
+	DEBIAN_FRONTEND=noninteractive \
+	hide_output \
+	apt-get -y install $PACKAGES
 }
 
 function get_default_hostname {

setup/mail-users.sh

@@ -26,31 +26,55 @@ fi
 
 # ### User Authentication
 
-# Have Dovecot query our database, and not system users, for authentication.
-sed -i "s/#*\(\!include auth-system.conf.ext\)/#\1/"  /etc/dovecot/conf.d/10-auth.conf
-sed -i "s/#\(\!include auth-sql.conf.ext\)/\1/"  /etc/dovecot/conf.d/10-auth.conf
+# Disable all of the built-in authentication mechanisms. (We formerly uncommented
+# a line to include auth-sql.conf.ext but we no longer use that.)
+sed -i "s/#*\(\!include auth-.*.conf.ext\)/#\1/"  /etc/dovecot/conf.d/10-auth.conf
 
-# Specify how the database is to be queried for user authentication (passdb)
-# and where user mailboxes are stored (userdb).
-cat > /etc/dovecot/conf.d/auth-sql.conf.ext << EOF;
+# Legacy: Delete our old sql conf files.
+rm -f /etc/dovecot/conf.d/auth-sql.conf.ext /etc/dovecot/dovecot-sql.conf.ext
+
+# Specify how Dovecot should perform user authentication (passdb) and how it knows
+# where user mailboxes are stored (userdb).
+#
+# For passwords, we would normally have Dovecot query our mail user database
+# directly. The way to do that is commented out below. Instead, in order to
+# provide our own authentication framework so we can handle two-factor auth,
+# we will use a custom system that hooks into the Mail-in-a-Box management daemon.
+#
+# The user part of this is standard. The mailbox path and Unix system user are the
+# same for all mail users, modulo string substitution for the mailbox path that
+# Dovecot handles.
+cat > /etc/dovecot/conf.d/10-auth-mailinabox.conf << EOF;
 passdb {
-  driver = sql
-  args = /etc/dovecot/dovecot-sql.conf.ext
+  driver = checkpassword
+  args = /usr/local/bin/dovecot-checkpassword
 }
 userdb {
   driver = static
   args = uid=mail gid=mail home=$STORAGE_ROOT/mail/mailboxes/%d/%n
 }
 EOF
+chmod 0600 /etc/dovecot/conf.d/10-auth-mailinabox.conf
 
-# Configure the SQL to query for a user's password.
-cat > /etc/dovecot/dovecot-sql.conf.ext << EOF;
-driver = sqlite
-connect = $db_path
-default_pass_scheme = SHA512-CRYPT
-password_query = SELECT email as user, password FROM users WHERE email='%u';
-EOF
-chmod 0600 /etc/dovecot/dovecot-sql.conf.ext # per Dovecot instructions
+# Copy dovecot-checkpassword into place.
+cp conf/dovecot-checkpassword.py /usr/local/bin/dovecot-checkpassword
+chown dovecot.dovecot /usr/local/bin/dovecot-checkpassword
+chmod 700 /usr/local/bin/dovecot-checkpassword
+
+# If we were having Dovecot query our database directly, which we did
+# originally, `/etc/dovecot/conf.d/10-auth-mailinabox.conf` would say:
+#
+#     passdb {
+#       driver = sql
+#       args = /etc/dovecot/dovecot-sql.conf.ext
+#     }
+#
+# and then `/etc/dovecot/dovecot-sql.conf.ext` (chmod 0600) would contain:
+#
+#    driver = sqlite
+#    connect = $db_path
+#    default_pass_scheme = SHA512-CRYPT
+#    password_query = SELECT email as user, password FROM users WHERE email='%u';
 
 # Have Dovecot provide an authorization service that Postfix can access & use.
 cat > /etc/dovecot/conf.d/99-local-auth.conf << EOF;

setup/management.sh

@@ -3,7 +3,10 @@
 source setup/functions.sh
 
 apt_install python3-flask links duplicity libyaml-dev python3-dnspython python3-dateutil
-hide_output pip3 install rtyaml
+hide_output pip3 install rtyaml 
+
+# For two-factor authentication, the management server uses:
+hide_output pip3 install git+https://github.com/mail-in-a-box/python-oath qrcode pillow
 
 # Create a backup directory and a random key for encrypting backups.
 mkdir -p $STORAGE_ROOT/backup
@@ -30,8 +33,5 @@ $(pwd)/management/backup.py
 EOF
 chmod +x /etc/cron.daily/mailinabox-backup
 
-# Start it. Remove the api key file first so that start.sh
-# can wait for it to be created to know that the management
-# server is ready.
-rm -f /var/lib/mailinabox/api.key
+# Start it.
 restart_service mailinabox

setup/network-checks.sh

@@ -1,6 +1,6 @@
 # Install the 'host', 'sed', and and 'nc' tools. This script is run before
 # the rest of the system setup so we may not yet have things installed.
-apt_get_quiet install bind9-host sed netcat-openbsd
+hide_output apt-get -y install bind9-host sed netcat-openbsd
 
 # Stop if the PRIMARY_HOSTNAME is listed in the Spamhaus Domain Block List.
 # The user might have chosen a name that was previously in use by a spammer

setup/questions.sh

@@ -4,7 +4,7 @@ if [ -z "$NONINTERACTIVE" ]; then
 	# e.g. if we piped a bootstrapping install script to bash to get started. In that
 	# case, the nifty '[ -t 0 ]' test won't work. But with Vagrant we must suppress so we
 	# use a shell flag instead. Really supress any output from installing dialog.
-	apt_get_quiet install dialog
+	hide_output apt-get -y install dialog
 	message_box "Mail-in-a-Box Installation" \
 		"Hello and thanks for deploying a Mail-in-a-Box!
 		\n\nI'm going to ask you a few questions.

setup/start.sh

@@ -87,33 +87,17 @@ if [ -z "$SKIP_NETWORK_CHECKS" ]; then
 	. setup/network-checks.sh
 fi
 
-# For the first time (if the config file (/etc/mailinabox.conf) not exists):
 # Create the user named "user-data" and store all persistent user
 # data (mailboxes, etc.) in that user's home directory.
-#
-# If the config file exists:
-# Apply the existing configuration options for STORAGE_USER/ROOT
-if [ -z "$STORAGE_USER" ]; then
-	STORAGE_USER=$([[ -z "$DEFAULT_STORAGE_USER" ]] && echo "user-data" || echo "$DEFAULT_STORAGE_USER")
-fi
-
 if [ -z "$STORAGE_ROOT" ]; then
-	STORAGE_ROOT=$([[ -z "$DEFAULT_STORAGE_ROOT" ]] && echo "/home/$STORAGE_USER" || echo "$DEFAULT_STORAGE_ROOT")
-fi
-
-# Create the STORAGE_USER if it not exists
-if ! id -u $STORAGE_USER >/dev/null 2>&1; then
-	useradd -m $STORAGE_USER
-fi
-
-# Create the STORAGE_ROOT if it not exists
-if [ ! -d $STORAGE_ROOT ]; then
+	STORAGE_USER=user-data
+	if [ ! -d /home/$STORAGE_USER ]; then useradd -m $STORAGE_USER; fi
+	STORAGE_ROOT=/home/$STORAGE_USER
 	mkdir -p $STORAGE_ROOT
 	echo $(setup/migrate.py --current) > $STORAGE_ROOT/mailinabox.version
 	chown $STORAGE_USER.$STORAGE_USER $STORAGE_ROOT/mailinabox.version
 fi
 
-
 # Save the global options in /etc/mailinabox.conf so that standalone
 # tools know where to look for data.
 cat > /etc/mailinabox.conf << EOF;
@@ -142,13 +126,10 @@ source setup/owncloud.sh
 source setup/zpush.sh
 source setup/management.sh
 
-# Ping the management daemon to write the DNS and nginx configuration files.
-while [ ! -f /var/lib/mailinabox/api.key ]; do
-	echo Waiting for the Mail-in-a-Box management daemon to start...
-	sleep 2
-done
-tools/dns_update
-tools/web_update
+# Write the DNS and nginx configuration files.
+sleep 5 # wait for the daemon to start
+curl -s -d POSTDATA --user $(</var/lib/mailinabox/api.key): http://127.0.0.1:10222/dns/update
+curl -s -d POSTDATA --user $(</var/lib/mailinabox/api.key): http://127.0.0.1:10222/web/update
 
 # If there aren't any mail users yet, create one.
 source setup/firstuser.sh

setup/system.sh

@@ -9,7 +9,7 @@ source setup/functions.sh # load our functions
 
 echo Updating system packages...
 hide_output apt-get update
-apt_get_quiet upgrade
+hide_output apt-get -y upgrade
 
 # Install basic utilities.
 #

setup/web.sh

@@ -53,16 +53,6 @@ cat conf/ios-profile.xml \
 	 > /var/lib/mailinabox/mobileconfig.xml
 chmod a+r /var/lib/mailinabox/mobileconfig.xml
 
-# Create the Mozilla Auto-configuration file which is exposed via the
-# nginx configuration at /.well-known/autoconfig/mail/config-v1.1.xml.
-# The format of the file is documented at:
-# https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat
-# and https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration/FileFormat/HowTo.
-cat conf/mozilla-autoconfig.xml \
-	| sed "s/PRIMARY_HOSTNAME/$PRIMARY_HOSTNAME/" \
-	 > /var/lib/mailinabox/mozilla-autoconfig.xml
-chmod a+r /var/lib/mailinabox/mozilla-autoconfig.xml
-
 # make a default homepage
 if [ -d $STORAGE_ROOT/www/static ]; then mv $STORAGE_ROOT/www/static $STORAGE_ROOT/www/default; fi # migration #NODOC
 mkdir -p $STORAGE_ROOT/www/default

setup/webmail.sh

@@ -30,7 +30,7 @@ apt_install \
 apt-get purge -qq -y roundcube* #NODOC
 
 # Install Roundcube from source if it is not already present or if it is out of date.
-VERSION=1.1.0
+VERSION=1.0.3
 needs_update=0 #NODOC
 if [ ! -f /usr/local/lib/roundcubemail/version ]; then
 	# not installed yet #NODOC
@@ -40,7 +40,7 @@ elif [[ $VERSION != `cat /usr/local/lib/roundcubemail/version` ]]; then
 	needs_update=1 #NODOC
 fi
 if [ $needs_update == 1 ]; then
-	echo installing roundcube webmail $VERSION...
+	echo installing roudcube webmail $VERSION...
 	rm -f /tmp/roundcube.tgz
 	wget -qO /tmp/roundcube.tgz http://downloads.sourceforge.net/project/roundcubemail/roundcubemail/$VERSION/roundcubemail-$VERSION.tar.gz
 	tar -C /usr/local/lib -zxf /tmp/roundcube.tgz