release v0.06

partialy reverts 93a722f; closes #287

CHANGELOG.md

@@ -1,8 +1,61 @@
 CHANGELOG
 =========
 
-To-be-released
+v0.06 (January 4, 2015)
---------------
+-----------------------
+
+Mail:
+
+* Set better default system limits to accommodate boxes handling mail for 20+ users.
+
+Contacts/calendar:
+
+* Update to ownCloud to 7.0.4.
+* Contacts syncing via ActiveSync wasn't working.
+
+Control panel:
+
+* New control panel for setting custom DNS settings (without having to use the API).
+* Status checks showed a false positive for Spamhause blacklists and for secondary DNS in some cases.
+* Status checks would fail to load if openssh-sever was not pre-installed, but openssh-server is not required.
+* The local DNS cache is cleared before running the status checks using 'rncd' now rather than restarting 'bind9', which should be faster and wont interrupt other services.
+* Multi-domain and wildcard certificate can now be installed through the control panel.
+* The DNS API now allows the setting of SRV records.
+
+Misc:
+
+* IPv6 configuration error in postgrey, nginx.
+* Missing dependency on sudo.
+
+v0.05 (November 18, 2014)
+-------------------------
+
+Mail:
+
+* The maximum size of outbound mail sent via webmail and Exchange/ActiveSync has been increased to 128 MB, the same as when using SMTP.
+* Spam is no longer wrapped as an attachment inside a scary Spamassassin explanation. The original message is simply moved straight to the Spam folder unchanged.
+* There is a new iOS/Mac OS X Configuration Profile link in the control panel which makes it easier to configure IMAP/SMTP/CalDAV/CardDAV on iOS devices and Macs.
+* "Domain aliases" can now be configured in the control panel.
+* Updated to [Roundcube 1.0.3](http://trac.roundcube.net/wiki/Changelog).
+* IMAP/SMTP is now recommended even on iOS devices as Exchange/ActiveSync is terribly buggy.
+
+Control panel:
+
+* Installing an SSL certificate for the primary hostname would cause problems until a restart (services needed to be restarted).
+* Installing SSL certificates would fail if /tmp was on a different filesystem.
+* Better error messages when installing a SSL certificate fails.
+* The local DNS cache is now cleared each time the system status checks are run.
+* Documented how to use +tag addressing.
+* Minor UI tweaks.
+
+Other:
+
+* Updated to [ownCloud 7.0.3](http://owncloud.org/changelog/).
+* The ownCloud API is now exposed properly.
+* DNSSEC now works on `.guide` domains now too (RSASHA256).
+
+v0.04 (October 15, 2014)
+------------------------
 
 Breaking changes:
 
@@ -31,6 +84,7 @@ Security:
 
 Other:
 
+* Spam filter learning by dragging mail in and out of the Spam folder should hopefully be working now.
 * Some things were broken if the machine had an IPv6 address.
 * Other things were broken if the machine was on a non-utf8 locale.
 * No longer implementing webfinger.

conf/ios-profile.xml

@@ -0,0 +1,128 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<!--
+ iOS/OS X Configuration Profile
+
+ Mobileconfig for iOS/OS X users to setup IMAP, SMTP, Contacts & Calendar
+
+ https://developer.apple.com/library/ios/featuredarticles/iPhoneConfigurationProfileRef/Introduction/Introduction.html
+-->
+<plist version="1.0">
+<dict>
+  <key>PayloadContent</key>
+  <array>
+    <dict>
+      <key>CalDAVAccountDescription</key>
+      <string>PRIMARY_HOSTNAME calendar</string>
+      <key>CalDAVHostName</key>
+      <string>PRIMARY_HOSTNAME</string>
+      <key>CalDAVPort</key>
+      <real>443</real>
+      <key>CalDAVPrincipalURL</key>
+      <string>/cloud/remote.php/caldav/calendars/</string>
+      <key>CalDAVUseSSL</key>
+      <true/>
+      <key>PayloadDescription</key>
+      <string>PRIMARY_HOSTNAME (Mail-in-a-Box)</string>
+      <key>PayloadDisplayName</key>
+      <string>PRIMARY_HOSTNAME calendar</string>
+      <key>PayloadIdentifier</key>
+      <string>email.mailinabox.mobileconfig.PRIMARY_HOSTNAME.CalDAV</string>
+      <key>PayloadOrganization</key>
+      <string></string>
+      <key>PayloadType</key>
+      <string>com.apple.caldav.account</string>
+      <key>PayloadUUID</key>
+      <string>UUID1</string>
+      <key>PayloadVersion</key>
+      <integer>1</integer>
+    </dict>
+    <dict>
+      <key>EmailAccountDescription</key>
+      <string>PRIMARY_HOSTNAME mail</string>
+      <key>EmailAccountType</key>
+      <string>EmailTypeIMAP</string>
+      <key>IncomingMailServerAuthentication</key>
+      <string>EmailAuthPassword</string>
+      <key>IncomingMailServerHostName</key>
+      <string>PRIMARY_HOSTNAME</string>
+      <key>IncomingMailServerPortNumber</key>
+      <integer>993</integer>
+      <key>IncomingMailServerUseSSL</key>
+      <true/>
+      <key>OutgoingMailServerAuthentication</key>
+      <string>EmailAuthPassword</string>
+      <key>OutgoingMailServerHostName</key>
+      <string>PRIMARY_HOSTNAME</string>
+      <key>OutgoingMailServerPortNumber</key>
+      <integer>587</integer>
+      <key>OutgoingMailServerUseSSL</key>
+      <true/>
+      <key>OutgoingPasswordSameAsIncomingPassword</key>
+      <true/>
+      <key>PayloadDescription</key>
+      <string>PRIMARY_HOSTNAME (Mail-in-a-Box)</string>
+      <key>PayloadDisplayName</key>
+      <string>PRIMARY_HOSTNAME mail</string>
+      <key>PayloadIdentifier</key>
+      <string>email.mailinabox.mobileconfig.PRIMARY_HOSTNAME.E-Mail</string>
+      <key>PayloadOrganization</key>
+      <string></string>
+      <key>PayloadType</key>
+      <string>com.apple.mail.managed</string>
+      <key>PayloadUUID</key>
+      <string>UUID2</string>
+      <key>PayloadVersion</key>
+      <integer>1</integer>
+      <key>PreventAppSheet</key>
+      <false/>
+      <key>PreventMove</key>
+      <false/>
+      <key>SMIMEEnabled</key>
+      <false/>
+    </dict>
+    <dict>
+      <key>CardDAVAccountDescription</key>
+      <string>PRIMARY_HOSTNAME contacts</string>
+      <key>CardDAVHostName</key>
+      <string>PRIMARY_HOSTNAME</string>
+      <key>CardDAVPort</key>
+      <integer>443</integer>
+      <key>CardDAVPrincipalURL</key>
+      <string>/cloud/remote.php/carddav/addressbooks/</string>
+      <key>CardDAVUseSSL</key>
+      <true/>
+      <key>PayloadDescription</key>
+      <string>PRIMARY_HOSTNAME (Mail-in-a-Box)</string>
+      <key>PayloadDisplayName</key>
+      <string>PRIMARY_HOSTNAME contacts</string>
+      <key>PayloadIdentifier</key>
+      <string>email.mailinabox.mobileconfig.PRIMARY_HOSTNAME.carddav</string>
+      <key>PayloadOrganization</key>
+      <string></string>
+      <key>PayloadType</key>
+      <string>com.apple.carddav.account</string>
+      <key>PayloadUUID</key>
+      <string>UUID3</string>
+      <key>PayloadVersion</key>
+      <integer>1</integer>
+    </dict>
+  </array>
+  <key>PayloadDescription</key>
+  <string>PRIMARY_HOSTNAME (Mail-in-a-Box)</string>
+  <key>PayloadDisplayName</key>
+  <string>PRIMARY_HOSTNAME</string>
+  <key>PayloadIdentifier</key>
+  <string>email.mailinabox.mobileconfig.PRIMARY_HOSTNAME</string>
+  <key>PayloadOrganization</key>
+  <string></string>
+  <key>PayloadRemovalDisallowed</key>
+  <false/>
+  <key>PayloadType</key>
+  <string>Configuration</string>
+  <key>PayloadUUID</key>
+  <string>UUID4</string>
+  <key>PayloadVersion</key>
+  <integer>1</integer>
+</dict>
+</plist>

conf/nginx-primaryonly.conf

@@ -18,8 +18,12 @@
 			deny all;
 	 	}
 	}
-	location ~ ^(/cloud)(/[^/]+\.php)(/.*)?$ {
+	location ~ ^(/cloud)((?:/ocs)?/[^/]+\.php)(/.*)?$ {
 		# note: ~ has precendence over a regular location block
+		# Accept URLs like:
+		# /cloud/index.php/apps/files/
+		# /cloud/index.php/apps/files/ajax/scan.php (it's really index.php; see 6fdef379adfdeac86cc2220209bdf4eb9562268d)
+		# /cloud/ocs/v1.php/apps/files_sharing/api/v1 (see #240)
 		include fastcgi_params;
 		fastcgi_param SCRIPT_FILENAME /usr/local/lib/owncloud/$2;
 		fastcgi_param SCRIPT_NAME $1$2;

conf/nginx.conf

@@ -13,6 +13,7 @@ server {
 # The secure HTTPS server.
 server {
 	listen 443 ssl;
+	listen [::]:443 ssl;
 
 	server_name $HOSTNAME;
 
@@ -34,6 +35,10 @@ server {
 		access_log off;
 	}
 
+	location = /mailinabox.mobileconfig {
+		alias /var/lib/mailinabox/mobileconfig.xml;
+	}
+
 	# Roundcube Webmail configuration.
 	rewrite ^/mail$ /mail/ redirect;
 	rewrite ^/mail/$ /mail/index.php;
@@ -52,7 +57,10 @@ server {
 		fastcgi_index index.php;
 		fastcgi_param SCRIPT_FILENAME /usr/local/lib/roundcubemail/$fastcgi_script_name;
 		fastcgi_pass php-fpm;
-		client_max_body_size 20M;
+
+		# Outgoing mail also goes through this endpoint, so increase the maximum
+		# file upload limit to match the corresponding Postfix limit.
+		client_max_body_size 128M;
 	}
 
 	# Z-Push (Microsoft Exchange ActiveSync)
@@ -62,6 +70,10 @@ server {
 		fastcgi_param PHP_VALUE "include_path=.:/usr/share/php:/usr/share/pear:/usr/share/awl/inc";
 		fastcgi_read_timeout 630;
 		fastcgi_pass php-fpm;
+
+		# Outgoing mail also goes through this endpoint, so increase the maximum
+		# file upload limit to match the corresponding Postfix limit.
+		client_max_body_size 128M;
 	}
 	location /autodiscover/autodiscover.xml {
 		include fastcgi_params;
@@ -73,4 +85,3 @@ server {
 
 	# ADDITIONAL DIRECTIVES HERE
 }
-

conf/sieve-spam.txt

@@ -1,7 +1,6 @@
 require ["regex", "fileinto", "imap4flags"];
 
 if allof (header :regex "X-Spam-Status" "^Yes") {
-  setflag "\\Seen";
   fileinto "Spam";
   stop;
 }

conf/zpush/backend_carddav.php

@@ -14,7 +14,7 @@ define('CARDDAV_DEFAULT_PATH', '/carddav/addressbooks/%u/contacts/'); /* subdire
 define('CARDDAV_GAL_PATH', ''); /* readonly, searchable, not syncd */
 define('CARDDAV_GAL_MIN_LENGTH', 5);
 define('CARDDAV_CONTACTS_FOLDER_NAME', '%u Addressbook');
-define('CARDDAV_SUPPORTS_SYNC', true);
+define('CARDDAV_SUPPORTS_SYNC', false);
 
 // If the CardDAV server supports the FN attribute for searches
 // DAViCal supports it, but SabreDav, Owncloud and SOGo don't

management/backup.py

@@ -38,6 +38,7 @@ def backup_status(env):
 	backups = { }
 	basedir = os.path.join(env['STORAGE_ROOT'], 'backup/duplicity/')
 	encdir = os.path.join(env['STORAGE_ROOT'], 'backup/encrypted/')
+	os.makedirs(basedir, exist_ok=True) # os.listdir fails if directory does not exist
 	for fn in os.listdir(basedir):
 		m = re.match(r"duplicity-(full|full-signatures|(inc|new-signatures)\.(?P<incbase>\d+T\d+Z)\.to)\.(?P<date>\d+T\d+Z)\.", fn)
 		if not m: raise ValueError(fn)

management/daemon.py

@@ -172,6 +172,12 @@ def mail_domains():
 
 # DNS
 
+@app.route('/dns/zones')
+@authorized_personnel_only
+def dns_zones():
+	from dns_update import get_dns_zones
+	return json_response([z[0] for z in get_dns_zones(env)])
+
 @app.route('/dns/update', methods=['POST'])
 @authorized_personnel_only
 def dns_update():
@@ -196,6 +202,17 @@ def dns_set_secondary_nameserver():
 	except ValueError as e:
 		return (str(e), 400)
 
+@app.route('/dns/set')
+@authorized_personnel_only
+def dns_get_records():
+	from dns_update import get_custom_dns_config, get_custom_records
+	additional_records = get_custom_dns_config(env)
+	records = get_custom_records(None, additional_records, env)
+	return json_response([{
+		"qname": r[0],
+		"rtype": r[1],
+		"value": r[2],
+		} for r in records])
 
 @app.route('/dns/set/<qname>', methods=['POST'])
 @app.route('/dns/set/<qname>/<rtype>', methods=['POST'])

management/dns_update.py

@@ -254,14 +254,17 @@ def build_zone(domain, all_domains, additional_records, env, is_zone=True):
 def get_custom_records(domain, additional_records, env):
 	for qname, value in additional_records.items():
 		# Is this record for the domain or one of its subdomains?
-		if qname != domain and not qname.endswith("." + domain): continue
+		# If `domain` is None, return records for all domains.
+		if domain is not None and qname != domain and not qname.endswith("." + domain): continue
 
 		# Turn the fully qualified domain name in the YAML file into
-		# our short form (None => domain, or a relative QNAME).
+		# our short form (None => domain, or a relative QNAME) if
-		if qname == domain:
+		# domain is not None.
-			qname = None
+		if domain is not None:
-		else:
+			if qname == domain:
-			qname = qname[0:len(qname)-len("." + domain)]
+				qname = None
+			else:
+				qname = qname[0:len(qname)-len("." + domain)]
 
 		# Short form. Mapping a domain name to a string is short-hand
 		# for creating A records.
@@ -490,7 +493,7 @@ zone:
 			# Get the IP address of the nameserver by resolving it.
 			hostname = additional_records.get("_secondary_nameserver")
 			resolver = dns.resolver.get_default_resolver()
-			response = dns.resolver.query(hostname, "A")
+			response = dns.resolver.query(hostname+'.', "A")
 			ipaddr = str(response[0])
 			nsdconf += """\tnotify: %s NOKEY
 	provide-xfr: %s NOKEY
@@ -511,8 +514,12 @@ zone:
 ########################################################################
 
 def dnssec_choose_algo(domain, env):
-	if domain.endswith(".email"):
+	if '.' in domain and domain.rsplit('.')[-1] in \
-		# At least at GoDaddy, this is the only algorithm supported.
+		("email", "guide", "fund"):
+		# At GoDaddy, RSASHA256 is the only algorithm supported
+		# for .email and .guide.
+		# A variety of algorithms are supported for .fund. This
+		# is preferred.
 		return "RSASHA256"
 
 	# For any domain we were able to sign before, don't change the algorithm
@@ -662,7 +669,7 @@ def set_custom_dns_record(qname, rtype, value, env):
 			v = ipaddress.ip_address(value)
 			if rtype == "A" and not isinstance(v, ipaddress.IPv4Address): raise ValueError("That's an IPv6 address.")
 			if rtype == "AAAA" and not isinstance(v, ipaddress.IPv6Address): raise ValueError("That's an IPv4 address.")
-		elif rtype in ("CNAME", "TXT"):
+		elif rtype in ("CNAME", "TXT", "SRV"):
 			# anything goes
 			pass
 		else:

management/mail_log.py

@@ -18,8 +18,9 @@ def scan_mail_log(logger, env):
 
 	for fn in ('/var/log/mail.log.1', '/var/log/mail.log'):
 		if not os.path.exists(fn): continue
-		with open(fn) as log:
+		with open(fn, 'rb') as log:
 			for line in log:
+				line = line.decode("utf8", errors='replace')
 				scan_mail_log_line(line.strip(), collector)
 
 	if collector["imap-logins"]:
@@ -96,6 +97,21 @@ def scan_postfix_smtpd_line(date, log, collector):
 		message, sender, recipient = m.groups()
 		if recipient in collector["real_mail_addresses"]:
 			# only log mail to real recipients
+
+			# skip this, is reported in the greylisting report
+			if "Recipient address rejected: Greylisted" in message:
+				return
+
+			# simplify this one
+			m = re.search(r"Client host \[(.*?)\] blocked using zen.spamhaus.org; (.*)", message)
+			if m:
+				message = "ip blocked: " + m.group(2)
+
+			# simplify this one too
+			m = re.search(r"Sender address \[.*@(.*)\] blocked using dbl.spamhaus.org; (.*)", message)
+			if m:
+				message = "domain blocked: " + m.group(2)
+
 			collector["rejected-mail"].setdefault(recipient, []).append( (date, sender, message) )
 
 

management/mailconfig.py

@@ -15,7 +15,7 @@ def validate_email(email, mode=None):
 	if mode == 'user':
 		# For Dovecot's benefit, only allow basic characters.
 		ATEXT = r'[\w\-]'
-	elif mode == 'alias':
+	elif mode in (None, 'alias'):
 		# For aliases, we can allow any valid email address.
 		# Based on RFC 2822 and https://github.com/SyrusAkbary/validate_email/blob/master/validate_email.py,
 		# these characters are permitted in email addresses.
@@ -27,7 +27,8 @@ def validate_email(email, mode=None):
 	DOT_ATOM_TEXT_LOCAL = ATEXT + r'+(?:\.' + ATEXT + r'+)*'
 	if mode == 'alias':
 		# For aliases, Postfix accepts '@domain.tld' format for
-		# catch-all addresses. Make the local part optional.
+		# catch-all addresses on the source side and domain aliases
+		# on the destination side. Make the local part optional.
 		DOT_ATOM_TEXT_LOCAL = '(?:' + DOT_ATOM_TEXT_LOCAL + ')?'
 
 	# as above, but we can require that the host part have at least
@@ -356,19 +357,28 @@ def add_mail_alias(source, destination, env, update_if_exists=False, do_kick=Tru
 	if not validate_email(source, mode='alias'):
 		return ("Invalid incoming email address (%s)." % source, 400)
 
-	# parse comma and \n-separated destination emails & validate
+	# validate destination
 	dests = []
-	for line in destination.split("\n"):
+	destination = destination.strip()
-		for email in line.split(","):
+	if validate_email(destination, mode='alias'):
-			email = email.strip()
+		# Oostfix allows a single @domain.tld as the destination, which means
-			if email == "": continue
+		# the local part on the address is preserved in the rewrite.
-			if not validate_email(email, mode='alias'):
+		dests.append(destination)
-				return ("Invalid destination email address (%s)." % email, 400)
+	else:
-			dests.append(email)
+		# Parse comma and \n-separated destination emails & validate. In this
+		# case, the recipients must be complete email addresses.
+		for line in destination.split("\n"):
+			for email in line.split(","):
+				email = email.strip()
+				if email == "": continue
+				if not validate_email(email):
+					return ("Invalid destination email address (%s)." % email, 400)
+				dests.append(email)
 	if len(destination) == 0:
 		return ("No destination email address(es) provided.", 400)
 	destination = ",".join(dests)
 
+	# save to db
 	conn, c = open_database(env, with_connection=True)
 	try:
 		c.execute("INSERT INTO aliases (source, destination) VALUES (?, ?)", (source, destination))

management/status_checks.py

@@ -18,6 +18,10 @@ from mailconfig import get_mail_domains, get_mail_aliases
 from utils import shell, sort_domains, load_env_vars_from_file
 
 def run_checks(env, output):
+	# clear bind9's DNS cache so our DNS checks are up to date
+	shell('check_call', ["/usr/sbin/rndc", "flush"])
+	
+	# perform checks
 	env["out"] = output
 	run_system_checks(env)
 	run_network_checks(env)
@@ -25,8 +29,17 @@ def run_checks(env, output):
 
 def run_system_checks(env):
 	env["out"].add_heading("System")
+	check_ssh_password(env)
+	check_software_updates(env)
+	check_system_aliases(env)
+	check_free_disk_space(env)
 
-	# Check that SSH login with password is disabled.
+def check_ssh_password(env):
+	# Check that SSH login with password is disabled. The openssh-server
+	# package may not be installed so check that before trying to access
+	# the configuration file.
+	if not os.path.exists("/etc/ssh/sshd_config"):
+		return
 	sshd = open("/etc/ssh/sshd_config").read()
 	if re.search("\nPasswordAuthentication\s+yes", sshd) \
 		or not re.search("\nPasswordAuthentication\s+no", sshd):
@@ -37,6 +50,7 @@ def run_system_checks(env):
 	else:
 		env['out'].print_ok("SSH disallows password-based login.")
 
+def check_software_updates(env):
 	# Check for any software package updates.
 	pkgs = list_apt_updates(apt_update=False)
 	if os.path.exists("/var/run/reboot-required"):
@@ -48,10 +62,12 @@ def run_system_checks(env):
 		for p in pkgs:
 			env['out'].print_line("%s (%s)" % (p["package"], p["version"]))
 
+def check_system_aliases(env):
 	# Check that the administrator alias exists since that's where all
 	# admin email is automatically directed.
 	check_alias_exists("administrator@" + env['PRIMARY_HOSTNAME'], env)
 
+def check_free_disk_space(env):
 	# Check free disk space.
 	st = os.statvfs(env['STORAGE_ROOT'])
 	bytes_total = st.f_blocks * st.f_frsize
@@ -175,9 +191,8 @@ def check_primary_hostname_dns(domain, env, dns_domains, dns_zonefiles):
 	elif tlsa25 is None:
 		env['out'].print_error("""The DANE TLSA record for incoming mail is not set. This is optional.""")
 	else:
-		env['out'].print_error("""The DANE TLSA record for incoming mail (%s) is not correct. It is '%s' but it should be '%s'. Try running tools/dns_update to
+		env['out'].print_error("""The DANE TLSA record for incoming mail (%s) is not correct. It is '%s' but it should be '%s'.
-			regenerate the record. It may take several hours for
+			It may take several hours for public DNS to update after a change."""
-                        public DNS to update after a change."""
                         % (tlsa_qname, tlsa25, tlsa25_expected))
 
 	# Check that the hostmaster@ email address exists.
@@ -204,10 +219,10 @@ def check_dns_zone(domain, env, dns_zonefiles):
 	# to do a DNS trace.
 	custom_dns = get_custom_dns_config(env)
 	existing_ns = query_dns(domain, "NS")
-	correct_ns = "; ".join([
+	correct_ns = "; ".join(sorted([
 		"ns1." + env['PRIMARY_HOSTNAME'],
 		custom_dns.get("_secondary_nameserver", "ns2." + env['PRIMARY_HOSTNAME']),
-		])
+		]))
 	if existing_ns.lower() == correct_ns.lower():
 		env['out'].print_ok("Nameservers are set correctly at registrar. [%s]" % correct_ns)
 	else:
@@ -344,7 +359,15 @@ def check_web_domain(domain, env):
 	check_ssl_cert(domain, env)
 
 def query_dns(qname, rtype, nxdomain='[Not Set]'):
-	resolver = dns.resolver.get_default_resolver()
+	# Make the qname absolute by appending a period. Without this, dns.resolver.query
+	# will fall back a failed lookup to a second query with this machine's hostname
+	# appended. This has been causing some false-positive Spamhaus reports. The
+	# reverse DNS lookup will pass a dns.name.Name instance which is already
+	# absolute so we should not modify that.
+	if isinstance(qname, str):
+		qname += "."
+
+	# Do the query.
 	try:
 		response = dns.resolver.query(qname, rtype)
 	except (dns.resolver.NoNameservers, dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
@@ -510,6 +533,9 @@ def check_certificate(domain, ssl_certificate, ssl_private_key):
 		# Certificate is self-signed.
 		return ("SELF-SIGNED", None)
 	elif retcode != 0:
+		if "unable to get local issuer certificate" in verifyoutput:
+			return ("The certificate is missing an intermediate chain or the intermediate chain is incorrect or incomplete. (%s)" % verifyoutput, None)
+
 		# There is some unknown problem. Return the `openssl verify` raw output.
 		return ("There is a problem with the SSL certificate.", verifyoutput.strip())
 	else:

management/templates/aliases.html

@@ -13,22 +13,26 @@
   <div class="form-group">
     <div class="col-sm-offset-1 col-sm-11">
       <div id="alias_type_buttons" class="btn-group btn-group-xs">
-        <button type="button" class="btn btn-default active">Regular</button>
+        <button type="button" class="btn btn-default active" data-mode="regular">Regular</button>
-        <button type="button" class="btn btn-default">Catch-All</button>
+        <button type="button" class="btn btn-default" data-mode="catchall">Catch-All</button>
+        <button type="button" class="btn btn-default" data-mode="domainalias">Domain Alias</button>
+      </div>
+      <div id="alias_mode_info" class="text-info small" style="display: none; margin: .5em 0 0 0;">
+        <span class="catchall hidden">A catch-all alias captures all otherwise unmatched email to a domain. Enter just a part of an email address starting with the @-sign.</span>
+        <span class="domainalias hidden">A domain alias forwards all otherwise unmatched mail from one domain to another domain, preserving the part before the @-sign.</span>
       </div>
-      <div id="alias_catchall_info" class="text-info small" style="display: none; margin: .5em 0 0 0;">A catch-all alias captures all otherwise unmatched email to a domain. Enter just a part of an email address starting with the @-sign.</div>
     </div>
   </div>
   <div class="form-group">
     <label for="addaliasEmail" class="col-sm-1 control-label">Alias</label>
     <div class="col-sm-10">
-      <input type="email" class="form-control" id="addaliasEmail" placeholder="incoming email address (you@yourdomain.com)">
+      <input type="email" class="form-control" id="addaliasEmail">
     </div>
   </div>
   <div class="form-group">
     <label for="addaliasTargets" class="col-sm-1 control-label">Forward To</label>
     <div class="col-sm-10">
-      <textarea class="form-control" rows="3" id="addaliasTargets" placeholder="forward to these email addresses (one per line or separated by commas)"></textarea>
+      <textarea class="form-control" rows="3" id="addaliasTargets"></textarea>
     </div>
   </div>
   <div class="form-group">
@@ -106,16 +110,28 @@ function show_aliases() {
     $('#alias_type_buttons button').off('click').click(function() {
       $('#alias_type_buttons button').removeClass('active');
       $(this).addClass('active');
-      if ($(this).text() == "Regular") {
+      if ($(this).attr('data-mode') == "regular") {
         $('#addaliasEmail').attr('type', 'email');
-        $('#addaliasEmail').attr('placeholder', 'incoming email address (you@yourdomain.com)');
+        $('#addaliasEmail').attr('placeholder', 'incoming email address (e.g. you@yourdomain.com)');
-        $('#alias_catchall_info').slideUp();
+	$('#addaliasTargets').attr('placeholder', 'forward to these email addresses (one per line or separated by commas)');
-      } else {
+        $('#alias_mode_info').slideUp();
+      } else if ($(this).attr('data-mode') == "catchall") {
         $('#addaliasEmail').attr('type', 'text');
-        $('#addaliasEmail').attr('placeholder', 'incoming catch-all address (@yourdomain.com)');
+        $('#addaliasEmail').attr('placeholder', 'incoming catch-all address (e.g. @yourdomain.com)');
-        $('#alias_catchall_info').slideDown();
+	$('#addaliasTargets').attr('placeholder', 'forward to these email addresses (one per line or separated by commas)');
+        $('#alias_mode_info').slideDown();
+        $('#alias_mode_info span').addClass('hidden');
+        $('#alias_mode_info span.catchall').removeClass('hidden');
+      } else if ($(this).attr('data-mode') == "domainalias") {
+        $('#addaliasEmail').attr('type', 'text');
+        $('#addaliasEmail').attr('placeholder', 'incoming domain (@yourdomain.com)');
+        $('#addaliasTargets').attr('placeholder', 'forward to domain (@yourdomain.com)');
+        $('#alias_mode_info').slideDown();
+        $('#alias_mode_info span').addClass('hidden');
+        $('#alias_mode_info span.domainalias').removeClass('hidden');
       }
     })
+    $('#alias_type_buttons button[data-mode="regular"]').click(); // init
   })
 }
 
@@ -166,6 +182,12 @@ function aliases_edit(elem) {
   $("#addaliasEmail").val(email);
   $("#addaliasTargets").val(targets);
   $('#add-alias-button').text('Update');
+  if (email.charAt(0) == '@' && targets.charAt(0) == '@')
+    $('#alias_type_buttons button[data-mode="domainalias"]').click();
+  else if (email.charAt(0) == '@')
+    $('#alias_type_buttons button[data-mode="catchall"]').click();
+  else
+    $('#alias_type_buttons button[data-mode="regular"]').click();
   $('body').animate({ scrollTop: 0 })
 }
 

management/templates/custom-dns.html

@@ -1,4 +1,7 @@
 <style>
+#custom-dns-current td.long {
+  word-break: break-all;
+}
 </style>
 
 <h2>Custom DNS</h2>
@@ -7,6 +10,60 @@
 
 <p>It is possible to set custom DNS records on domains hosted here.</p>
 
+<h3>Set Custom DNS Records</h3>
+
+<p>You can set additional DNS records, such as if you have a website running on another server, to add DKIM records for external mail providers, or for various confirmation-of-ownership tests.</p>
+
+<form class="form-horizontal" role="form" onsubmit="do_set_custom_dns(); return false;">
+  <div class="form-group">
+    <label for="customdnsQname" class="col-sm-1 control-label">Name</label>
+    <div class="col-sm-10">
+      <table style="max-width: 400px">
+      <tr><td>
+        <input type="text" class="form-control" id="customdnsQname" placeholder="subdomain">
+      </td><td style="padding: 0 1em; font-weight: bold;">.</td><td>
+        <select id="customdnsZone" class="form-control"> </select>
+      </td></tr></table>
+      <div class="text-info" style="margin-top: .5em">Leave the left field blank to set a record on the chosen domain name, or enter a subdomain.</div>
+    </div>
+  </div>
+  <div class="form-group">
+    <label for="customdnsType" class="col-sm-1 control-label">Type</label>
+    <div class="col-sm-10">
+      <select id="customdnsType" class="form-control" style="max-width: 400px" onchange="show_customdns_rtype_hint()">
+        <option value="A" data-hint="Enter an IPv4 address (i.e. a dotted quad, such as 123.456.789.012).">A (IPv4 address)</option>
+        <option value="AAAA" data-hint="Enter an IPv6 address.">AAAA (IPv6 address)</option>
+        <option value="CNAME" data-hint="Enter another domain name followed by a period at the end (e.g. mypage.github.io.).">CNAME (DNS forwarding)</option>
+        <option value="TXT" data-hint="Enter arbitrary text.">TXT (text record)</option>
+      </select>
+    </div>
+  </div>
+  <div class="form-group">
+    <label for="customdnsValue" class="col-sm-1 control-label">Value</label>
+    <div class="col-sm-10">
+      <input type="text" class="form-control" id="customdnsValue" placeholder="">
+      <div id="customdnsTypeHint" class="text-info" style="margin-top: .5em"></div>
+    </div>
+  </div>
+  <div class="form-group">
+    <div class="col-sm-offset-1 col-sm-11">
+      <button type="submit" class="btn btn-primary">Set Record</button>
+    </div>
+  </div>
+</form>
+
+<table id="custom-dns-current" class="table" style="width: auto; display: none">
+  <thead>
+    <th>Domain Name</th>
+    <th>Record Type</th>
+    <th>Value</th>
+    <th></th>
+  </thead>
+  <tbody>
+    <tr><td colspan="4">Loading...</td></tr>
+  </tbody>
+</table>
+
 <h3>Using a Secondary Nameserver</h3>
 
 <p>If your TLD requires you to have two separate nameservers, you can either set up a secondary (aka &ldquo;slave&rdquo;) nameserver or, alternatively, set up <a href="#" onclick="return show_panel('external_dns')">external DNS</a> and ignore the DNS server on this box. If you choose to use a seconday/slave nameserver, you must find a seconday/slave nameserver service provider. Your domain name registrar or virtual cloud provider may provide this service for you. Once you set up the seconday/slave nameserver service, enter the hostname of <em>their</em> secondary nameserver:</p>
@@ -30,6 +87,7 @@
   </div>
 </form>
 
+
 <h3>Custom DNS API</h3>
 
 <p>Use your box&rsquo;s DNS API to set custom DNS records on domains hosted here. For instance, you can create your own dynamic DNS service.</p>
@@ -66,11 +124,17 @@ curl -d "" --user me@mydomain.com:###### https://{{hostname}}/admin/dns/set/bar.
 
 # sets a TXT record using the alternate value syntax
 curl -d "value=something%20here" --user me@mydomain.com:###### https://{{hostname}}/admin/dns/set/foo.mydomain.com/txt
+
+# sets a <a href="http://en.wikipedia.org/wiki/SRV_record">SRV record</a> for the "service" and "protocol" hosted on "target" server
+curl -d "" --user me@mydomain.com:###### https://{{hostname}}/admin/dns/set/_service._protocol.{{hostname}}/srv/"priority weight port target"
+
+# sets a SRV record using the value syntax
+curl -d "value=priority weight port target" --user me@mydomain.com:###### https://{{hostname}}/admin/dns/set/_service._protocol.host/srv
 </pre>
 
 <script>
 function show_custom_dns() {
- api(
+  api(
     "/dns/secondary-nameserver",
     "GET",
     { },
@@ -78,6 +142,52 @@ function show_custom_dns() {
       $('#secondarydnsHostname').val(data.hostname ? data.hostname : '');
       $('#secondarydns-clear-instructions').toggle(data.hostname != null);
     });
+
+  api(
+    "/dns/zones",
+    "GET",
+    { },
+    function(data) {
+      $('#customdnsZone').text('');
+      for (var i = 0; i < data.length; i++) {
+        $('#customdnsZone').append($('<option/>').text(data[i]));
+      }
+    });
+
+  show_current_custom_dns();
+  show_customdns_rtype_hint();
+}
+
+function show_current_custom_dns() {
+  api(
+    "/dns/set",
+    "GET",
+    { },
+    function(data) {
+      if (data.length > 0)
+        $('#custom-dns-current').fadeIn();
+      else
+        $('#custom-dns-current').fadeOut();
+
+      $('#custom-dns-current').find("tbody").text('');
+      for (var i = 0; i < data.length; i++) {
+        var tr = $("<tr/>");
+        $('#custom-dns-current').find("tbody").append(tr);
+        tr.attr('data-qname', data[i].qname);
+        tr.attr('data-rtype', data[i].rtype);
+        tr.append($('<td class="long"/>').text(data[i].qname));
+        tr.append($('<td/>').text(data[i].rtype));
+        tr.append($('<td class="long"/>').text(data[i].value));
+        tr.append($('<td>[<a href="#" onclick="return delete_custom_dns_record(this)">delete</a>]</td>'));
+      }
+    });
+}
+
+function delete_custom_dns_record(elem) {
+  var qname = $(elem).parents('tr').attr('data-qname');
+  var rtype = $(elem).parents('tr').attr('data-rtype');
+  do_set_custom_dns(qname, rtype, "__delete__");
+  return false;
 }
 
 function do_set_secondary_dns() {
@@ -96,4 +206,34 @@ function do_set_secondary_dns() {
       show_modal_error("Secondary DNS", $("<pre/>").text(err));
     });
 }
+
+function do_set_custom_dns(qname, rtype, value) {
+  if (!qname) {
+    if ($('#customdnsQname').val() != '')
+      qname = $('#customdnsQname').val() + '.' + $('#customdnsZone').val();
+    else
+      qname = $('#customdnsZone').val();
+    rtype = $('#customdnsType').val();
+    value = $('#customdnsValue').val();
+  }
+
+  api(
+    "/dns/set/" + qname + "/" + rtype,
+    "POST",
+    {
+      value: value
+    },
+    function(data) {
+      if (data == "") return; // nothing updated
+      show_modal_error("Custom DNS", $("<pre/>").text(data));
+      show_current_custom_dns();
+    },
+    function(err) {
+      show_modal_error("Custom DNS", $("<pre/>").text(err));
+    });
+}
+
+function show_customdns_rtype_hint() {
+  $('#customdnsTypeHint').text($("#customdnsType").find('option:selected').attr('data-hint'));
+}
 </script>

management/templates/external-dns.html

@@ -12,8 +12,9 @@
   border: 0;
 	padding-top: .75em;
 	padding-bottom: 0;
-	max-width: 50vw;
+}
-	word-wrap: break-word;
+#external_dns_settings .value {
+	word-break: break-all;
 }
 #external_dns_settings .explanation td {
 	border: 0;

management/templates/index.html

@@ -46,14 +46,22 @@
               margin-bottom: 13px;
               margin-top: 30px;
             }
+              .panel-heading h3 {
+                border: none;
+                padding: 0;
+                margin: 0;
+              }
 
             h4 {
               font-size: 110%;
               margin-bottom: 13px;
               margin-top: 18px;
             }
+              h4:first-child {
+                margin-top: 6px;
+              }
 
-            .panel {
+            .admin_panel {
               display: none;
             }
 
@@ -92,7 +100,7 @@
                 <li><a href="#ssl" onclick="return show_panel(this);">SSL Certificates</a></li>
                 <li><a href="#system_backup" onclick="return show_panel(this);">Backup Status</a></li>
                 <li class="divider"></li>
-                <li class="dropdown-header">Super Advanced Options</li>
+                <li class="dropdown-header">Advanced Options</li>
                 <li><a href="#custom_dns" onclick="return show_panel(this);">Custom DNS</a></li>
                 <li><a href="#external_dns" onclick="return show_panel(this);">External DNS</a></li>
               </ul>
@@ -115,48 +123,48 @@
       </div>
     </div>
 
-    <div class="container-fluid">
+    <div class="container">
-      <div id="panel_system_status" class="container panel">
+      <div id="panel_system_status" class="admin_panel">
       {% include "system-status.html" %}
       </div>
 
-      <div id="panel_system_backup" class="container panel">
+      <div id="panel_system_backup" class="admin_panel">
       {% include "system-backup.html" %}
       </div>
 
-      <div id="panel_external_dns" class="container panel">
+      <div id="panel_external_dns" class="admin_panel">
       {% include "external-dns.html" %}
       </div>
 
-      <div id="panel_custom_dns" class="container panel">
+      <div id="panel_custom_dns" class="admin_panel">
       {% include "custom-dns.html" %}
       </div>
 
-      <div id="panel_login" class="panel">
+      <div id="panel_login" class="admin_panel">
       {% include "login.html" %}
       </div>
 
-      <div id="panel_mail-guide" class="container panel">
+      <div id="panel_mail-guide" class="admin_panel">
       {% include "mail-guide.html" %}
       </div>
 
-      <div id="panel_users" class="container panel">
+      <div id="panel_users" class="admin_panel">
       {% include "users.html" %}
       </div>
 
-      <div id="panel_aliases" class="container panel">
+      <div id="panel_aliases" class="admin_panel">
       {% include "aliases.html" %}
       </div>
 
-      <div id="panel_sync_guide" class="container panel">
+      <div id="panel_sync_guide" class="admin_panel">
       {% include "sync-guide.html" %}
       </div>
 
-      <div id="panel_web" class="container panel">
+      <div id="panel_web" class="admin_panel">
       {% include "web.html" %}
       </div>
 
-      <div id="panel_ssl" class="container panel">
+      <div id="panel_ssl" class="admin_panel">
       {% include "ssl.html" %}
       </div>
 
@@ -256,12 +264,13 @@ function show_modal_confirm(title, question, verb, yes_callback, cancel_callback
   $('#global_modal').modal({});
 }
 
-var is_ajax_loading = false;
+var ajax_num_executing_requests = 0;
 function ajax(options) {
-  setTimeout("if (is_ajax_loading) $('#ajax_loading_indicator').fadeIn()", 100);
+  setTimeout("if (ajax_num_executing_requests > 0) $('#ajax_loading_indicator').fadeIn()", 100);
   function hide_loading_indicator() {
-    is_ajax_loading = false;
+    ajax_num_executing_requests--;
-    $('#ajax_loading_indicator').hide();
+    if (ajax_num_executing_requests == 0)
+      $('#ajax_loading_indicator').stop().hide(); // stop() prevents an ongoing fade from causing the thing to be shown again after this call
   }
   var old_success = options.success;
   var old_error = options.error;
@@ -279,7 +288,7 @@ function ajax(options) {
     else
       old_error(jqxhr.responseText, jqxhr);
   };
-  is_ajax_loading = true;
+  ajax_num_executing_requests++;
   $.ajax(options);
 }
 
@@ -349,7 +358,7 @@ function show_panel(panelid) {
     // we might be passed an HTMLElement <a>.
     panelid = panelid.getAttribute('href').substring(1);
 
-  $('.panel').hide();
+  $('.admin_panel').hide();
   $('#panel_' + panelid).show();
   if (typeof localStorage != 'undefined')
     localStorage.setItem("miab-cp-lastpanel", panelid);

management/templates/login.html

@@ -1,7 +1,7 @@
 <h1 style="margin: 1em; text-align: center">{{hostname}}</h1>
 
 {% if no_admins_exist %}
-<div class="container">
+<div class="row">
 <div class="col-md-offset-2 col-md-8">
   <p class="text-danger">There are no administrative users on this system! To make an administrative user,
   log into this machine using SSH (like when you first set it up) and run:</p>
@@ -12,27 +12,24 @@ sudo tools/mail.py user make-admin your@emailaddress.com</pre>
 </div>  
 {% endif %}
 
-<div class="row">
+<p style="margin: 2em; text-align: center;">Log in here for your Mail-in-a-Box control panel.</p>
-  <div class="col-sm-offset-2 col-sm-8 col-md-offset-3 col-md-6 col-lg-offset-4 col-lg-4">
-    <center>
-      <p style="margin: 2em">Log in here for your Mail-in-a-Box control panel.</p>
-    </center>
 
+<div style="margin: 0 auto; max-width: 32em;">
   <form class="form-horizontal" role="form" onsubmit="do_login(); return false;">
     <div class="form-group">
-      <label for="inputEmail3" class="col-sm-2 control-label">Email</label>
+      <label for="inputEmail3" class="col-sm-3 control-label">Email</label>
-      <div class="col-sm-10">
+      <div class="col-sm-9">
         <input name="email" type="email" class="form-control" id="loginEmail" placeholder="Email">
       </div>
     </div>
     <div class="form-group">
-      <label for="inputPassword3" class="col-sm-2 control-label">Password</label>
+      <label for="inputPassword3" class="col-sm-3 control-label">Password</label>
-      <div class="col-sm-10">
+      <div class="col-sm-9">
         <input name="password" type="password" class="form-control" id="loginPassword" placeholder="Password">
       </div>
     </div>
     <div class="form-group">
-      <div class="col-sm-offset-2 col-sm-10">
+      <div class="col-sm-offset-3 col-sm-9">
         <div class="checkbox">
           <label>
             <input name='remember' type="checkbox" id="loginRemember"> Remember me
@@ -41,12 +38,11 @@ sudo tools/mail.py user make-admin your@emailaddress.com</pre>
       </div>
     </div>
     <div class="form-group">
-      <div class="col-sm-offset-2 col-sm-10">
+      <div class="col-sm-offset-3 col-sm-9">
         <button type="submit" class="btn btn-default">Sign in</button>
       </div>
     </div>
   </form>
-  </div>
 </div>
 
 

management/templates/mail-guide.html

@@ -1,56 +1,46 @@
 <style>#panel_mail-guide table.table { width: auto; margin-left: .5em; }</style>
 
-<div class="container">
+<div>
 	<h2 style="margin-bottom: 0">Checking and Sending Mail</h2>
 
 	<div class="row">
-		<div class="col-sm-6">
+		<div class="col-sm-7">
-			<h3>How to log in</h3>
-
-			<p>Your username and password are the same no matter how you check your mail:</p>
-
-			<table class="table" style="max-width: 30em">
-			<tr><th>Username:</th> <td>Your whole email address.</td></tr>
-			<tr><th>Password:</th> <td>Your mail password.</td></tr>
-			</table>
-
 			<h3>Webmail</h3>
 
 			<p>Webmail lets you check your email from any web browser. Your webmail site is:</p>
 			<p style="margin-left: 2em"><strong><a href="https://{{hostname}}/mail">https://{{hostname}}/mail</a></strong></p>
+			<p>Your username is your whole email address.</p>
+
 
 			<h3>Mobile/desktop apps</h3>
 
-			<p>When you set up your email on your phone, desktop, or other device, you will be asked to choose a protocol.</p>
+				<h4>Automatic configuration</h4>
 
-			<ul>
+					<p>iOS and OS X only: Open <a style="font-weight: bold" href="https://{{hostname}}/mailinabox.mobileconfig">this configuration link</a> on your iOS device or on your Mac desktop to easily set up mail (IMAP/SMTP), Contacts, and Calendar. Your username is your whole email address.</p>
-				<li>On Android devices, look for IMAP and SMTP.</li>
+	
-				<li>On iOS devices, look for Exchange or ActiveSync.</li>
+				<h4>Manual configuration</h4>
-			</ul>
 
-			<div class="row">
+					<p>Use the following settings when you set up your email on your phone, desktop, or other device:</p>
-				<div class="col-lg-6">
-					<h4>IMAP/SMTP settings</h4>
-
-					<p>This method is preferred on Android devices but is not available on iOS devices.</p>
-
-					<p>Your mail server is <strong>{{hostname}}</strong>. Use the following settings when prompted:</p>
 
 					<table class="table">
 					<thead>
-						<tr><th>Protocol</th> <th>Port</th> <th>Options</th></tr>
+						<tr><th>Option</th> <th>Value</th></tr>
 					</thead>
-					<tr><th>IMAP</th> <td>993</td> <td>SSL</td></tr>
+					<tr><th>Protocol/Method</th> <td>IMAP</td></tr>
-					<tr><th>SMTP</th> <td>587</td> <td>STARTTLS <span>(&ldquo;always&rdquo; or &ldquo;required&rdquo;, if prompted)</span></td></tr>
+					<tr><th>Mail server</th> <td>{{hostname}}</td>
+					<tr><th>IMAP Port</th> <td>993</td></tr>
+					<tr><th>IMAP Sercurity</th> <td>SSL</td></tr>
+					<tr><th>SMTP Port</th> <td>587</td></tr>
+					<tr><th>SMTP Security</td> <td>STARTTLS <small>(&ldquo;always&rdquo; or &ldquo;required&rdquo;, if prompted)</small></td></tr>
+					<tr><th>Username:</th> <td>Your whole email address.</td></tr>
+					<tr><th>Password:</th> <td>Your mail password.</td></tr>
 					</table>
 
 					<p>In addition to setting up your email, you&rsquo;ll also need to set up <a href="#sync_guide" onclick="return show_panel(this);">contacts and calendar synchronization</a> separately.</p>
-				</div>
 
-				<div class="col-lg-6">
+				<h4>Exchange/ActiveSync settings</h4>
-					<h4>Exchange/ActiveSync settings</h4>
 
-					<p>On iOS devices and devices on this <a href="http://z-push.org/compatibility/">compatibility list</a>, set up your mail as an Exchange or ActiveSync server. Use these settings when prompted:</p>
+					<p>On iOS devices and devices on this <a href="http://z-push.org/compatibility/">compatibility list</a>, you may set up your mail as an Exchange or ActiveSync server. However, we&rsquo;ve found this to be more buggy than using IMAP. If you encounter any problems, please use the manual settings above.</p>
 
 					<table class="table">
 					<tr><th>Server</th> <td>{{hostname}}</td></tr>
@@ -58,18 +48,24 @@
 					</table>
 
 					<p>Your device should also provide a contacts list and calendar that syncs to this box when you use this method.</p>
-				</div>
-			</div>
 		</div>
 
-		<div class="col-sm-6">
+		<div class="col-sm-5">
+		<div class="panel panel-default">
+		<div class="panel-heading">
 			<h3>Other information about mail on your box</h3>
-
+		</div>
+		<div class="panel-body">
 			<h4>Greylisting</h4>
 			<p>Your box using a technique called greylisting to cut down on spam. Greylisting works by delaying mail from people you haven&rsquo;t received mail from before for up to about 10 minutes. The vast majority of spam gets tricked by this. If you are waiting for an email from someone new, such as if you are registering on a new website and are waiting for an email confirmation, please give it up to 10-15 minutes to arrive.</p>
 
-			<h4>Use this box to send as you</h4>
+			<h4>+tag addresses</h4>
+			<p>Every incoming email address also receives mail for <code>+tag</code> addresses. If your email address is <code>you@yourdomain.com</code>, you can also accept mail at <code>you+anythinghere@yourdomain.com</code>. Use this as a fast way to create aliases or to segment incoming mail for your own filtering rules.</p>
+
+			<h4>Use only this box to send as you</h4>
 			<p>Your box sets strict email sending policies for your domain names to make it harder for spam and other fraudulent mail to claim to be you. Only this machine is authorized to send email on behalf of your domain names. If you use any other service to send email as you, it will likely get spam filtered by recipients.</p>
 		</div>
+		</div>
+		</div>
 	</div>
 </div>

management/templates/ssl.html

@@ -18,9 +18,11 @@
   </tbody>
 </table>
 
+<p>Advanced:<br>Install a multi-domain or wildcard certificate for the <code>{{hostname}}</code> domain to have it automatically applied to any domains it is valid for.</p>
+
 <h3 id="ssl_install_header">Install SSL Certificate</h3>
 
-<p>There are many places where you can get a free or cheap SSL certificate. We recommend <a href="https://www.namecheap.com/cart/remove.aspx?itemid=47016639&i=i2">Namecheap&rsquo;s $9 certificate</a> or <a href="https://www.startssl.com/">StartSSL&rsquo;s free express lane</a>.</p>
+<p>There are many places where you can get a free or cheap SSL certificate. We recommend <a href="https://www.namecheap.com/security/ssl-certificates/domain-validation.aspx">Namecheap&rsquo;s $9 certificate</a> or <a href="https://www.startssl.com/">StartSSL&rsquo;s free express lane</a>.</p>
 
 <p>Which domain are you getting an SSL certificate for?</p>
 
@@ -83,7 +85,7 @@ function ssl_install(elem) {
   $('#csr_info').slideDown();
   $('#ssl_csr').text('Loading...');
   show_csr();
-  $('html, body').animate({ scrollTop: $('#ssl_install_header').offset().top })
+  $('html, body').animate({ scrollTop: $('#ssl_install_header').offset().top - $('.navbar-fixed-top').height() - 20 })
   return false;
 }
 

management/templates/sync-guide.html

@@ -1,4 +1,4 @@
-<div class="container">
+<div>
 	<h2>Contacts &amp; Calendar Synchronization</h2>
 
 	<p>This box can hold your contacts and calendar, just like it holds your email.</p>

management/templates/system-backup.html

@@ -28,7 +28,7 @@
   </tbody>
 </table>
 
-<p><small>The size column in the table indicates the size of the encrpyted backup, but the total size on disk shown above includes storage for unencrpyted intermediate files.</small></p>
+<p style="margin-top: 2em"><small>The size column in the table indicates the size of the encrpyted backup, but the total size on disk shown above includes storage for unencrpyted intermediate files.</small></p>
 
 <script>
 function nice_size(bytes) {
@@ -59,6 +59,12 @@ function show_system_backup() {
 
       $('#backup-status tbody').html("");
       var total_disk_size = 0;
+
+      if (r.backups.length == 0) {
+        var tr = $('<tr><td colspan="3">No backups have been made yet.</td></tr>');
+        $('#backup-status tbody').append(tr);
+      }
+
       for (var i = 0; i < r.backups.length; i++) {
         var b = r.backups[i];
         var tr = $('<tr/>');

management/web_update.py

@@ -2,10 +2,10 @@
 # domains for which a mail account has been set up.
 ########################################################################
 
-import os, os.path, re, rtyaml
+import os, os.path, shutil, re, rtyaml
 
 from mailconfig import get_mail_domains
-from dns_update import get_custom_dns_config
+from dns_update import get_custom_dns_config, do_dns_update
 from utils import shell, safe_domain_name, sort_domains
 
 def get_web_domains(env):
@@ -115,6 +115,8 @@ def make_domain_config(domain, template, template_for_primaryhost, env):
 			yaml = yaml[domain]
 			for path, url in yaml.get("proxies", {}).items():
 				nginx_conf += "\tlocation %s {\n\t\tproxy_pass %s;\n\t}\n" % (path, url)
+			for path, url in yaml.get("redirects", {}).items():
+				nginx_conf += "\trewrite %s %s permanent;\n" % (path, url)
 
 	# Add in any user customizations in the includes/ folder.
 	nginx_conf_custom_include = os.path.join(env["STORAGE_ROOT"], "www", safe_domain_name(domain) + ".conf")
@@ -133,7 +135,7 @@ def get_web_root(domain, env, test_exists=True):
 		if os.path.exists(root) or not test_exists: break
 	return root
 
-def get_domain_ssl_files(domain, env):
+def get_domain_ssl_files(domain, env, allow_shared_cert=True):
 	# What SSL private key will we use? Allow the user to override this, but
 	# in many cases using the same private key for all domains would be fine.
 	# Don't allow the user to override the key for PRIMARY_HOSTNAME because
@@ -157,7 +159,7 @@ def get_domain_ssl_files(domain, env):
 		# But we can be smart and reuse the main SSL certificate if is has
 		# a Subject Alternative Name matching this domain. Don't do this if
 		# the user has uploaded a different private key for this domain.
-		if not ssl_key_is_alt:
+		if not ssl_key_is_alt and allow_shared_cert:
 			from status_checks import check_certificate
 			if check_certificate(domain, ssl_certificate_primary, None)[0] == "OK":
 				ssl_certificate = ssl_certificate_primary
@@ -223,20 +225,35 @@ def install_cert(domain, ssl_cert, ssl_chain, env):
 
 	# Do validation on the certificate before installing it.
 	from status_checks import check_certificate
-	ssl_key, ssl_certificate, ssl_csr_path = get_domain_ssl_files(domain, env)
+	ssl_key, ssl_certificate, ssl_csr_path = get_domain_ssl_files(domain, env, allow_shared_cert=False)
 	cert_status, cert_status_details = check_certificate(domain, fn, ssl_key)
 	if cert_status != "OK":
 		if cert_status == "SELF-SIGNED":
 			cert_status = "This is a self-signed certificate. I can't install that."
 		os.unlink(fn)
+		if cert_status_details is not None:
+			cert_status += " " + cert_status_details
 		return cert_status
 
 	# Copy the certificate to its expected location.
 	os.makedirs(os.path.dirname(ssl_certificate), exist_ok=True)
-	os.rename(fn, ssl_certificate)
+	shutil.move(fn, ssl_certificate)
+
+	ret = []
+
+	# When updating the cert for PRIMARY_HOSTNAME, also update DNS because it is
+	# used in the DANE TLSA record and restart postfix and dovecot which use
+	# that certificate.
+	if domain == env['PRIMARY_HOSTNAME']:
+		ret.append( do_dns_update(env) )
+
+		shell('check_call', ["/usr/sbin/service", "postfix", "restart"])
+		shell('check_call', ["/usr/sbin/service", "dovecot", "restart"])
+		ret.append("mail services restarted")
 
 	# Kick nginx so it sees the cert.
-	return do_web_update(env, ok_status="")
+	ret.append( do_web_update(env, ok_status="") )
+	return "\n".join(r for r in ret if r.strip() != "")
 
 def get_web_domains_info(env):
 	def check_cert(domain):
@@ -246,7 +263,11 @@ def get_web_domains_info(env):
 			return ("danger", "No Certificate Installed")
 		cert_status, cert_status_details = check_certificate(domain, ssl_certificate, ssl_key)
 		if cert_status == "OK":
-			return ("success", "Signed & valid. " + cert_status_details)
+			if domain == env['PRIMARY_HOSTNAME'] or ssl_certificate != get_domain_ssl_files(env['PRIMARY_HOSTNAME'], env)[1]:
+				return ("success", "Signed & valid. " + cert_status_details)
+			else:
+				# This is an alternate domain but using the same cert as the primary domain.
+				return ("success", "Signed & valid. Using multi/wildcard certificate of %s." % env['PRIMARY_HOSTNAME'])
 		elif cert_status == "SELF-SIGNED":
 			return ("warning", "Self-signed. Get a signed certificate to stop warnings.")
 		else:

setup/bootstrap.sh

@@ -7,7 +7,7 @@
 #########################################################
 
 if [ -z "$TAG" ]; then
-	TAG=v0.03
+	TAG=v0.06
 fi
 
 # Are we running as root?
@@ -17,12 +17,12 @@ if [[ $EUID -ne 0 ]]; then
 fi
 
 # Clone the Mail-in-a-Box repository if it doesn't exist.
-if [ ! -d mailinabox ]; then
+if [ ! -d $HOME/mailinabox ]; then
 	echo Installing git . . .
 	DEBIAN_FRONTEND=noninteractive apt-get -q -q install -y git < /dev/null
 	echo
 
-	echo Downloading Mail-in-a-Box . . .
+	echo Downloading Mail-in-a-Box $TAG. . .
 	git clone \
 		-b $TAG --depth 1 \
 		https://github.com/mail-in-a-box/mailinabox \
@@ -38,7 +38,7 @@ cd $HOME/mailinabox
 # Update it.
 if [ "$TAG" != `git describe` ]; then
 	echo Updating Mail-in-a-Box to $TAG . . .
-	git fetch
+	git fetch --depth 1 --force --prune origin tag $TAG
 	if ! git checkout -q $TAG; then
 		echo "Update failed. Did you modify something in `pwd`?"
 		exit

setup/dkim.sh

@@ -1,3 +1,4 @@
+#!/bin/bash
 # OpenDKIM
 # --------
 #
@@ -6,6 +7,7 @@
 # The DNS configuration for DKIM is done in the management daemon.
 
 source setup/functions.sh # load our functions
+source /etc/mailinabox.conf # load global vars
 
 # Install DKIM...
 apt_install opendkim opendkim-tools

setup/dns.sh

@@ -51,6 +51,11 @@ mkdir -p "$STORAGE_ROOT/dns/dnssec";
 # Requires `RSASHA256`
 #
 #  * .email
+#  * .guide
+#
+# Supports `RSASHA256` (and defaulting to this)
+#
+#  * .fund
 
 FIRST=1 #NODOC
 for algo in RSASHA1-NSEC3-SHA1 RSASHA256; do

setup/mail-dovecot.sh

@@ -26,6 +26,23 @@ apt_install \
 
 # The `dovecot-imapd` and `dovecot-lmtpd` packages automatically enable IMAP and LMTP protocols.
 
+# Set basic daemon options.
+
+# The `default_process_limit` is 100, which constrains the total number
+# of active IMAP connections (at, say, 5 open connections per user that
+# would be 20 users). Set it to 250 times the number of cores this
+# machine has, so on a two-core machine that's 500 processes/100 users).
+tools/editconf.py /etc/dovecot/conf.d/10-master.conf \
+	default_process_limit=$(echo "`nproc` * 250" | bc)
+
+# The inotify `max_user_instances` default is 128, which constrains
+# the total number of watched (IMAP IDLE push) folders by open connections.
+# See http://www.dovecot.org/pipermail/dovecot/2013-March/088834.html.
+# A reboot is required for this to take effect (which we don't do as
+# as a part of setup). Test with `cat /proc/sys/fs/inotify/max_user_instances`.
+tools/editconf.py /etc/sysctl.conf \
+	fs.inotify.max_user_instances=1024
+
 # Set the location where we'll store user mailboxes. '%d' is the domain name and '%n' is the
 # username part of the user's email address. We'll ensure that no bad domains or email addresses
 # are created within the management daemon.

setup/mail-postfix.sh

@@ -160,7 +160,13 @@ tools/editconf.py /etc/postfix/main.cf \
 	smtpd_sender_restrictions="reject_non_fqdn_sender,reject_unknown_sender_domain,reject_rhsbl_sender dbl.spamhaus.org" \
 	smtpd_recipient_restrictions=permit_sasl_authenticated,permit_mynetworks,"reject_rbl_client zen.spamhaus.org",reject_unlisted_recipient,"check_policy_service inet:127.0.0.1:10023"
 
+# Postfix connects to Postgrey on the 127.0.0.1 interface specifically. Ensure that
+# Postgrey listens on the same interface (and not IPv6, for instance).
+tools/editconf.py /etc/default/postgrey \
+	POSTGREY_OPTS=\"--inet=127.0.0.1:10023\"
+
 # Increase the message size limit from 10MB to 128MB.
+# The same limit is specified in nginx.conf for mail submitted via webmail and Z-Push.
 tools/editconf.py /etc/postfix/main.cf \
 	message_size_limit=134217728
 

setup/owncloud.sh

@@ -15,7 +15,7 @@ apt_install \
 apt-get purge -qq -y owncloud*
 
 # Install ownCloud from source of this version:
-owncloud_ver=7.0.2
+owncloud_ver=7.0.4
 
 # Check if ownCloud dir exist, and check if version matches owncloud_ver (if either doesn't - install/upgrade)
 if [ ! -d /usr/local/lib/owncloud/ ] \

setup/spamassassin.sh

@@ -30,6 +30,20 @@ hide_output pyzor discover
 # We've already configured Dovecot to listen on this port.
 tools/editconf.py /etc/default/spampd DESTPORT=10026
 
+# Spamassassin normally wraps spam as an attachment inside a fresh
+# email with a report about the message. This also protects the user
+# from accidentally openening a message with embedded malware.
+#
+# It's nice to see what rules caused the message to be marked as spam,
+# but it's also annoying to get to the original message when it is an
+# attachment, modern mail clients are safer now and don't load remote
+# content or execute scripts, and it is probably confusing to most users.
+#
+# Tell Spamassassin not to modify the original message except for adding
+# the X-Spam-Status mail header and related headers.
+tools/editconf.py /etc/spamassassin/local.cf -s \
+	report_safe=0
+
 # Bayesean learning
 # -----------------
 #

setup/start.sh

@@ -160,4 +160,3 @@ openssl x509 -in $STORAGE_ROOT/ssl/ssl_certificate.pem -noout -fingerprint \
 echo
 echo Then you can confirm the security exception and continue.
 echo
-

setup/system.sh

@@ -19,9 +19,12 @@ hide_output apt-get -y upgrade
 # * unattended-upgrades: Apt tool to install security updates automatically.
 # * ntp: keeps the system time correct
 # * fail2ban: scans log files for repeated failed login attempts and blocks the remote IP at the firewall
+# * sudo: allows privileged users to execute commands as root without being root
+# * coreutils: includes `nproc` tool to report number of processors
+# * bc: allows us to do math to compute sane defaults
 
 apt_install python3 python3-dev python3-pip \
-	wget curl \
+	wget curl sudo coreutils bc \
 	haveged unattended-upgrades ntp fail2ban
 
 # Allow apt to install system updates automatically every day.

setup/web.sh

@@ -40,6 +40,19 @@ tools/editconf.py /etc/php5/fpm/pool.d/www.conf -c ';' \
 # since it depends on what domains we're serving, which we don't know
 # until mail accounts have been created.
 
+# Create the iOS/OS X Mobile Configuration file which is exposed via the
+# nginx configuration at /mailinabox-mobileconfig.
+mkdir -p /var/lib/mailinabox
+chmod a+rx /var/lib/mailinabox
+cat conf/ios-profile.xml \
+	| sed "s/PRIMARY_HOSTNAME/$PRIMARY_HOSTNAME/" \
+	| sed "s/UUID1/$(cat /proc/sys/kernel/random/uuid)/" \
+	| sed "s/UUID2/$(cat /proc/sys/kernel/random/uuid)/" \
+	| sed "s/UUID3/$(cat /proc/sys/kernel/random/uuid)/" \
+	| sed "s/UUID4/$(cat /proc/sys/kernel/random/uuid)/" \
+	 > /var/lib/mailinabox/mobileconfig.xml
+chmod a+r /var/lib/mailinabox/mobileconfig.xml
+
 # make a default homepage
 if [ -d $STORAGE_ROOT/www/static ]; then mv $STORAGE_ROOT/www/static $STORAGE_ROOT/www/default; fi # migration #NODOC
 mkdir -p $STORAGE_ROOT/www/default

setup/webmail.sh

@@ -1,3 +1,4 @@
+#!/bin/bash
 # Webmail with Roundcube
 # ----------------------
 
@@ -29,7 +30,7 @@ apt_install \
 apt-get purge -qq -y roundcube* #NODOC
 
 # Install Roundcube from source if it is not already present or if it is out of date.
-VERSION=1.0.2
+VERSION=1.0.3
 needs_update=0 #NODOC
 if [ ! -f /usr/local/lib/roundcubemail/version ]; then
 	# not installed yet #NODOC
@@ -43,6 +44,7 @@ if [ $needs_update == 1 ]; then
 	rm -f /tmp/roundcube.tgz
 	wget -qO /tmp/roundcube.tgz http://downloads.sourceforge.net/project/roundcubemail/roundcubemail/$VERSION/roundcubemail-$VERSION.tar.gz
 	tar -C /usr/local/lib -zxf /tmp/roundcube.tgz
+	rm -rf /usr/local/lib/roundcubemail
 	mv /usr/local/lib/roundcubemail-$VERSION/ /usr/local/lib/roundcubemail
 	rm -f /tmp/roundcube.tgz
 	echo $VERSION > /usr/local/lib/roundcubemail/version

setup/zpush.sh

@@ -3,7 +3,7 @@
 # Z-Push: The Microsoft Exchange protocol server
 # ----------------------------------------------
 #
-# Mostly for use on iOS which doesn't support IMAP.
+# Mostly for use on iOS which doesn't support IMAP IDLE.
 #
 # Although Ubuntu ships Z-Push (as d-push) it has a dependency on Apache
 # so we won't install it that way.

tools/parse-nginx-log-bootstrap-accesses.py

@@ -0,0 +1,56 @@
+#!/usr/bin/python3
+#
+# This is a tool Josh uses on his box serving mailinabox.email to parse the nginx
+# access log to see how many people are installing Mail-in-a-Box each day, by
+# looking at accesses to the bootstrap.sh script.
+
+import re, glob, gzip, os.path, json
+import dateutil.parser
+
+outfn = "/home/user-data/www/mailinabox.email/install-stats.json"
+
+# Make a unique list of (date, ip address) pairs so we don't double-count
+# accesses that are for the same install.
+accesses = set()
+
+# Scan the current and rotated access logs.
+for fn in glob.glob("/var/log/nginx/access.log*"):
+	# Gunzip if necessary.
+	if fn.endswith(".gz"):
+		f = gzip.open(fn)
+	else:
+		f = open(fn, "rb")
+
+	# Loop through the lines in the access log.
+	with f:
+		for line in f:
+			# Find lines that are GETs on /bootstrap.sh by either curl or wget.
+			m = re.match(rb"(?P<ip>\S+) - - \[(?P<date>.*?)\] \"GET /bootstrap.sh HTTP/.*\" 200 \d+ .* \"(?:curl|wget)", line, re.I)
+			if m:
+				date, time = m.group("date").decode("ascii").split(":", 1)
+				date = dateutil.parser.parse(date).date().isoformat()
+				ip = m.group("ip").decode("ascii")
+				accesses.add( (date, ip) )
+
+# Aggregate by date.
+by_date = { }
+for date, ip in accesses:
+	by_date[date] = by_date.get(date, 0) + 1
+
+# Since logs are rotated, store the statistics permanently in a JSON file.
+# Load in the stats from an existing file.
+if os.path.exists(outfn):
+	existing_data = json.load(open(outfn))
+	for date, count in existing_data:
+		if date not in by_date:
+			by_date[date] = count
+
+# Turn into a list rather than a dict structure to make it ordered.
+by_date = sorted(by_date.items())
+
+# Pop the last one because today's stats are incomplete.
+by_date.pop(-1)
+
+# Write out.
+with open(outfn, "w") as f:
+	json.dump(by_date, f, sort_keys=True, indent=True)