Merge remote-tracking branch 'upstream/master'

CHANGELOG.md

@@ -1,13 +1,16 @@
 CHANGELOG
 =========
 
-In Development
+v0.50 (September 25, 2020)
---------------
+--------------------------
+
+Setup:
+
+* When upgrading from versions before v0.40, setup will now warn that ownCloud/Nextcloud data cannot be migrated rather than failing the installation.
 
 Mail:
 
 * An MTA-STS policy for incoming mail is now published (in DNS and over HTTPS) when the primary hostname and email address domain both have a signed TLS certificate installed, allowing senders to know that an encrypted connection should be enforced.
-* MTA-STS reporting is enabled with reports sent to administrator@ the primary hostname.
 * The per-IP connection limit to the IMAP server has been doubled to allow more devices to connect at once, especially with multiple users behind a NAT.
 
 DNS:
@@ -21,6 +24,7 @@ TLS:
 
 Control Panel:
 
+* The control panel API is now fully documented at https://mailinabox.email/api-docs.html.
 * User passwords can now have spaces.
 * Status checks for automatic subdomains have been moved into the section for the parent domain.
 * Typo fixed.

README.md

@@ -19,20 +19,21 @@ Our goals are to:
 
 Additionally, this project has a [Code of Conduct](CODE_OF_CONDUCT.md), which supersedes the goals above. Please review it when joining our community.
 
-The Box
+
--------
+In The Box
+----------
 
 Mail-in-a-Box turns a fresh Ubuntu 18.04 LTS 64-bit machine into a working mail server by installing and configuring various components.
 
-It is a one-click email appliance. There are no user-configurable setup options. It "just works".
+It is a one-click email appliance. There are no user-configurable setup options. It "just works."
 
 The components installed are:
 
-* SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), and Exchange ActiveSync ([z-push](http://z-push.org/)) servers
+* SMTP ([postfix](http://www.postfix.org/)), IMAP ([Dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), and Exchange ActiveSync ([z-push](http://z-push.org/)) servers
-* Webmail ([Roundcube](http://roundcube.net/)), mail filter rules (also using dovecot), and email client autoconfig settings (served by [nginx](http://nginx.org/))
+* Webmail ([Roundcube](http://roundcube.net/)), mail filter rules (thanks to Roundcube and Dovecot), and email client autoconfig settings (served by [nginx](http://nginx.org/))
 * Spam filtering ([spamassassin](https://spamassassin.apache.org/)) and greylisting ([postgrey](http://postgrey.schweikert.ch/))
 * DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), [MTA-STS](https://tools.ietf.org/html/rfc8461), and [SSHFP](https://tools.ietf.org/html/rfc4255) policy records automatically set
-* HTTPS TLS certificates are automatically provisioned using [Let's Encrypt](https://letsencrypt.org/) (needed for webmail, CardDAV/CalDAV, ActiveSync, MTA-STS policy, etc.).
+* TLS certificates are automatically provisioned using [Let's Encrypt](https://letsencrypt.org/) for protecting https and all of the other services on the box
 * Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), and basic system monitoring ([munin](http://munin-monitoring.org/))
 
 It also includes system management tools:
@@ -41,10 +42,11 @@ It also includes system management tools:
 * A control panel for adding/removing mail users, aliases, custom DNS records, configuring backups, etc.
 * An API for all of the actions on the control panel
 
-It also supports static website hosting since the box is serving HTTPS anyway.
+It also supports static website hosting since the box is serving HTTPS anyway. (To serve a website for your domains elsewhere, just add a custom DNS "A" record in you Mail-in-a-Box's control panel to point domains to another server.)
 
 For more information on how Mail-in-a-Box handles your privacy, see the [security details page](security.md).
 
+
 Installation
 ------------
 
@@ -63,7 +65,7 @@ by him:
 	$ curl -s https://keybase.io/joshdata/key.asc | gpg --import
 	gpg: key C10BDD81: public key "Joshua Tauberer <jt@occams.info>" imported
 
-	$ git verify-tag v0.48
+	$ git verify-tag v0.50
 	gpg: Signature made ..... using RSA key ID C10BDD81
 	gpg: Good signature from "Joshua Tauberer <jt@occams.info>"
 	gpg: WARNING: This key is not certified with a trusted signature!
@@ -76,7 +78,7 @@ and on his [personal homepage](https://razor.occams.info/). (Of course, if this
 
 Checkout the tag corresponding to the most recent release:
 
-	$ git checkout v0.48
+	$ git checkout v0.50
 
 Begin the installation.
 
@@ -86,6 +88,9 @@ For help, DO NOT contact Josh directly --- I don't do tech support by email or t
 
 Post your question on the [discussion forum](https://discourse.mailinabox.email/) instead, where maintainers and Mail-in-a-Box users may be able to help you.
 
+Note that while we want everything to "just work," we can't control the rest of the Internet. Other mail services might block or spam-filter email sent from your Mail-in-a-Box.
+This is a challenge faced by everyone who runs their own mail server, with or without Mail-in-a-Box. See our discussion forum for tips about that.
+
 Contributing and Development
 ----------------------------
 
@@ -99,6 +104,7 @@ This project was inspired in part by the ["NSA-proof your email in 2 hours"](htt
 
 Mail-in-a-Box is similar to [iRedMail](http://www.iredmail.org/) and [Modoboa](https://github.com/tonioo/modoboa).
 
+
 The History
 -----------
 

conf/mta-sts.txt

@@ -1,4 +1,4 @@
 version: STSv1
 mode: MODE
 mx: PRIMARY_HOSTNAME
-max_age: 86400
+max_age: 604800

management/dns_update.py

@@ -351,14 +351,10 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
 			("_mta-sts", "TXT", "v=STSv1; id=" + mta_sts_policy_id, "Optional. Part of the MTA-STS policy for incoming mail. If set, a MTA-STS policy must also be published.")
 		])
 
-		# Rules can be custom configured accoring to https://tools.ietf.org/html/rfc8460.
+		# Enable SMTP TLS reporting (https://tools.ietf.org/html/rfc8460) if the user has set a config option.
 		# Skip if the rules below if the user has set a custom _smtp._tls record.
-		if not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"):
+		if env.get("MTA_STS_TLSRPT_RUA") and not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"):
-			tls_rpt_string = ""
+			mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1; rua=" + env["MTA_STS_TLSRPT_RUA"], "Optional. Enables MTA-STS reporting."))
-			tls_rpt_email = env.get("MTA_STS_TLSRPT_EMAIL", "postmaster@%s" % env['PRIMARY_HOSTNAME'])
-			if tls_rpt_email: # if a reporting address is not cleared
-				tls_rpt_string = " rua=mailto:%s" % tls_rpt_email
-			mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1;%s" % tls_rpt_string, "Optional. Enables MTA-STS reporting."))
 	for qname, rtype, value, explanation in mta_sts_records:
 		if value is None or value.strip() == "": continue # skip IPV6 if not set
 		if not has_rec(qname, rtype):

management/ssl_certificates.py

@@ -606,10 +606,10 @@ def check_certificate(domain, ssl_certificate, ssl_private_key, warn_if_expiring
 		ndays = (cert_expiration_date-now).days
 		if not rounded_time or ndays <= 10:
 			# Yikes better renew soon!
-			expiry_info = "The certificate expires in %d days on %s." % (ndays, cert_expiration_date.strftime("%x"))
+			expiry_info = "The certificate expires in %d days on %s." % (ndays, cert_expiration_date.date().isoformat())
 		else:
 			# We'll renew it with Lets Encrypt.
-			expiry_info = "The certificate expires on %s." % cert_expiration_date.strftime("%x")
+			expiry_info = "The certificate expires on %s." % cert_expiration_date.date().isoformat()
 
 		if warn_if_expiring_soon and ndays <= warn_if_expiring_soon:
 			# Warn on day 10 to give 4 days for us to automatically renew the

management/templates/system-backup.html

@@ -113,10 +113,10 @@
   </div>
   <!-- Common -->
   <div class="form-group backup-target-local backup-target-rsync backup-target-s3">
-    <label for="min-age" class="col-sm-2 control-label">Days:</label>
+    <label for="min-age" class="col-sm-2 control-label">Retention Days:</label>
     <div class="col-sm-8">
       <input type="number" class="form-control" rows="1" id="min-age">
-      <div class="small" style="margin-top: 2px">This is the <i>minimum</i> number of days backup data is kept for. The box makes an incremental backup, so backup data is often kept much longer. An incremental backup file that is less than this number of days old requires that all previous increments back to the most recent full backup, plus that full backup, remain available.</div>
+      <div class="small" style="margin-top: 2px">This is the minimum time backup data is kept for. The box makes an incremental backup most nights, which requires that previous backups back to the most recent full backup be preserved, so backup data is often kept much longer than this setting. Full backups are made periodically when the incremental backup data size exceeds a limit.</div>
     </div>
   </div>
   <div class="form-group">

management/web_update.py

@@ -146,7 +146,7 @@ def make_domain_config(domain, templates, ssl_certificates, env):
 		finally:
 			f.close()
 		return sha1.hexdigest()
-	nginx_conf_extra += "# ssl files sha1: %s / %s\n" % (hashfile(tls_cert["private-key"]), hashfile(tls_cert["certificate"]))
+	nginx_conf_extra += "\t# ssl files sha1: %s / %s\n" % (hashfile(tls_cert["private-key"]), hashfile(tls_cert["certificate"]))
 
 	# Add in any user customizations in YAML format.
 	hsts = "yes"
@@ -188,9 +188,9 @@ def make_domain_config(domain, templates, ssl_certificates, env):
 
 	# Add the HSTS header.
 	if hsts == "yes":
-		nginx_conf_extra += "add_header Strict-Transport-Security \"max-age=15768000\" always;\n"
+		nginx_conf_extra += "\tadd_header Strict-Transport-Security \"max-age=15768000\" always;\n"
 	elif hsts == "preload":
-		nginx_conf_extra += "add_header Strict-Transport-Security \"max-age=15768000; includeSubDomains; preload\" always;\n"
+		nginx_conf_extra += "\tadd_header Strict-Transport-Security \"max-age=15768000; includeSubDomains; preload\" always;\n"
 
 	# Add in any user customizations in the includes/ folder.
 	nginx_conf_custom_include = os.path.join(env["STORAGE_ROOT"], "www", safe_domain_name(domain) + ".conf")

setup/bootstrap.sh

@@ -20,7 +20,7 @@ if [ -z "$TAG" ]; then
 	# want to display in status checks.
 	if [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' `" == "Ubuntu 18.04 LTS" ]; then
 		# This machine is running Ubuntu 18.04.
-		TAG=v0.48
+		TAG=v0.50
 
 	elif [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/14\.04\.[0-9]/14.04/' `" == "Ubuntu 14.04 LTS" ]; then
 		# This machine is running Ubuntu 14.04.

setup/nextcloud.sh

@@ -134,11 +134,11 @@ if [ ! -d /usr/local/lib/owncloud/ ] || [[ ! ${CURRENT_NEXTCLOUD_VER} =~ ^$nextc
 		# Database migrations from ownCloud are no longer possible because ownCloud cannot be run under
 		# PHP 7.
 		if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^[89] ]]; then
-			echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 8 or 9) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup aborting."
+			echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 8 or 9) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup will continue, but skip the Nextcloud migration."
-			exit 1
+			return 0
 		elif [[ ${CURRENT_NEXTCLOUD_VER} =~ ^1[012] ]]; then
-			echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 10, 11 or 12) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup aborting."
+			echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 10, 11 or 12) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup will continue, but skip the Nextcloud migration."
-			exit 1
+			return 0
 		elif [[ ${CURRENT_NEXTCLOUD_VER} =~ ^13 ]]; then
 			# If we are running Nextcloud 13, upgrade to Nextcloud 14
 			InstallNextcloud 14.0.6 4e43a57340f04c2da306c8eea98e30040399ae5a