* drop the ondrej/php PPA since PHP 7.x is available directly from Ubuntu 18.04
* intall PHP 7.2 which is just the "php" package in Ubuntu 18.04
* some package names changed, some unnecessary packages are no longer provided
* update paths
@joshdata squashed pull request #1398, removed some comments, and added these notes:
* The old init.d script for the management daemon is replaced with a systemd service.
* A systemd service configuration is added to configure permissions for munin on startup.
* nginx SSL settings are updated because nginx's options and defaults have changed, and we now enable http2.
* Automatic SSHFP record generation is updated to know that 22 is the default SSH daemon port, since it is no longer explicit in sshd_config.
* The dovecot-lucene package is dropped because the Mail-in-a-Box PPA where we built the package has not been updated for Ubuntu 18.04.
* The stock postgrey package is installed instead of the one from our PPA (which we no longer support), which loses the automatic whitelisting of DNSWL.org-whitelisted senders.
* Drop memcached and the status check for memcached, which we used to use with ownCloud long ago but are no longer installing.
* Other minor changes.
Passwords must be eight characters long; when passwords are changed via the users page the dialog states that passwords need to be at least four characters but only eight or more are acceptable.
The cryptography package has created all sorts of installation trouble over the last few years, probably because of mismatches between OS-installed packages and pip-installed packages. Using a virtualenv for all Python packages used by the management daemon should make sure everything is consistent.
See #1298, see #1264.
* The Mozilla recommendations must have been updated in the last few years.
* The HSTS header must have >=6 months to get an A+ at ssllabs.com/ssltest.
* [Issue #1159] Remove any +tag name in email alias before checking privileges
* Move priprivileged email check after the conversion to unicode so only IDNA serves as input
* Fixed issue with relative path for rsync relative names
Actually using the parsed URL `path` part, instead of doing a lousy split().
Renamed the `p` variable into something more sensible (`target`).
Fixes: #1019
* Added more verbose error messages upon rsync failures
fixes#1033
* Added command to test file listing
* Added support for backup to a remote server using rsync
* updated web interface to get data from user
* added way to list files from server
It’s not using the “username” field of the yaml configuration
file to minimise the amount of patches needed. So the username
is actually sorted within the rsync URL.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Added ssh key generation upon installation for root user.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Removed stale blank lines, and fixed typo
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* fix backup-location lines, by switching it from id to class
* Various web UI fixes
- fixed user field being shadowed ;
- fixed settings reading comparaison ;
- fixed forgotten min-age field.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Added SSH Public Key shown on the web interface UI
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* trailing spaces.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* fixed the extraneous environment
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Updated key setup
- made key lower in bits, but stronger (using -a option),
- made ssh-keygen run in background using nohup,
- added independent key file, as id_rsa_miab,
- added ssh-options to all duplicity calls to use the id_rsa_miab keyfile,
- changed path to the public key display
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* added rsync options for ssh identity support
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* removed strict host checking for all backup operations
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Remove nohup from ssh-keygen so errors aren't hidden. Also only generate a key if none exists yet
* Add trailing slash when checking a remote backup. Also check if we actually can read the remote size
* Factorisation of the repeated rsync/ssh options
cf https://github.com/mail-in-a-box/mailinabox/pull/678#discussion_r81478919
* Updated message SSH key creation
https://github.com/mail-in-a-box/mailinabox/pull/678#discussion_r81478886
* sshfp records from nonstandard ports
If port 22 is not open, dns_update.py will not create SSHFP records
because it only scans port 22 for keys. This commit modifies
dns_update.py to parse the sshd_config file for open ports, and
then obtains keys from one of them (even if port 22 is not open).
* modified test of s per JoshData request
* edit CHANGELOG per JoshData
* fix typo
Seems like if REQUEST_METHOD is set to GET, then we can drop two redundant ways the query string is given. munin-cgi-graph itself reads the environment variables only, but its calls to Perl's CGI::param will look at the command line if REQUEST_METHOD is not used, otherwise it uses environment variables like CGI used to work.
Since this is all behind admin auth anyway, there isn't a public vulnerability. #914 was opened without comment which lead me to notice the redundancy and worry about a vulnerability, before I realized this is admin-only anyway.
DavDroid's latest version's account configuration no longer just asked for a hostname. Its email address & password configuration mode did not work without a SRV record.
adding :
target="_blank"
to
<li><a href="/admin/munin">Munin Monitoring</a></li> on line 96
Why ?
Because when you click on munin link, and follow links, you lose your index, or click back many times...
So i propose my pull request.
Et voilà ^^
On some machines localhost is defined as something other than 127.0.0.1, and if we mix "127.0.0.1" and "localhost" then some connections won't be to to the address a service is actually running on.
This was the case with DKIM: It was running on "localhost" but Postfix was connecting to it at 127.0.0.1. (https://discourse.mailinabox.email/t/opendkim-is-not-running-port-8891/1188/12.)
I suppose "localhost" could be an alias to an IPv6 address? We don't really want local services binding on IPv6, so use "127.0.0.1" to be explicit and don't use "localhost" to be sure we get an IPv4 address.
Fixes#797
This was originally fixed in 143bbf37f4 (February 16, 2015). Then I broke it in 7a93d219ef (November 2015) while doing some refactoring ahead of v0.15.
https://discourse.mailinabox.email/t/status-check-emails-empty-after-upgrading-to-v0-16/1082/3
A user on that thread suggests an alternate solution, adding `PYTHONIOENCODING=utf-8` to `/etc/environment`. Python docs say that affects stdin/out/err. But we also use these environment variables elsewhere to ensure that config files we read/write are opened with UTF8 too. Maybe all that can be simplified too.
Refactor by moving the email-the-admin code out of the status checks and into a new separate tool.
This is why I suppressed non-error output of the backups last commit - so it doesn't send a daily email.
For HTTPS for the non-primary domains, instead of selecting an SSL certificate by expecting it to be in a directory named after the domain name (with special-case lookups
for www domains, and reusing the server certificate where possible), now scan all of the certificates that have been installed and just pick the best to use for each domain.
If no certificate is available, don't create a self-signed certificate anymore. This wasn't ever really necessary. Instead just use the server certificate.
* going from s3 to file target wasn't working
* use 'local' in the config instead of a file: url, for the local target, so it is not path-specific
* break out the S3 fields since users can't be expected to know how to form a URL
* use boto to generate a list of S3 hosts
* use boto to validate that the user input for s3 is valid
* fix lots of html errors in the backup admin
Backup location and maximum age can now be configured in the admin panel.
For now only S3 is supported, but adding other duplicity supported backends should be straightforward.
This is an extension of #427. Building on that change it adds support in the
aliases table for flagging aliases as:
1. Applicable to inbound and outbound mail.
2. Applicable to inbound mail only.
3. Applicable to outbound mail only.
4. Disabled.
The aliases UI is also updated to allow administrators to set the direction of
each alias.
Using this extra information, the sqlite queries executed by Postfix are
updated so only the relevant alias types are checked.
The goal and result of this change is that outbound-only catch-all aliases can
now be defined (in fact catch-all aliases of any type can be defined).
This allow us to continue supporting relaying as described at
https://mailinabox.email/advanced-configuration.html#relay
without requiring that administrators either create regular aliases for each
outbound *relay* address, or that they create a catch-all alias and then face a
flood of spam.
I have tested the code as it is in this commit and fixed every issue I found,
so in that regard the change is complete. However I see room for improvement
in terms of updating terminology to make the UI etc. easier to understand.
I'll make those changes as subsequent commits so that this tested checkpoint is
not lost, but also so they can be rejected independently of the actual change
if not wanted.
While not widely supported, there are some browser addons that can
validate DNSSEC and TLSA for additional out-of-band verification of
certificates when browsing the web. Costs nothing to implement and
might improve security in some situations.
* Add a migration to delete any existing DKIM key so that existing machines get a fresh 2048-bit key. (Sadly we don't support key rotation so the change is immediate.)
* Because the DNS record for a 2048-bit key is so much longer, the way we read OpenDKIM's DNS record text file had to be modified to combine an arbitrary number of TXT record quoted ("...") strings.
* When writing out the TXT record value, the string must be split into quoted ("...") strings with a maximum length of 255 bytes each, per the DNS spec.
* Added a changelog entry.
* Use `cryptography` instead of parsing openssl's output.
* When checking if we can reuse the primary domain certificate or a www-parent-domain certificate for a domain, avoid shelling out to openssl entirely.
* Split the nginx templates again so we have just the part needed to make a domain do a redirect separate from the rest.
* Add server blocks to the nginx config for these domains.
* List these domains in the SSL certificate install admin panel.
* Generate default 'www' records just for domains we provide default redirects for.
Fixes#321.
Unfortunately our users/aliases database is case sensitive. (Perhaps I should have defined the columns with COLLATE NOCASE, see https://www.sqlite.org/datatype3.html.) Postfix always queries the tables in lowecase, so mail delivery would fail if a user or alias were defined with any capital letters. It would have also been possible to add multiple euqivalent addresses into the database with different case.
This commit rejects new mail users that have capital letters and forces new aliases to lowecase. I prefer to reject rather than casefold user accounts so that the login credentials the user gave are exactly what goes into the database.
https://discourse.mailinabox.email/t/recipient-address-rejected-user-unknown-in-virtual-mailbox-table/512/4
This also includes fixes for a typo and some whitespace inconsistencies in
mailconfig.py. In fact the capitalisation change and those fixes are the
remnants of a patch I had been running that changed the default aliases - it
was through developing it that I found the issues.
(I wanted to bring the number of patches I apply before deploying to zero and
in the case of this one I've come to view the way MIAB already is as superior,
so I've undone the core of my patch and these tiny issues are all that remain).
* install the munin-node package
* don't install munin-plugins-extra (if the user wants it they can add it)
* expose the munin www directory via the management daemon so that it can handle authorization, rather than manintaining a separate password file
The OVH VPS provider creates systems without /dev/stdout. I have never seen that before. But fine. We were passing it as a command line option to `openssl req`, but outputting to stdout is the default so it's not necessary to specify /dev/stdout.
Fixes#277. Also https://discourse.mailinabox.email/t/500-internal-server-error/475/10.
* use the AES256 cipher, be explicit that only the first line of secret_key.txt is used, and sanity check that the passphrase is long enough
* change overship of the encrypted files to the user-data user
* simplify variable names in management/backup.py
* although I appreciate long comments I am trimming the commentary about the backup migration
* revise the control panel template to not refer to the old unencrypted files
* add CHANGELOG entry
[Josh merged some subsequent commits:]
* Guard via idempotency against termination between migration operations
* Final corrections and tweaks
* Pass passphrase through to all duplicity calls
Empirical evidence (a failed cron job) shows that cleanup requires the
passphrase (so it presumably needs to decrypt metadata), and though
remove-older-than has been working fine without it, it won't do any harm
to set it in case that changes or there are any special cases.
* Add back the archive-dir override but locate it at STORAGE_ROOT/backup/cache
I changed my mind. In 1bf8f1991f I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.
It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).
This partially reverts commit 1bf8f1991f, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.
A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.
Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.
* For non-ASCII domain names, we will keep the Unicode encoding in our users/aliases table. This is nice for the user and also simplifies things like sorting domain names (using Unicode lexicographic order is good, using ASCII lexicogrpahic order on IDNA is confusing).
* Write nsd config, nsd zone files, nginx config, and SSL CSRs with domains in IDNA-encoded ASCII.
* When checking SSL certificates, treat the CN and SANs as IDNA.
* Since Chrome has an interesting feature of converting Unicode to IDNA in <input type="email"> form fields, we'll also forcibly convert IDNA to Unicode in the domain part of email addresses before saving email addresses in the users/aliases tables so that the table is normalized to Unicode.
* Don't allow non-ASCII characters in user account email addresses. Dovecot gets confused when querying the Sqlite database (which we observed even for non-word ASCII characters too, so it may not be related to the character encoding).
