Commit Graph

131 Commits

Author SHA1 Message Date
Hugh Secker-Walker 820a39b865
chore(python open): Refactor open and gzip.open to use context manager (#2203)
Co-authored-by: Hugh Secker-Walker <hsw+miac@hodain.net>
2023-01-15 08:28:43 -05:00
KiekerJan 0fc5105da5
Fixes to DNS lookups during status checks when there are timeouts, enforce timeouts better (#2191)
* add dns query handling changes

* replace exception pass with error message

* simplify dns exception catching

* Add not set case to blacklist lookup result handling
2023-01-15 08:20:08 -05:00
Steve Hay 7cda439c80
Port boto to boto3 and fix asyncio issue in the management daemon (#2156)
Co-authored-by: Steve Hay <hay.steve@gmail.com>
2022-09-17 07:57:12 -04:00
Rauno Moisto 78569e9a88 Fix DeprecationWarning in dnspython query vs resolve method
The resolve method disables resolving relative names by default. This change probably makes a7710e90 unnecessary. @JoshData added some additional changes from query to resolve.
2022-07-28 14:02:46 -04:00
Joshua Tauberer 99474b348f Update backup to be compatible with duplicity 0.8.23
We were using duplicity 0.8.21-ppa202111091602~ubuntu1 from the duplicity PPA probably until June 5, which is when my box automatically updated to 0.8.23-ppa202205151528~ubuntu18.04.1. Starting with that version, two changes broke backups:

* The default s3 backend was changed to boto3. But boto3 depends on the AWS SDK which does not support Ubuntu 18.04, so we can't install it. Instead, we map s3: backup target URLs to the boto+s3 scheme which tells duplicity to use legacy boto. This should be reverted when we can switch to boto3.
* Contrary to the documentation, the s3 target no longer accepts a S3 hostname in the URL. It now reads the bucket from the hostname part of the URL. So we now drop the hostname from our target URL before passing it to duplicity and we pass the endpoint URL in a separate command-line argument. (The boto backend was dropped from duplicity's "uses_netloc" in 74d4cf44b1 (f5a07610d36bd242c3e5b98f8348879a468b866a_37_34), but other changes may be related.)

The change of target URL (due to both changes) seems to also cause duplicity to store cached data in a different directory within $STORAGE_ROOT/backup/cache, so on the next backup it will re-download cached manifest/signature files. Since the cache directory will still hold the prior data which is no longer needed, it might be a good idea to clear out the cache directory to save space. A system status checks message is added about that.

Fixes #2123
2022-06-12 08:17:48 -04:00
lamberete 6e40c69cb5
Error message using IPv4 instead of failing IPv6.
One of the error messages around IPv6 was using the IPv4 for the output, making the error message confusing.
2022-03-26 13:50:24 +01:00
lamberete c0e54f87d7
Sorting ds records on report.
When building the part of the report about the current DS records founded, they are added in the same order as they were received when calling query_dns(), which can differ from run to run. This was making the difflib.SequenceMatcher() method to find the same line removed and added one line later, and sending an Status Checks Change Notice email with the same line added and removed when there was actually no real changes.
2022-03-26 13:45:49 +01:00
steadfasterX aac878dce5
fix: key flag id for KSK, fix format (#2063)
as mentioned (https://github.com/mail-in-a-box/mailinabox/pull/2033#issuecomment-976365087) KSK is 257, not 256
2021-11-23 11:06:17 -05:00
mailinabox-contributor 91079ab934
add numeric flag value to DNSSEC DS status message (#2033)
Some registrars (e.g. Porkbun) accept Key Data when creating a DS RR,
but accept only a numeric flags value to indicate the key type (256 for KSK, 257 for ZSK).

https://datatracker.ietf.org/doc/html/rfc5910#section-4.3
2021-09-10 16:12:41 -04:00
Joshua Tauberer 67b5711c68 Recommend that DS records be updated to not use SHA1 and exclude MUST NOT methods (SHA1) and the unlikely option RSASHA1-NSEC3-SHA1 (7) + SHA-384 (4) from the DS record suggestions 2021-08-22 14:43:46 -04:00
myfirstnameispaul 20ccda8710 Re-order DS record algorithms by digest type and revise warning message.
Note that 7, 4 is printed last in the status checks page but does not appear in the file, and I couldn't figure out why.
2021-08-22 14:29:36 -04:00
Joshua Tauberer d510c8ae2a Enable and recommend port 465 for mail submission instead of port 587 (fixes #1849)
Port 465 with "implicit" (i.e. always-on) TLS is a more secure approach than port 587 with explicit (i.e. optional and only on with STARTTLS). Although we reject credentials on port 587 without STARTTLS, by that point credentials have already been sent.
2021-05-15 16:42:14 -04:00
Joshua Tauberer aaa81ec879 Fix indentation issue in bc4ae51c2d 2021-05-08 09:06:18 -04:00
Hala Alajlan bc4ae51c2d
Handle query dns timeout unhandled error (#1950)
Co-authored-by: hala alajlan <halalajlan@gmail.com>
2021-05-08 08:26:40 -04:00
Joshua Tauberer 178c587654 Migrate to the ECDSAP256SHA256 (13) DNSSEC algorithm
* Stop generating RSASHA1-NSEC3-SHA1 keys on new installs since it is no longer recommended, but preserve the key on existing installs so that we continue to sign zones with existing keys to retain the chain of trust with existing DS records.
* Start generating ECDSAP256SHA256 keys during setup, the current best practice (in addition to RSASHA256 which is also ok). See https://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml#dns-sec-alg-numbers-1 and https://www.cloudflare.com/dns/dnssec/ecdsa-and-dnssec/.
* Sign zones using all available keys rather than choosing just one based on the TLD to enable rotation/migration to the new key and to give the user some options since not every registrar/TLD supports every algorithm.
* Allow a user to drop a key from signing specific domains using DOMAINS= in our key configuration file. Signing the zones with extraneous keys may increase the size of DNS responses, which isn't ideal, although I don't know if this is a problem in practice. (Although a user can delete the RSASHA1-NSEC3-SHA1 key file, the other keys will be re-generated on upgrade.)
* When generating zonefiles, add a hash of all of the DNSSEC signing keys so that when the keys change the zone is definitely regenerated and re-signed.
* In status checks, if DNSSEC is not active (or not valid), offer to use all of the keys that have been generated (for RSASHA1-NSEC3-SHA1 on existing installs, RSASHA256, and now ECDSAP256SHA256) with all digest types, since not all registers support everything, but list them in an order that guides users to the best practice.
* In status checks, if the deployed DS record doesn't use a ECDSAP256SHA256 key, prompt the user to update their DS record.
* In status checks, if multiple DS records are set, only fail if none are valid. If some use ECDSAP256SHA256 and some don't, remind the user to delete the DS records that don't.
* Don't fail if the DS record uses the SHA384 digest (by pre-generating a DS record with that digest type) but don't recommend it because it is not in the IANA mandatory list yet (https://www.iana.org/assignments/ds-rr-types/ds-rr-types.xhtml).

See #1953
2021-04-12 19:42:12 -04:00
Felix Spöttel e3d98b781e
Warn when connection to Spamhaus times out (#1817) 2021-01-28 18:22:43 -05:00
David Duque 94da7bb088
status_checks.py: Properly terminate the process pools (#1795)
* Only spawn a thread pool when strictly needed

For --check-primary-hostname, the pool is not used.
When exiting, the other processes are left alive and will hang.

* Acquire pools with the 'with' statement
2020-08-09 11:42:39 -04:00
Joshua Tauberer b805f8695e Move status checks for www, autoconfig, autodiscover, and mta-sts to within the section for the parent domain
Since we're checking the MTA-STS policy, there's no need to check that the domain resolves etc. directly.
2020-05-29 15:38:13 -04:00
Joshua Tauberer 10bedad3a3 MTA-STS tweaks, add status check using postfix-mta-sts-resolver, change to enforce 2020-05-29 15:36:52 -04:00
Snacho 08021ea19f Fix an issue when Secondary NS has multiple A records (#1633)
If a custom secondary NS server has multiple A records status_checks.py will fail with a timeout and Web UI won't load.
2019-08-31 07:58:12 -04:00
Joshua Tauberer 60f9c9e3b7 show the Mail-in-a-Box version in the system status checks even when the new-version check is disabled
fixes #922
2018-11-30 10:46:54 -05:00
Christopher A. DeFlumeri d96613b8fe minimal changeset to get things working on 18.04
@joshdata squashed pull request #1398, removed some comments, and added these notes:

* The old init.d script for the management daemon is replaced with a systemd service.
* A systemd service configuration is added to configure permissions for munin on startup.
* nginx SSL settings are updated because nginx's options and defaults have changed, and we now enable http2.
* Automatic SSHFP record generation is updated to know that 22 is the default SSH daemon port, since it is no longer explicit in sshd_config.
* The dovecot-lucene package is dropped because the Mail-in-a-Box PPA where we built the package has not been updated for Ubuntu 18.04.
* The stock postgrey package is installed instead of the one from our PPA (which we no longer support), which loses the automatic whitelisting of DNSWL.org-whitelisted senders.
* Drop memcached and the status check for memcached, which we used to use with ownCloud long ago but are no longer installing.
* Other minor changes.
2018-10-03 13:00:06 -04:00
Joshua Tauberer 8be23d5ef6 ssl_certificates: reuse query_dns function in status_checks and simplify calls by calling normalize_ip within query_dns 2018-06-29 16:46:21 -04:00
Joshua Tauberer 0088fb4553 install Python 3 packages in a virtualenv
The cryptography package has created all sorts of installation trouble over the last few years, probably because of mismatches between OS-installed packages and pip-installed packages. Using a virtualenv for all Python packages used by the management daemon should make sure everything is consistent.

See #1298, see #1264.
2018-01-15 13:27:04 -05:00
Michael Kroes 2c324d0bc9 web_domains should also normalize ipv6 addresses (#1201) 2017-07-13 07:16:12 -04:00
Michael Kroes a072730fb8 Wrap normalize_ip in try..except (#1139)
closes #1134
2017-04-03 16:53:53 -04:00
Joshua Tauberer a24977a96e normalize_ip for ipv6 still not correct, was broken if box has no IPv6 address 2017-01-18 07:51:59 -05:00
Jonathan Chun 584cfe42c4 compare IPv6 addresses correctly with normalization (#1052) 2017-01-15 10:41:12 -05:00
Michael Kroes 41601a592f Improve error handling when doing update checks (#1065)
* Added an error message to handle exceptions when the setup script is trying to determine the latest Miab version
2017-01-15 10:35:33 -05:00
Joshua Tauberer 99d0afd650 secondary nameserver check fails if domain has custom DNS (round-robin) multiple A records
fixes #834
2016-12-07 07:02:52 -05:00
yodax 3b78a8d9d6 If ufw isn't installed on the machine the status checks shouldn't fail 2016-11-12 09:25:34 -05:00
Michael Kroes a658abc95f Fix status checks for ufw when the system doesn't support iptables (#961) 2016-10-08 14:35:19 -04:00
Mathis Hoffmann 163daea41c dnspython exception with IPv6 addresses
see #945, merges #947
2016-09-23 07:35:53 -04:00
cs@twoflower 00bd23eb04 fix status_checks.py free disk space reporting #932 2016-09-15 17:01:21 +01:00
Joshua Tauberer b044dda28f put the ufw status checks in the network section, add a punctuation mark, add changelog entry 2016-07-29 09:23:36 -04:00
Michael Kroes 736b3de221 Improve matching of ufw output. Reuse network service list. Improve messages 2016-04-07 16:03:28 +02:00
Michael Kroes c9f30e8059 Add status checks for ufw 2016-04-02 13:41:16 +02:00
Joshua Tauberer 67555679bd move the reboot button, fix grammar, refactor check for DRY, add changelog entry 2016-03-23 16:37:15 -04:00
yodax 1b24e2cbaf Reduce percentages for required memory checks 2016-02-22 17:49:19 +01:00
Wolf-Bastian Pöttner 239eac662c Fix: Correct IP is reported when using custom DNS
Fix bug that reports wrong ip, when custom DNS is enabled
2016-02-04 21:32:11 +01:00
Joshua Tauberer 83ffc99b9c change the public URL of bootstrap.sh to setup.sh 2016-01-30 11:19:51 -05:00
Joshua Tauberer 2ad7d0830e add exception handling for what_version_is_this, fixes #659 2016-01-09 09:23:07 -05:00
Joshua Tauberer 812ef024ef status checks: check that the non-primary domains also resolve over IPv6, if configured 2016-01-04 18:43:17 -05:00
Joshua Tauberer 40cdc5aa30 status checks: if a domain's DNS isnt working dont check the TLS certificate because we cant automatically provision one now anyway 2016-01-04 18:43:17 -05:00
Joshua Tauberer 4b4f670adf s/SSL/TLS/ in user-visible text throughout the project 2016-01-04 18:43:16 -05:00
Joshua Tauberer b6933a73fa provision and install free SSL certificates from Let's Encrypt 2016-01-04 18:43:16 -05:00
Joshua Tauberer 5033042b8c backups: email the administrator when there's a problem
Refactor by moving the email-the-admin code out of the status checks and into a new separate tool.

This is why I suppressed non-error output of the backups last commit - so it doesn't send a daily email.
2016-01-04 18:43:02 -05:00
Joshua Tauberer 06a0e7f3fe merge #584 - Add checks to the management interface to report memory usage 2016-01-01 18:13:21 -05:00
Joshua Tauberer a9cd72bbf9 tighten the status text strings for free memory, add changelog entry 2016-01-01 18:12:36 -05:00
Joshua Tauberer 682b1dea5e changelog/status checks updated for opening the sieve port 2016-01-01 17:53:05 -05:00