Commit Graph

172 Commits

Author SHA1 Message Date
KiekerJan b051137f36 more resilient nameserver usage in query dns 2022-08-26 11:29:23 +02:00
KiekerJan 9c80ce8ff6 backup root is configurable 2022-07-20 16:15:16 +02:00
KiekerJan 69e15fa942 merge upstream v57 2022-06-13 20:07:50 +02:00
Joshua Tauberer 99474b348f Update backup to be compatible with duplicity 0.8.23
We were using duplicity 0.8.21-ppa202111091602~ubuntu1 from the duplicity PPA probably until June 5, which is when my box automatically updated to 0.8.23-ppa202205151528~ubuntu18.04.1. Starting with that version, two changes broke backups:

* The default s3 backend was changed to boto3. But boto3 depends on the AWS SDK which does not support Ubuntu 18.04, so we can't install it. Instead, we map s3: backup target URLs to the boto+s3 scheme which tells duplicity to use legacy boto. This should be reverted when we can switch to boto3.
* Contrary to the documentation, the s3 target no longer accepts a S3 hostname in the URL. It now reads the bucket from the hostname part of the URL. So we now drop the hostname from our target URL before passing it to duplicity and we pass the endpoint URL in a separate command-line argument. (The boto backend was dropped from duplicity's "uses_netloc" in 74d4cf44b1 (f5a07610d36bd242c3e5b98f8348879a468b866a_37_34), but other changes may be related.)

The change of target URL (due to both changes) seems to also cause duplicity to store cached data in a different directory within $STORAGE_ROOT/backup/cache, so on the next backup it will re-download cached manifest/signature files. Since the cache directory will still hold the prior data which is no longer needed, it might be a good idea to clear out the cache directory to save space. A system status checks message is added about that.

Fixes #2123
2022-06-12 08:17:48 -04:00
KiekerJan 02f2a34bcf remove deprecated method call 2022-05-30 19:37:00 +02:00
KiekerJan bf4ec5697b merge upstream 2022-05-12 22:15:52 +02:00
github@kiekerjan.isdronken.nl 6b30ee8665 skip retry on spamhaus dns lookups 2022-04-20 23:42:34 +02:00
KiekerJan a1851a413b use actual unbound command to flush cache 2022-04-18 21:52:33 +02:00
github@kiekerjan.isdronken.nl aaa7702d9d make dns resolver retrying explicit 2022-04-18 21:40:20 +02:00
KiekerJan 1b0f7991db fix spelling error 2022-04-18 08:30:22 +02:00
KiekerJan d35b068a73 add dns exception handling 2022-04-17 22:56:30 +02:00
KiekerJan 9b252e0209 retrying dns timeouts 2022-04-04 22:31:54 +02:00
KiekerJan 7ac4b412b0 attempts to reduce unnecessary dns update messages 2022-04-03 16:37:51 +02:00
lamberete 6e40c69cb5
Error message using IPv4 instead of failing IPv6.
One of the error messages around IPv6 was using the IPv4 for the output, making the error message confusing.
2022-03-26 13:50:24 +01:00
lamberete c0e54f87d7
Sorting ds records on report.
When building the part of the report about the current DS records founded, they are added in the same order as they were received when calling query_dns(), which can differ from run to run. This was making the difflib.SequenceMatcher() method to find the same line removed and added one line later, and sending an Status Checks Change Notice email with the same line added and removed when there was actually no real changes.
2022-03-26 13:45:49 +01:00
KiekerJan 640751b606 initial changes to use unbound as local dns resolver instead of bind 2022-03-20 20:57:19 +01:00
KiekerJan aefc4536d4 fix error when secondary dns server cannot be resolved, turn it into a warning 2022-03-15 21:41:59 +01:00
steadfasterX 3a739823af fix: key flag id for KSK, fix format (#2063)
as mentioned (https://github.com/mail-in-a-box/mailinabox/pull/2033#issuecomment-976365087) KSK is 257, not 256
2022-02-01 21:28:33 +01:00
github@kiekerjan.isdronken.nl ded1b55ebd First steps in migrating to dkimpy-milter 2021-12-11 00:54:56 +01:00
steadfasterX aac878dce5
fix: key flag id for KSK, fix format (#2063)
as mentioned (https://github.com/mail-in-a-box/mailinabox/pull/2033#issuecomment-976365087) KSK is 257, not 256
2021-11-23 11:06:17 -05:00
github@kiekerjan.isdronken.nl eeada2b9b5 merge changes from V55 upstream 2021-10-19 23:07:02 +02:00
KiekerJan e54dc19854 slightly change dns resolver call 2021-09-21 22:17:10 +02:00
mailinabox-contributor 91079ab934
add numeric flag value to DNSSEC DS status message (#2033)
Some registrars (e.g. Porkbun) accept Key Data when creating a DS RR,
but accept only a numeric flags value to indicate the key type (256 for KSK, 257 for ZSK).

https://datatracker.ietf.org/doc/html/rfc5910#section-4.3
2021-09-10 16:12:41 -04:00
kiekerjan 98c00d1c6a
Merge branch 'mail-in-a-box:main' into master 2021-08-28 13:38:15 +02:00
Joshua Tauberer 67b5711c68 Recommend that DS records be updated to not use SHA1 and exclude MUST NOT methods (SHA1) and the unlikely option RSASHA1-NSEC3-SHA1 (7) + SHA-384 (4) from the DS record suggestions 2021-08-22 14:43:46 -04:00
myfirstnameispaul 20ccda8710 Re-order DS record algorithms by digest type and revise warning message.
Note that 7, 4 is printed last in the status checks page but does not appear in the file, and I couldn't figure out why.
2021-08-22 14:29:36 -04:00
KiekerJan eb36091d41 syntax error fix 2021-06-24 12:56:18 +02:00
github@kiekerjan.isdronken.nl 4f7957a5ab check presence of dnssec key file before reading it 2021-06-24 12:47:46 +02:00
KiekerJan 56f9df738f version recognition 2021-06-23 21:02:21 +02:00
github@kiekerjan.isdronken.nl ca5fb3c2e0 Merge changes from upstream v0.54 2021-06-20 23:36:54 +02:00
kiekerjan c25bb085d6
Merge pull request #3 from kiekerjan/develop-dns-mods
Develop dns mods
2021-05-29 22:39:31 +02:00
KiekerJan 28b828be12 check service on ipv6 if it is not found on ipv4 2021-05-28 23:36:25 +02:00
github@kiekerjan.isdronken.nl 1d96be9ea9 take hidden master dns into account for the status checks 2021-05-24 21:32:13 +02:00
Joshua Tauberer d510c8ae2a Enable and recommend port 465 for mail submission instead of port 587 (fixes #1849)
Port 465 with "implicit" (i.e. always-on) TLS is a more secure approach than port 587 with explicit (i.e. optional and only on with STARTTLS). Although we reject credentials on port 587 without STARTTLS, by that point credentials have already been sent.
2021-05-15 16:42:14 -04:00
KiekerJan aadd37e248 correct python spacing, sigh 2021-05-10 09:42:03 +02:00
KiekerJan 764a81d335 Merge branch 'develop-xapian-fts' 2021-05-09 21:20:58 +02:00
github@kiekerjan.isdronken.nl 2865cad111 take possible kiekerjan edition into account in tag 2021-05-09 21:16:22 +02:00
github@kiekerjan.isdronken.nl d875c9ff70 remove check on solr service 2021-05-08 23:04:13 +02:00
Joshua Tauberer aaa81ec879 Fix indentation issue in bc4ae51c2d 2021-05-08 09:06:18 -04:00
Hala Alajlan bc4ae51c2d
Handle query dns timeout unhandled error (#1950)
Co-authored-by: hala alajlan <halalajlan@gmail.com>
2021-05-08 08:26:40 -04:00
github@kiekerjan.isdronken.nl 3609a9e96c fix Solr report 2021-04-29 23:11:19 +02:00
github@kiekerjan.isdronken.nl e946276f15 install solr without ubuntu package 2021-04-21 22:26:49 +02:00
github@kiekerjan.isdronken.nl 4aaee13c1c Add solr full text search based on https://github.com/jvolkenant/mailinabox/tree/solr-jetty 2021-04-17 23:00:14 +02:00
github@kiekerjan.isdronken.nl c24ca5abd4 include changes from v0.53. Remove some POWER modifications to closer follow original mialinabox 2021-04-13 09:50:23 +02:00
Joshua Tauberer 178c587654 Migrate to the ECDSAP256SHA256 (13) DNSSEC algorithm
* Stop generating RSASHA1-NSEC3-SHA1 keys on new installs since it is no longer recommended, but preserve the key on existing installs so that we continue to sign zones with existing keys to retain the chain of trust with existing DS records.
* Start generating ECDSAP256SHA256 keys during setup, the current best practice (in addition to RSASHA256 which is also ok). See https://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml#dns-sec-alg-numbers-1 and https://www.cloudflare.com/dns/dnssec/ecdsa-and-dnssec/.
* Sign zones using all available keys rather than choosing just one based on the TLD to enable rotation/migration to the new key and to give the user some options since not every registrar/TLD supports every algorithm.
* Allow a user to drop a key from signing specific domains using DOMAINS= in our key configuration file. Signing the zones with extraneous keys may increase the size of DNS responses, which isn't ideal, although I don't know if this is a problem in practice. (Although a user can delete the RSASHA1-NSEC3-SHA1 key file, the other keys will be re-generated on upgrade.)
* When generating zonefiles, add a hash of all of the DNSSEC signing keys so that when the keys change the zone is definitely regenerated and re-signed.
* In status checks, if DNSSEC is not active (or not valid), offer to use all of the keys that have been generated (for RSASHA1-NSEC3-SHA1 on existing installs, RSASHA256, and now ECDSAP256SHA256) with all digest types, since not all registers support everything, but list them in an order that guides users to the best practice.
* In status checks, if the deployed DS record doesn't use a ECDSAP256SHA256 key, prompt the user to update their DS record.
* In status checks, if multiple DS records are set, only fail if none are valid. If some use ECDSAP256SHA256 and some don't, remind the user to delete the DS records that don't.
* Don't fail if the DS record uses the SHA384 digest (by pre-generating a DS record with that digest type) but don't recommend it because it is not in the IANA mandatory list yet (https://www.iana.org/assignments/ds-rr-types/ds-rr-types.xhtml).

See #1953
2021-04-12 19:42:12 -04:00
github@kiekerjan.isdronken.nl 12d0aee27a Add own changes 2021-04-11 12:14:41 +02:00
David Duque 4829e687ff
Merge changes from master 2021-01-31 16:20:15 +00:00
Felix Spöttel e3d98b781e
Warn when connection to Spamhaus times out (#1817) 2021-01-28 18:22:43 -05:00
David Duque 94da7bb088
status_checks.py: Properly terminate the process pools (#1795)
* Only spawn a thread pool when strictly needed

For --check-primary-hostname, the pool is not used.
When exiting, the other processes are left alive and will hang.

* Acquire pools with the 'with' statement
2020-08-09 11:42:39 -04:00
David Duque 5e597bb536
Update deprecated function from dnspython 2020-07-26 01:00:17 +01:00