Commit Graph

83 Commits

Author SHA1 Message Date
Joshua Tauberer 89cd9fb611 Increase gunicorn's worker timeout since some /admin commands take a long time 2022-10-08 08:23:48 -04:00
Joshua Tauberer 22a6270657 Remove old setup step to uninstall acme library 2022-10-08 08:23:48 -04:00
Steve Hay 3fd2e3efa9
Replace Flask built-in WSGI server with gunicorn (#2158) 2022-09-17 08:03:16 -04:00
Joshua Tauberer 91fc74b408 Setup fixes for Ubuntu 22.04
Nextcloud:
* The Nextcloud user_external 1.0.0 package for Nextcloud 21.0.7 isn't available from Nextcloud's releases page, but it's not needed in an intermediate upgrade step (hopefully), so we can skip it.
* Nextcloud updgrade steps should not be elifs because multiple intermediate upgrades may be needed.
* Continue if the user_external backend migration fails. Maybe it's not necessary. It gives a scary error message though.
* Remove a line that removes an old file that hasn't been in use since 2019 and the expectation is that Ubuntu 22.04 installations are on fresh machines.

Backups:
* For duplicity, we now need boto3 for AWS.
2022-09-03 07:50:36 -04:00
Joshua Tauberer ab71abbc7c Update to latest cryptography Python package, add missing source at top of management.sh so it can run standalone (needs STORAGE_ROOT) 2022-07-28 14:42:51 -04:00
Joshua Tauberer 78d71498fa Upgrade from PHP 7.2 to 8.0 for Ubuntu 22.04
* Add the PHP PPA.
* Specify the version when invoking the php CLI.
* Specify the version in package names.
* Update paths to 8.0 (using a variable in the setup scripts).
* Update z-push's php-xsl dependency to php8.0-xml.
* php-json is now built-into PHP.

Although PHP 8.1 is the stock version in Ubuntu 22.04, it's not supported by Nextcloud yet, and it likely will never be supported by the the version of Nextcloud that succeeds the last version of Nextcloud that supports PHP 7.2, and we have to install the next version so that an upgrade is permitted, so skipping to PHP 8.1 may not be easily possible.
2022-07-28 14:02:46 -04:00
Joshua Tauberer b41a0ad80e Drop some hacks that we needed for Ubuntu 18.04
* certbot's PPA is no longer needed because a recent version is now included in the Ubuntu respository.
* Un-pin b2sdk (reverts 69d8fdef99 and d829d74048).
* Revert boto+s3 workaround for duplicity (partial revert of 99474b348f).
* Revert old "fix boto 2 conflict on Google Compute Engine instances" (cf33be4596) which is probably no longer needed.
2022-07-28 14:02:46 -04:00
Joshua Tauberer d829d74048 Pin b2sdk to version 1.14.1 in the virtualenv also
We install b2sdk in two places: Once globally for duplicity (see
9d8fdef9915127f016eb6424322a149cdff25d7 for #2125) and once in
a virtualenv used by our control panel. The latter wasn't pinned
when the former was but should be to fix new Python compatibility
issues.

Anyone who updated Python packages recently (so anyone who upgraded
Mail-in-a-Box) started encountering these issues.

Fixes #2131.

See https://discourse.mailinabox.email/t/backblaze-b2-backup-not-working-since-v57/9231.
2022-06-18 13:15:59 -04:00
m-picc 69d8fdef99
Specify b2sdk version 1.14.1 (#2125)
pin b2sdk version to 1.14.1 to resolve exception that occurs when attempting to use backblaze backups. See https://github.com/mail-in-a-box/mailinabox/issues/2124 for details.
2022-06-05 09:24:32 -04:00
Ilnahro 50a5cb90bc
Include rsync to the installed basic packages (#2067)
Some VPS providers strip this package from their Ubuntu 18.04 VM images. This will help avoid errors.
2021-11-30 19:50:01 -05:00
Joshua Tauberer e884c4774f Replace HMAC-based session API keys with tokens stored in memory in the daemon process
Since the session cache clears keys after a period of time, this fixes #1821.

Based on https://github.com/mail-in-a-box/mailinabox/pull/2012, and so:

Co-Authored-By: NewbieOrange <NewbieOrange@users.noreply.github.com>

Also fixes #2029 by not revealing through the login failure error message whether a user exists or not.
2021-09-06 09:23:58 -04:00
Joshua Tauberer 9b07d86bf7 Use $(...) notation instead of legacy backtick notation for embedded shell commands
shellcheck reported

    SC2006: Use $(...) notation instead of legacy backticked `...`.

Fixed by applying shellcheck's diff output as a patch.
2021-05-03 19:28:23 -04:00
Joshua Tauberer d36a2cc938 Enable Backblaze B2 backups
This reverts commit b1d703a5e7 and adds python3-setuptools per the first version of #1899 which fixes an installation error for the b2sdk Python package.
2021-02-28 08:04:14 -05:00
Joshua Tauberer b1d703a5e7 Disable Backblaze B2 backups until #1899 is resolved 2021-01-31 08:33:56 -05:00
Hilko 3422cc61ce
Include en_US.UTF-8 locale in daemon startup (#1883)
Fixes #1881.
2020-12-19 19:11:58 -05:00
Hilko 8664afa997
Implement Backblaze for Backup (#1812)
* Installing b2sdk for b2 support
* Added Duplicity PPA so the most recent version is used
* Implemented list_target_files for b2
* Implemented b2 in frontend
* removed python2 boto package
2020-11-26 07:13:31 -05:00
Felix Spöttel f205c48564 Use pyotp for validating TOTP codes
* also implements resynchronisation support via `pyotp`'s `valid_window option
2020-09-02 19:12:15 +02:00
Felix Spöttel a7a66929aa add user interface for managing 2fa
* update user schema with 2fa columns
2020-09-02 16:48:23 +02:00
Joshua Tauberer 10bedad3a3 MTA-STS tweaks, add status check using postfix-mta-sts-resolver, change to enforce 2020-05-29 15:36:52 -04:00
clonejo 8fe33da85d Run nightly tasks on a random minute after 03:00 to avoid overload (#1754)
- The MIAB version check regularly fails at 03:00, presumably because a
  large portion of installations is checking mailinabox.email at the same
  time.
- At installation time, the time of the nightly clock is configured to
  run at a random minute after 03:00, but before 04:00.
- Users might expect the nightly tasks to be over at a certain time and
  run their own custom tasks afterwards. This could thus interfere with
  custom backup routines.
- This breaks reproducibility of the installation process.
- Users might also be surprised by the nightly task time changing after
  updating MIAB.
2020-05-10 19:54:45 -04:00
Joshua Tauberer 8567a9b719 Fix upgrade issue broken by 802e7a1f4d 2019-12-01 17:44:12 -05:00
Carl Reinke 802e7a1f4d Copy systemd service files before linking to avoid issue with order of mounting filesystems (#1688) 2019-12-01 16:15:04 -05:00
jvolkenant 6e5ceab0f8 hide virtualenv output (#1578) 2019-05-15 11:59:32 -07:00
Joshua Tauberer 149552f79b systemctl link should use -f to avoid an error if a system service already exists with that name but points to a different file
https://discourse.mailinabox.email/t/new-error-failed-systemctl-link-conf-mailinabox-service/4626/2
2019-02-26 18:16:26 -05:00
jvolkenant c9b3d88108 Fixes #1437 - package python-virtualenv is now called just virtualenv (#1452) 2018-10-24 17:20:48 -04:00
Joshua Tauberer f6a641ad23 remove some cleanup steps that are no longer needed since we aren't supporting upgrades of existing machines and, even if we did, we aren't supporting upgrades from really old versions of Mail-in-a-Box 2018-10-03 13:00:15 -04:00
Christopher A. DeFlumeri d96613b8fe minimal changeset to get things working on 18.04
@joshdata squashed pull request #1398, removed some comments, and added these notes:

* The old init.d script for the management daemon is replaced with a systemd service.
* A systemd service configuration is added to configure permissions for munin on startup.
* nginx SSL settings are updated because nginx's options and defaults have changed, and we now enable http2.
* Automatic SSHFP record generation is updated to know that 22 is the default SSH daemon port, since it is no longer explicit in sshd_config.
* The dovecot-lucene package is dropped because the Mail-in-a-Box PPA where we built the package has not been updated for Ubuntu 18.04.
* The stock postgrey package is installed instead of the one from our PPA (which we no longer support), which loses the automatic whitelisting of DNSWL.org-whitelisted senders.
* Drop memcached and the status check for memcached, which we used to use with ownCloud long ago but are no longer installing.
* Other minor changes.
2018-10-03 13:00:06 -04:00
Joshua Tauberer 2a72c800f6 replace free_tls_certificates with certbot 2018-06-29 16:46:21 -04:00
Joshua Tauberer f9a0e39cc9 cryptography is now distributed as a wheel and no longer needs system development packages to be installed or pip/setuptools workarounds 2018-06-29 16:46:21 -04:00
Joshua Tauberer 35fed8606e only spawn one process for the management daemon
In 0088fb4553 I changed the management daemon's startup
script from a symlink to a Python script to a bash script that activated the new virtualenv
and then launched Python. As a result, the init.d script that starts the daemon would
write the pid of bash to the pidfile, and when trying to kill it, it would kill bash but
not the Python process.

Using exec to start Python fixes this problem by making the Python process have the pid
that the init.d script knows about.

fixes #1339
2018-01-28 09:08:19 -05:00
Joshua Tauberer e7150e3bc6 pin acme to v0.20, which is the last version compatible with free_tls_certificates
free_tls_certificates uses acme.jose, which in acme v0.21 was moved to a new Python package.

See #1328
2018-01-20 11:23:45 -05:00
Joshua Tauberer 0088fb4553 install Python 3 packages in a virtualenv
The cryptography package has created all sorts of installation trouble over the last few years, probably because of mismatches between OS-installed packages and pip-installed packages. Using a virtualenv for all Python packages used by the management daemon should make sure everything is consistent.

See #1298, see #1264.
2018-01-15 13:27:04 -05:00
Michael Kroes e49c99890b fetch whole bootstrap - fixes missing icons in admin (#1185) 2017-05-31 07:36:17 -04:00
Git Repository 18f1689f45 changed the location we store the web-assets for the admin pages to /usr/local/mailinabox (#1179) 2017-05-23 19:22:53 -04:00
Git Repository 8234a5a9f4 download jQuery and Bootstrap during setup and serve locally so that we don't rely on a CDN which is blocked in some parts of the world (#1167) (#1171) 2017-05-08 07:25:16 -04:00
Norman S f6b20a810f Enforce pip to use python 2.7 for boto (#1093) 2017-02-10 09:44:40 -05:00
Joshua Tauberer 2c86fa3755 merge v0.21c hot fix release 2017-02-01 11:26:32 -05:00
Joshua Tauberer 2e00530944 upgrade acme package 2017-02-01 11:01:11 -05:00
Joshua Tauberer 32d6728dc9 fix pip breaking due to setuptools/pip/cryptography problem
pip<6.1 + setuptools>=34 have a problem with packages that
try to update setuptools during installation, like cryptography.
See https://github.com/pypa/pip/issues/4253. The Ubuntu 14.04
package versions are pip 1.5.4 and setuptools 3.3. When we
install cryptography under those versions, it tries to update
setuptools to version 34, which became available about 10 days
ago, and then pip gets permanently broken with errors like
"ImportError: No module named 'packaging'".

The easiest work-around on systems that aren't already broken is
to upgrade pip and setuptools individually before we install any
package that tries to update setuptools.

Also try to detect a broken system and forcibly remove setuptools
first before trying to install/upgrade pip.

fixes #1080, fixes #1081, fixes #1086
see #1083
see https://discourse.mailinabox.email/t/error-with-pip-and-python/1880
see https://discourse.mailinabox.email/t/error-installing-mib/1875
2017-02-01 10:29:28 -05:00
Joshua Tauberer a081d04082 move the custom exclusive process code from utils.py into a new python package named exclusiveprocess 2017-01-15 11:02:23 -05:00
Joshua Tauberer 86368ed165 clean up apt_install lines and comments in setup/management.sh 2016-02-18 06:59:38 -05:00
Joshua Tauberer 5e4c0ed825 Revert "install boto (py2) via the package manager, not pip (used by duplicity)"
This reverts commit b32cb6229b.

Fixes #627. Fixes #653. Closes #714.
2016-02-18 06:54:23 -05:00
Joshua Tauberer 098e250cc4 bump free_tls_certificates, fixes #695, if a challenge fails dont cache it permanently (or at all) 2016-02-16 09:08:58 -05:00
Joshua Tauberer b6933a73fa provision and install free SSL certificates from Let's Encrypt 2016-01-04 18:43:16 -05:00
Joshua Tauberer 06a0e7f3fe merge #584 - Add checks to the management interface to report memory usage 2016-01-01 18:13:21 -05:00
Joshua Tauberer 4305a71916 merge #587 - move backup and nightly status checks to 3am in system time
previously these were run in a cron.daily script which per crontab is run at 6:25 am local time
2015-12-26 08:42:58 -05:00
Joshua Tauberer a4d8e12fd7 clean up the backup time patch: dont choose timezone here, move status checks into the same 3am script 2015-12-26 08:41:37 -05:00
BuildTools 8a35905d2e add timezone selection 2015-12-23 17:29:13 -05:00
Joshua Tauberer b32cb6229b install boto (py2) via the package manager, not pip (used by duplicity) 2015-11-26 14:20:59 +00:00
Michael Kroes 59f8aa1c31 Add checks to the management interface to report memory usage 2015-11-20 01:48:59 -05:00