(Updated by @JoshData from the original commit which was for Ubuntu 20.04 using PHP 7.4. And although 8.1 seems to be available, it's not supported by Nextcloud yet, and it likely will never be supported by the the version of Nextcloud that succeeds the last version of Nextcloud that supports PHP 7.2, and we have to install the next version so that an upgrade is permitted, so skipping to PHP 8.1 may not be easily possible.)
This adds a new section to the admin panel called "Activity", that
supplies charts, graphs and details about messages entering and leaving
the host.
A new daemon captures details of system mail activity by monitoring
the /var/log/mail.log file, summarizing it into a sqllite database
that's kept in user-data.
* Installing b2sdk for b2 support
* Added Duplicity PPA so the most recent version is used
* Implemented list_target_files for b2
* Implemented b2 in frontend
* removed python2 boto package
This commit will:
1. Change the user account database from sqlite to OpenLDAP
2. Add policyd-spf to postfix for SPF validation
3. Add a test runner with some automated test suites
Notes:
User account password hashes are preserved.
There is a new Roundcube contact list called "Directory" that lists the users in LDAP (MiaB users), similar to what Google Suite does.
Users can still change their password in Roundcube.
OpenLDAP is configured with TLS, but all remote access is blocked by firewall rules. Manual changes are required to open it for remote access (eg. "ufw allow proto tcp from <HOST> to any port ldaps").
The test runner is started by executing tests/runner.sh. Be aware that it will make changes to your system, including adding new users, domains, mailboxes, start/stop services, etc. It is highly unadvised to run it on a production system!
The LDAP schema that supports mail delivery with postfix and dovecot is located in conf/postfix.schema. This file is copied verbatim from the LdapAdmin project (GPL, ldapadmin.org). Instead of including the file in git, it could be referenced by URL and downloaded by the setup script if GPL is an issue or apply for a PEN from IANA.
Mangement console and other services should not appear or behave any differently than before.
removed explicit call of the system python, cause the file has a shebang with the mail-in-a-box shipped python.
for me the system python complaint, that it is missing some modules
The resulting command had nested single quotes which doesn't work
I think this fixes all/most of the issues in #1627. I am getting a full backup, then the next time it's run I get an incremental. running from the CLI with --status looks good, --verify looks good, and --list looks good.
The elif needed to check to see if the string was in the listing of results of the shell command. As it was the conditional was just the string which always evaluates to true and was therefore giving a misleading error message.
* drop the ondrej/php PPA since PHP 7.x is available directly from Ubuntu 18.04
* intall PHP 7.2 which is just the "php" package in Ubuntu 18.04
* some package names changed, some unnecessary packages are no longer provided
* update paths
The cryptography package has created all sorts of installation trouble over the last few years, probably because of mismatches between OS-installed packages and pip-installed packages. Using a virtualenv for all Python packages used by the management daemon should make sure everything is consistent.
See #1298, see #1264.
* Fixed issue with relative path for rsync relative names
Actually using the parsed URL `path` part, instead of doing a lousy split().
Renamed the `p` variable into something more sensible (`target`).
Fixes: #1019
* Added more verbose error messages upon rsync failures
fixes#1033
* Added command to test file listing
* Added support for backup to a remote server using rsync
* updated web interface to get data from user
* added way to list files from server
It’s not using the “username” field of the yaml configuration
file to minimise the amount of patches needed. So the username
is actually sorted within the rsync URL.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Added ssh key generation upon installation for root user.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Removed stale blank lines, and fixed typo
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* fix backup-location lines, by switching it from id to class
* Various web UI fixes
- fixed user field being shadowed ;
- fixed settings reading comparaison ;
- fixed forgotten min-age field.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Added SSH Public Key shown on the web interface UI
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* trailing spaces.
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* fixed the extraneous environment
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Updated key setup
- made key lower in bits, but stronger (using -a option),
- made ssh-keygen run in background using nohup,
- added independent key file, as id_rsa_miab,
- added ssh-options to all duplicity calls to use the id_rsa_miab keyfile,
- changed path to the public key display
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* added rsync options for ssh identity support
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* removed strict host checking for all backup operations
Signed-off-by: Bernard `Guyzmo` Pratz <guyzmo+github@m0g.net>
* Remove nohup from ssh-keygen so errors aren't hidden. Also only generate a key if none exists yet
* Add trailing slash when checking a remote backup. Also check if we actually can read the remote size
* Factorisation of the repeated rsync/ssh options
cf https://github.com/mail-in-a-box/mailinabox/pull/678#discussion_r81478919
* Updated message SSH key creation
https://github.com/mail-in-a-box/mailinabox/pull/678#discussion_r81478886
* going from s3 to file target wasn't working
* use 'local' in the config instead of a file: url, for the local target, so it is not path-specific
* break out the S3 fields since users can't be expected to know how to form a URL
* use boto to generate a list of S3 hosts
* use boto to validate that the user input for s3 is valid
* fix lots of html errors in the backup admin
Backup location and maximum age can now be configured in the admin panel.
For now only S3 is supported, but adding other duplicity supported backends should be straightforward.
* use the AES256 cipher, be explicit that only the first line of secret_key.txt is used, and sanity check that the passphrase is long enough
* change overship of the encrypted files to the user-data user
* simplify variable names in management/backup.py
* although I appreciate long comments I am trimming the commentary about the backup migration
* revise the control panel template to not refer to the old unencrypted files
* add CHANGELOG entry
[Josh merged some subsequent commits:]
* Guard via idempotency against termination between migration operations
* Final corrections and tweaks
* Pass passphrase through to all duplicity calls
Empirical evidence (a failed cron job) shows that cleanup requires the
passphrase (so it presumably needs to decrypt metadata), and though
remove-older-than has been working fine without it, it won't do any harm
to set it in case that changes or there are any special cases.
* Add back the archive-dir override but locate it at STORAGE_ROOT/backup/cache
Duplicity will manage the process of creating incremental backups for us.
Although duplicity can both encrypt & copy files to a remote host, I really
don't like PGP and so I don't want to use that.
Instead, we'll back up to a local directory unencrypted, then manually
encrypt the full & incremental backup files. Synchronizing the encrypted
backup directory to a remote host is a TODO.
