Commit Graph

46 Commits

Author SHA1 Message Date
Joshua Tauberer cf33be4596 fix boto 2 conflict on Google Compute Engine instances
GCE installs some Python-2-only boto plugin that conflicts with boto running under Python 3. It gives a SyntaxError in /usr/share/google/boto/boto_plugins/compute_auth.py (https://github.com/GoogleCloudPlatform/compute-image-packages).

Disabling boto's default configuration file prior to importing boto so that GCE's plugin is not loaded.

See https://discourse.mailinabox.email/t/500-internal-server-error-for-admin/942.
2015-11-26 14:51:44 +00:00
Joshua Tauberer 161d096139 add a way to dump backup status from the command line 2015-11-26 14:34:07 +00:00
yodax 280de022cb Change order in which service stop 2015-11-17 05:22:42 -05:00
yodax fa1cad7fb2 During the backup you will get login failures which will confuse iOS, so it is better to stop php-fpm as well 2015-11-17 02:57:14 -05:00
Joshua Tauberer dbfd158388 dont refresh the backup page when there's an error saving the config 2015-08-28 12:33:07 +00:00
Joshua Tauberer 2b1f7da654 S3 credentials for backup should not be displayed in the control panel, fixes #529 2015-08-28 12:33:07 +00:00
Joshua Tauberer 848dea83ab additional error handling for backups with an invalid target 2015-08-12 11:19:59 +00:00
Leo Koppelkamm f96bef43cc If no prefix is specified, set the path to '', otherwise boto won't list the files 2015-08-11 13:54:30 +02:00
Joshua Tauberer 9ca116d545 add an option to disable backups 2015-08-09 20:15:43 +00:00
Joshua Tauberer cdd3a64638 after-backup was run with the wrong environment 2015-08-09 20:08:33 +00:00
Joshua Tauberer 99e51f8a52 use boto to get actual file sizes of backup files when S3 is used 2015-08-09 20:08:33 +00:00
Joshua Tauberer 3b4b57c081 switching between backup options in the admin wasn't working at all
* going from s3 to file target wasn't working
* use 'local' in the config instead of a file: url, for the local target, so it is not path-specific
* break out the S3 fields since users can't be expected to know how to form a URL
* use boto to generate a list of S3 hosts
* use boto to validate that the user input for s3 is valid
* fix lots of html errors in the backup admin
2015-08-09 20:08:33 +00:00
Joshua Tauberer c7f8ead496 clean up the new backup configuration panel 2015-08-09 20:08:30 +00:00
Joshua Tauberer 3f15879578 remove global variables in backup.py 2015-08-09 17:54:46 +00:00
Leo Koppelkamm ba9065cada Don't write collection_status output to file but parse it directly 2015-07-27 22:30:22 +02:00
Leo Koppelkamm e693802091 Rename max_age to min_age
Also clarify a comment and remove an unneeded type check
2015-07-27 22:18:19 +02:00
Leo Koppelkamm fa0dd684da Add archive-dir argument to collection-status 2015-07-27 22:13:28 +02:00
Leo Koppelkamm 43fb7fe635 Remove unused variable 2015-07-27 22:11:43 +02:00
Leo Koppelkamm 91e4ea6e2f Infer target_type from url 2015-07-27 22:09:58 +02:00
Leo Koppelkamm 2e6c410336 Make backups more configurable
Backup location and maximum age can now be configured in the admin panel.
For now only S3 is supported, but adding other duplicity supported backends should be straightforward.
2015-07-27 21:53:34 +02:00
Joshua Tauberer 2b341d884f merge #396 - allow the backup process to work after a hostname change 2015-05-30 13:55:08 +00:00
Joshua Tauberer 141a09b31e changelog, comments for duplicity --allow-source-mismatch 2015-05-30 13:46:39 +00:00
StevesMonkey 05438d047d Fixing minor misspelling of the word: encrypted 2015-05-25 10:15:57 +09:30
Xoib 202e49a897 allow the backup process to work after a hostname change 2015-05-13 13:52:23 +02:00
Joshua Tauberer febfa72d60 race condition between backups and status checks - connection refused
At the end of the backup, wait a bit for dovecot and postfix to finish restarting.

Hopefully fixes #381.
2015-04-29 21:06:38 +00:00
Joshua Tauberer 36168b4609 add a 'backup --verify' command to run duplicity's verify command to check that the backup files are OK 2015-04-11 18:43:46 +00:00
Joshua Tauberer d8279c48ac new backup method tweaks
* use the AES256 cipher, be explicit that only the first line of secret_key.txt is used, and sanity check that the passphrase is long enough
* change overship of the encrypted files to the user-data user
* simplify variable names in management/backup.py
* although I appreciate long comments I am trimming the commentary about the backup migration
* revise the control panel template to not refer to the old unencrypted files
* add CHANGELOG entry
2015-04-11 18:32:22 +00:00
David Piggott 4232245546 Use built in duplicity encryption (GPG) for backups, closes #362, closes #363
[Josh merged some subsequent commits:]

* Guard via idempotency against termination between migration operations
* Final corrections and tweaks
* Pass passphrase through to all duplicity calls

Empirical evidence (a failed cron job) shows that cleanup requires the
passphrase (so it presumably needs to decrypt metadata), and though
remove-older-than has been working fine without it, it won't do any harm
to set it in case that changes or there are any special cases.

* Add back the archive-dir override but locate it at STORAGE_ROOT/backup/cache
2015-04-11 17:51:44 +00:00
Joshua Tauberer c18d58b13f backups: predict when the next backup will occur 2015-03-21 15:22:45 +00:00
Joshua Tauberer 86a5394f07 fix control panel when no backup has been made yet 2014-10-15 12:31:08 -04:00
Joshua Tauberer 048e35a80f fix display of backups that are past due to be reaped 2014-10-13 14:12:16 +00:00
Joshua Tauberer fb3045f456 retain backups only for 3 days; beyond that the user is responsible for copying files off of the machine 2014-10-13 14:12:11 +00:00
Joshua Tauberer 4cf53cd8ee backup status relativedelta was displaying wrong for deltas greater than 1 month 2014-10-05 18:23:29 +00:00
Joshua Tauberer 98fc449b49 only hold onto backups for 14 days (not 31) and show when the backups will be deleted in the control panel 2014-09-08 20:09:18 +00:00
Joshua Tauberer bab8b515ea new logic for determining when to take a full backup 2014-09-08 19:42:54 +00:00
Joshua Tauberer 3853e8dd93 show the status of backups in the control panel 2014-09-01 13:06:53 +00:00
Joshua Tauberer a32806da32 create STORAGE_ROOT/backup/duplicity if it doesn't exist
fixes #158
2014-08-25 23:29:00 +00:00
Joshua Tauberer 18f0406541 update comments in backup.py 2014-08-25 23:28:43 +00:00
Joshua Tauberer 1312b0254b backup: dont remove old increments because then we lose the backup history right before the last full backup, instead let them disappear along with full backups when a whole chain becomes very old 2014-08-11 11:45:40 +00:00
Joshua Tauberer f66914d634 backup: automatically take a full backup when the sum of the increments get very large 2014-08-11 11:38:32 +00:00
Joshua Tauberer 58e300e113 backup must be full on the first run because incremental backup will fail, fixes #134 2014-08-11 07:16:58 -04:00
Joshua Tauberer 6a512042dc after creating the local encrypted backup, execute the after-backup script if the user has provided one to copy the files to a remote location 2014-08-02 14:16:08 +00:00
Joshua Tauberer 8bd62aa3bc increase duplicity's volume size from the default of 25MB to 100MB so we create fewer files 2014-06-09 13:47:41 +00:00
Joshua Tauberer 5490142df5 re-do the backup script to use the duplicity program
Duplicity will manage the process of creating incremental backups for us.
Although duplicity can both encrypt & copy files to a remote host, I really
don't like PGP and so I don't want to use that.

Instead, we'll back up to a local directory unencrypted, then manually
encrypt the full & incremental backup files. Synchronizing the encrypted
backup directory to a remote host is a TODO.
2014-06-09 09:34:52 -04:00
Joshua Tauberer cecda9cec5 management: shell out external programs in a more secure way 2014-06-09 08:09:45 -04:00
Joshua Tauberer 89730bd643 new backup script, see #11 2014-06-03 21:16:38 +00:00