265 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dillard	569077fa69	Restore without the recovery kit: the key from the location, with a passphrase ... On a server without a backup key, backup:target:add finds the backups at the location and takes their key from its config with one of the old server's passphrases. Backups then wait until a new recovery kit is confirmed. The deletion delay and the repository id are now stored on the location, sealed with the key, so the delay survives a restore without a kit and --repair-config can write a lost config again from the kit. The key page warns against creating a new key when restoring a server. New test: tests/integration/recovery-scenarios.sh, and a no-kit mode in full-restore-roundtrip.sh.	2026-11-07 12:22:38 +01:00
dillard	51a4a67bc4	Heavy background jobs of other apps pause while a backup runs ... Reserved in Nextcloud's job list, so no app is turned off and a job that is running goes on. At most 6 hours in a row, then they get an hour; Nextcloud ends a reservation by itself after 12 hours, should the end of a pause never come. On by default, with a setting to turn it off.	2026-11-07 01:56:44 +01:00
dillard	a8b5e85c9b	All folders get their original modification time back on a restore ... The backup records the time of the folders with something in them, packed in the meta record, so an older version still reads the snapshot. A restore through Nextcloud leaves a folder that was already there alone. The round trip checks what Nextcloud shows, not only the disk.	2026-11-07 01:07:09 +01:00
dillard	36f2427e9a	Tell administrators about a location without a schedule ... A notification when a schedule is turned off, then a reminder: once a week while another location is backed up, every day when none is. The round trip now also checks the text of these notifications and of the one about names, and that one location never gets two runs.	2026-11-07 00:23:28 +01:00
dillard	e58b86fc48	A schedule per location, and locations backed up one after the other ... Off, once a day or every N hours from a time, each in its own time zone. Existing locations keep the time that held for all of them. A backup that has to wait shows as queued, and its restore point has the time it really began.	2026-11-07 00:04:00 +01:00
dillard	193f65940b	Pin a restore point and keep a note on it ... Both live in the catalog on the location, so a new server sees them; the database holds a local copy for the lists. The retention policy leaves a pinned restore point alone and does not count it. Available on the restore page and as occ backup:pin and backup:note.	2026-11-06 23:49:50 +01:00
dillard	6540a9314a	A name a snapshot cannot hold gets a notification of its own ... Reported apart from unreadable files, with example paths and the advice to rename them, instead of advice about permissions. Start of the 1.1.0 plan in PLAN.md.	2026-11-06 23:38:40 +01:00
Dillard Blom	f81c5e13e9	Merge pull request #30 from dillardblom/1.0.3 ... Release 1.0.3 v1.0.3	2026-11-06 20:29:09 +01:00
dillard	1b621b7149	Backup state stays valid JSON with a name that is not UTF-8; Bin reads on from an id, not a position	2026-11-06 20:02:09 +01:00
dillard	1736005648	Review of 1.0.3: names a snapshot cannot hold, restore-directory errors under salvage, browse speed ... - Leave out and report a file or folder whose name a snapshot cannot hold - A staging write failure stops a full restore under --salvage - Browse reuses its repository; empty folders read along with the entries - Keep leases alive while empty folders are created; exact-prefix empty folder is the target - Clean up tables that were partly created or before the transaction started - Bin: no empty message after a read error; longer pages	2026-11-06 19:44:52 +01:00
dillard	c7a651f269	Version 1.0.3: changelog, README status and roadmap	2026-11-06 19:05:54 +01:00
dillard	8897d8074f	Restore through Nextcloud: also a restored folder that is itself empty gets its time back	2026-11-06 18:40:10 +01:00
dillard	0b9fa1ae92	Restore through Nextcloud: empty folders get their time back	2026-11-06 18:36:15 +01:00
dillard	ee406c481e	Restore: go on after a partial write; created tables stay once the database is committed	2026-11-06 18:27:35 +01:00
dillard	dd2e1af997	Roadmap: original modification time for all folders; guide: folders reported as updated	2026-11-06 18:08:34 +01:00
dillard	d96229f370	Release the locks of a killed restore when it runs again; legacy test without empty folders ... A killed restore left its leases for up to two hours, so the first backup of the recovered server was refused. A backup from before 1.0.3 records no empty folders: the test now expects that.	2026-11-06 17:34:33 +01:00
dillard	b14b16c66c	Empty folders in backup and restore, a full disk stops a restore, progress in the Bin ... - Record folders without files in the meta record of a snapshot and restore them (#27) - Errors writing to the restore directory stop the restore, also with --salvage - Resume: check the folder before skipping, clean temporaries in folders that are complete - Clean only read-ahead folders of our own; refresh the lease on skipped files - Drop tables a failed database restore created; keep --salvage from calling a restored database fresh - Retry a pack that fails to open on the storage (SMB) - Without cron, running backups take turns from the open page - Bin: read in parts with a counter and progress bar, show read errors, refresh when a run ends	2026-11-06 15:50:02 +01:00
dillard	ce40053cec	Merge remote-tracking branch 'github/main' into 1.0.3	2026-11-06 12:23:09 +01:00
Dillard Blom	46e0186f35	Merge pull request #29 from dillardblom/polish ... Docs: --salvage after a stopped restore; a full disk during a restore	2026-11-06 12:22:58 +01:00
dillard	d5eb7abbff	Guide: after a full disk, run the restore again as in step 8	2026-11-06 12:02:38 +01:00
dillard	4c973d5cd8	Docs: a full disk during a restore looks like a damaged backup, to be fixed in 1.0.3	2026-11-06 12:00:52 +01:00
dillard	e9592cb6c6	PLAN: findings from the reviews of 1.0.2, for 1.0.3	2026-11-06 11:57:08 +01:00
dillard	b86b07a49c	Disaster recovery guide: --salvage after a restore that stopped halfway	2026-11-06 11:56:10 +01:00
dillard	ecae6e9cdd	PLAN: 1.0.2 released; a branch per version	2026-11-06 11:29:57 +01:00
Dillard Blom	d36c40e77b	Merge pull request #28 from dillardblom/release-1.0.2 ... Release 1.0.2 v1.0.2	2026-11-06 11:29:14 +01:00
dillard	50c2636926	Version 1.0.2: changelog, README status and roadmap	2026-11-06 11:03:14 +01:00
dillard	d1e9a53ce5	App store text: the line asking for a star in bold	2026-11-06 10:42:16 +01:00
dillard	b625fe7666	A restore that stops at a file gone from the location points at --salvage; test a full restore with a damaged and a missing pack ... The DR test of 1.0.2 over SMB is done: result in PLAN, empty folders as a known issue (#27).	2026-11-06 10:27:41 +01:00
dillard	74bfa7052f	PLAN: DR test 1.0.2 over SMB, findings so far	2026-11-05 08:35:46 +01:00
dillard	bf48d339e4	Remove the read-ahead folder of a killed restore with what it kept; changelog and DR guide: a stopped full restore goes on where it was	2026-11-05 08:35:28 +01:00
dillard	029b3b9cc7	Full restore goes on where it stopped; SMB recovers from a failed read ... The DR test of 1.0.2 over SMB stopped after 392 GiB of 983: the connection to the Storage Box dropped, and the smbclient extension then returns a string of length -1 for a failed read (it stores smbc_read()'s -1 in an unsigned length). The read loop counted that as data and went backwards forever at 100% CPU. Reads over SMB are now checked; a failed one is read again the usual way over a new connection. backup:restore:full that runs again (--force) leaves the data files already in place with the snapshot's size and modification time, as a backup takes them as unchanged, and reads ahead only what is still missing: after 392 GiB it goes on instead of starting over.	2026-11-05 08:18:50 +01:00
dillard	dce93aa75e	Over SMB, read index files in fewer round trips ... files_external's fopen() first stats the file and icewind's stream reads 1 MiB at a time. Open, read and close straight through the native extension's state instead: from Canada to a Hetzner Storage Box (130 ms) 0.66 instead of 1.05 s per index file, about 5 instead of 8 hours for the 27,000 index files of a 1 TB repository in backup:target:add. Anything unexpected falls back to the usual way.	2026-11-04 20:18:46 +01:00
dillard	3ec23a3072	PLAN: DR test for 1.0.2 over SMB; screenshots approved	2026-11-04 13:28:04 +01:00
dillard	4a395c75f2	PLAN: 1.0.2 status, DR test planned before the release	2026-11-04 04:22:40 +01:00
dillard	36a30732ad	Without system cron: warning and a backup that moves on while the page is open (#13); progress of the database phase; Bin tab ... - The admin page and backup:status warn when background jobs do not run on a system cron, or the cron stopped, with a link to how to set one up. - Without a working system cron, the open admin page moves a running backup forward through /api/tick, a short step per poll. - The database phase shows the tables dumped ("db: 47/183 tables"); a backup that has not moved for 15 minutes says it is waiting for background jobs. - The trash of restore points is on a tab of its own, Bin. - New screenshots; changelog and plan for 1.0.2.	2026-11-04 04:08:20 +01:00
dillard	d7c7ca093b	target:add: estimate the time left only once enough of the index is in ... The first index file also carries the time to connect, which made an S3 location say "about 642 min left" for something done within a minute.	2026-11-04 04:08:20 +01:00
dillard	05d02f3c70	Faster full restores from S3 and SMB ... S3: the packs are read ahead, eight ranges at a time, straight into files in the spool. SMB: ranges are read in 8 MiB reads through libsmbclient instead of icewind's 1 MiB reads; over a line with a 130 ms round trip, reading was four times faster in a test (18 instead of 4.5 MiB/s).	2026-11-04 04:08:20 +01:00
dillard	dfe1fe9aad	--salvage for backup:restore and backup:restore:full: restore what can be read from a damaged backup ... A file whose data cannot be read or decrypted is left out and listed, and the command exits with code 2. When the database cannot be restored (its dump is damaged, or it does not fit this installation), backup:restore:full leaves the fresh database as it is and puts the data directory and config of the backup in ngb-salvage-<snapshot> in the data directory. Without --salvage a restore of damaged data stops as before and points at the option.	2026-11-04 04:08:20 +01:00
dillard	f0293e6d01	Roadmap 1.0.2: the trash of restore points on a tab of its own (Bin)	2026-11-01 17:13:50 +01:00
dillard	f7c9894b78	PLAN: 1.0.1 released; new screenshots in 1.0.2	2026-11-01 12:59:49 +01:00
Dillard Blom	be3746e1c3	Merge pull request #26 from dillardblom/release-1.0.1 ... Release 1.0.1 v1.0.1	2026-11-01 12:58:34 +01:00
dillard	f5473495ce	App description and README: ask for a star in the app store and on GitHub	2026-11-01 12:08:01 +01:00
dillard	3694184674	Version 1.0.1: changelog, README status and roadmap	2026-11-01 12:03:19 +01:00
dillard	a0fc91a700	Integration tests: grep the whole occ output instead of grep -q, which stopped reading and failed the pipe under pipefail	2026-10-31 23:13:27 +01:00
dillard	a11e6c7254	1.0.1: label for "Back up now", retention explained in the trash, occ times in the schedule's zone, single-file restore to a directory, leftovers of a stopped restore removed ... - Admin page: an optional label (at most 100 characters) for a backup started with "Back up now"; an (i) by "removed by retention policy" in the trash that explains what it means and until when the restore point can be brought back. backup:trash says the same below its list. - backup:status, backup:list, backup:trash and backup:browse show times in the zone of the daily schedule and name it, instead of the server's UTC. - backup:restore --to-directory of a file restored nothing: the path got a "/" appended. - A restore removes "<name>.ngb-restore-<hex>" temporaries that a stopped restore left in the folders it writes to. - Roadmap: the cron warning (#13), reading ahead from S3/SMB and --salvage move to 1.0.2.	2026-10-30 19:32:53 +01:00
dillard	6f1a37f4f4	Changelog: release candidate headings carry their app version, so the app store no longer shows all of them under 1.0.0	2026-10-29 13:07:25 +01:00
dillard	5cde57bc1b	Changelog: 1.0.0 says what the app is instead of release bookkeeping; rc.6 drops its outdated outlook	2026-10-29 12:42:44 +01:00
dillard	3eec2e9554	Full restore on SQLite: take the write lock first, so another writer is waited for instead of "database is locked" ... SQLite in WAL mode starts a transaction as a reader; a write from another process between our first read and our first write makes our first write fail at once, whatever the busy timeout. The SQLite full-restore test now runs a second writer alongside the restore.	2026-10-29 03:06:05 +01:00
dillard	cdf0b70e25	Changelog: the admin page's single-line key field as the cause of #24; PLAN 1.0.1: #24, database locked at the start of a full restore on SQLite	2026-10-29 02:40:03 +01:00
dillard	66615c2a88	Merge branch 'main' into polish ... # Conflicts: # CHANGELOG.md	2026-10-29 02:27:01 +01:00