273 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dillard	373d7e0424	Screenshots of 1.1.0: a schedule per location, a pinned restore point with a note, passphrase slots, the Bin	2026-11-08 16:12:07 +01:00
dillard	c5024d3ddc	Version 1.1.0: changelog, README status and roadmap	2026-11-08 13:02:28 +01:00
dillard	345b1d545e	Keep the list of backups within the page: long text wraps instead of scrolling	2026-11-08 12:28:23 +01:00
dillard	98067f7746	Roadmap: what 1.1.0 brings, parallel uploads in 1.2.0	2026-11-08 12:11:15 +01:00
dillard	cd951af1a1	Test the update from a released version on a server that was backing up	2026-11-08 11:58:36 +01:00
dillard	cf9cf50286	Replace a backup key that was never used by the key of existing backups ... A key made by mistake on a server that is being restored left no way back: key:import and the passphrase route refused because a key existed, and nothing removed it. As long as the key never protected anything (no location created or adopted with it, no location or run in the database), occ backup:key:import --replace-unused-key and occ backup:target:add --replace-unused-key replace it, after the new key was opened. A key that was used is never replaced, also after its location was removed. The refusal names the option, and all administrators are notified of a replacement.	2026-11-08 11:53:55 +01:00
dillard	2b21596b3a	Plan: speed measurements, parallel uploads planned for 1.2.0	2026-11-08 11:37:13 +01:00
dillard	953312bf44	Read files for a backup in pieces of 4 MiB instead of 8 KiB ... PHP reads a file 8 KiB at a time by default. On a disk where every request waits, such as a network or cloud volume, that made reading five times slower: on the test server 5 MiB/s instead of 27 MiB/s. Without PHP's read buffer every read of a blob is one request to the disk. The same for packs on a local location.	2026-11-07 12:44:57 +01:00
dillard	569077fa69	Restore without the recovery kit: the key from the location, with a passphrase ... On a server without a backup key, backup:target:add finds the backups at the location and takes their key from its config with one of the old server's passphrases. Backups then wait until a new recovery kit is confirmed. The deletion delay and the repository id are now stored on the location, sealed with the key, so the delay survives a restore without a kit and --repair-config can write a lost config again from the kit. The key page warns against creating a new key when restoring a server. New test: tests/integration/recovery-scenarios.sh, and a no-kit mode in full-restore-roundtrip.sh.	2026-11-07 12:22:38 +01:00
dillard	51a4a67bc4	Heavy background jobs of other apps pause while a backup runs ... Reserved in Nextcloud's job list, so no app is turned off and a job that is running goes on. At most 6 hours in a row, then they get an hour; Nextcloud ends a reservation by itself after 12 hours, should the end of a pause never come. On by default, with a setting to turn it off.	2026-11-07 01:56:44 +01:00
dillard	a8b5e85c9b	All folders get their original modification time back on a restore ... The backup records the time of the folders with something in them, packed in the meta record, so an older version still reads the snapshot. A restore through Nextcloud leaves a folder that was already there alone. The round trip checks what Nextcloud shows, not only the disk.	2026-11-07 01:07:09 +01:00
dillard	36f2427e9a	Tell administrators about a location without a schedule ... A notification when a schedule is turned off, then a reminder: once a week while another location is backed up, every day when none is. The round trip now also checks the text of these notifications and of the one about names, and that one location never gets two runs.	2026-11-07 00:23:28 +01:00
dillard	e58b86fc48	A schedule per location, and locations backed up one after the other ... Off, once a day or every N hours from a time, each in its own time zone. Existing locations keep the time that held for all of them. A backup that has to wait shows as queued, and its restore point has the time it really began.	2026-11-07 00:04:00 +01:00
dillard	193f65940b	Pin a restore point and keep a note on it ... Both live in the catalog on the location, so a new server sees them; the database holds a local copy for the lists. The retention policy leaves a pinned restore point alone and does not count it. Available on the restore page and as occ backup:pin and backup:note.	2026-11-06 23:49:50 +01:00
dillard	6540a9314a	A name a snapshot cannot hold gets a notification of its own ... Reported apart from unreadable files, with example paths and the advice to rename them, instead of advice about permissions. Start of the 1.1.0 plan in PLAN.md.	2026-11-06 23:38:40 +01:00
Dillard Blom	f81c5e13e9	Merge pull request #30 from dillardblom/1.0.3 ... Release 1.0.3 v1.0.3	2026-11-06 20:29:09 +01:00
dillard	1b621b7149	Backup state stays valid JSON with a name that is not UTF-8; Bin reads on from an id, not a position	2026-11-06 20:02:09 +01:00
dillard	1736005648	Review of 1.0.3: names a snapshot cannot hold, restore-directory errors under salvage, browse speed ... - Leave out and report a file or folder whose name a snapshot cannot hold - A staging write failure stops a full restore under --salvage - Browse reuses its repository; empty folders read along with the entries - Keep leases alive while empty folders are created; exact-prefix empty folder is the target - Clean up tables that were partly created or before the transaction started - Bin: no empty message after a read error; longer pages	2026-11-06 19:44:52 +01:00
dillard	c7a651f269	Version 1.0.3: changelog, README status and roadmap	2026-11-06 19:05:54 +01:00
dillard	8897d8074f	Restore through Nextcloud: also a restored folder that is itself empty gets its time back	2026-11-06 18:40:10 +01:00
dillard	0b9fa1ae92	Restore through Nextcloud: empty folders get their time back	2026-11-06 18:36:15 +01:00
dillard	ee406c481e	Restore: go on after a partial write; created tables stay once the database is committed	2026-11-06 18:27:35 +01:00
dillard	dd2e1af997	Roadmap: original modification time for all folders; guide: folders reported as updated	2026-11-06 18:08:34 +01:00
dillard	d96229f370	Release the locks of a killed restore when it runs again; legacy test without empty folders ... A killed restore left its leases for up to two hours, so the first backup of the recovered server was refused. A backup from before 1.0.3 records no empty folders: the test now expects that.	2026-11-06 17:34:33 +01:00
dillard	b14b16c66c	Empty folders in backup and restore, a full disk stops a restore, progress in the Bin ... - Record folders without files in the meta record of a snapshot and restore them (#27) - Errors writing to the restore directory stop the restore, also with --salvage - Resume: check the folder before skipping, clean temporaries in folders that are complete - Clean only read-ahead folders of our own; refresh the lease on skipped files - Drop tables a failed database restore created; keep --salvage from calling a restored database fresh - Retry a pack that fails to open on the storage (SMB) - Without cron, running backups take turns from the open page - Bin: read in parts with a counter and progress bar, show read errors, refresh when a run ends	2026-11-06 15:50:02 +01:00
dillard	ce40053cec	Merge remote-tracking branch 'github/main' into 1.0.3	2026-11-06 12:23:09 +01:00
Dillard Blom	46e0186f35	Merge pull request #29 from dillardblom/polish ... Docs: --salvage after a stopped restore; a full disk during a restore	2026-11-06 12:22:58 +01:00
dillard	d5eb7abbff	Guide: after a full disk, run the restore again as in step 8	2026-11-06 12:02:38 +01:00
dillard	4c973d5cd8	Docs: a full disk during a restore looks like a damaged backup, to be fixed in 1.0.3	2026-11-06 12:00:52 +01:00
dillard	e9592cb6c6	PLAN: findings from the reviews of 1.0.2, for 1.0.3	2026-11-06 11:57:08 +01:00
dillard	b86b07a49c	Disaster recovery guide: --salvage after a restore that stopped halfway	2026-11-06 11:56:10 +01:00
dillard	ecae6e9cdd	PLAN: 1.0.2 released; a branch per version	2026-11-06 11:29:57 +01:00
Dillard Blom	d36c40e77b	Merge pull request #28 from dillardblom/release-1.0.2 ... Release 1.0.2 v1.0.2	2026-11-06 11:29:14 +01:00
dillard	50c2636926	Version 1.0.2: changelog, README status and roadmap	2026-11-06 11:03:14 +01:00
dillard	d1e9a53ce5	App store text: the line asking for a star in bold	2026-11-06 10:42:16 +01:00
dillard	b625fe7666	A restore that stops at a file gone from the location points at --salvage; test a full restore with a damaged and a missing pack ... The DR test of 1.0.2 over SMB is done: result in PLAN, empty folders as a known issue (#27).	2026-11-06 10:27:41 +01:00
dillard	74bfa7052f	PLAN: DR test 1.0.2 over SMB, findings so far	2026-11-05 08:35:46 +01:00
dillard	bf48d339e4	Remove the read-ahead folder of a killed restore with what it kept; changelog and DR guide: a stopped full restore goes on where it was	2026-11-05 08:35:28 +01:00
dillard	029b3b9cc7	Full restore goes on where it stopped; SMB recovers from a failed read ... The DR test of 1.0.2 over SMB stopped after 392 GiB of 983: the connection to the Storage Box dropped, and the smbclient extension then returns a string of length -1 for a failed read (it stores smbc_read()'s -1 in an unsigned length). The read loop counted that as data and went backwards forever at 100% CPU. Reads over SMB are now checked; a failed one is read again the usual way over a new connection. backup:restore:full that runs again (--force) leaves the data files already in place with the snapshot's size and modification time, as a backup takes them as unchanged, and reads ahead only what is still missing: after 392 GiB it goes on instead of starting over.	2026-11-05 08:18:50 +01:00
dillard	dce93aa75e	Over SMB, read index files in fewer round trips ... files_external's fopen() first stats the file and icewind's stream reads 1 MiB at a time. Open, read and close straight through the native extension's state instead: from Canada to a Hetzner Storage Box (130 ms) 0.66 instead of 1.05 s per index file, about 5 instead of 8 hours for the 27,000 index files of a 1 TB repository in backup:target:add. Anything unexpected falls back to the usual way.	2026-11-04 20:18:46 +01:00
dillard	3ec23a3072	PLAN: DR test for 1.0.2 over SMB; screenshots approved	2026-11-04 13:28:04 +01:00
dillard	4a395c75f2	PLAN: 1.0.2 status, DR test planned before the release	2026-11-04 04:22:40 +01:00
dillard	36a30732ad	Without system cron: warning and a backup that moves on while the page is open (#13); progress of the database phase; Bin tab ... - The admin page and backup:status warn when background jobs do not run on a system cron, or the cron stopped, with a link to how to set one up. - Without a working system cron, the open admin page moves a running backup forward through /api/tick, a short step per poll. - The database phase shows the tables dumped ("db: 47/183 tables"); a backup that has not moved for 15 minutes says it is waiting for background jobs. - The trash of restore points is on a tab of its own, Bin. - New screenshots; changelog and plan for 1.0.2.	2026-11-04 04:08:20 +01:00
dillard	d7c7ca093b	target:add: estimate the time left only once enough of the index is in ... The first index file also carries the time to connect, which made an S3 location say "about 642 min left" for something done within a minute.	2026-11-04 04:08:20 +01:00
dillard	05d02f3c70	Faster full restores from S3 and SMB ... S3: the packs are read ahead, eight ranges at a time, straight into files in the spool. SMB: ranges are read in 8 MiB reads through libsmbclient instead of icewind's 1 MiB reads; over a line with a 130 ms round trip, reading was four times faster in a test (18 instead of 4.5 MiB/s).	2026-11-04 04:08:20 +01:00
dillard	dfe1fe9aad	--salvage for backup:restore and backup:restore:full: restore what can be read from a damaged backup ... A file whose data cannot be read or decrypted is left out and listed, and the command exits with code 2. When the database cannot be restored (its dump is damaged, or it does not fit this installation), backup:restore:full leaves the fresh database as it is and puts the data directory and config of the backup in ngb-salvage-<snapshot> in the data directory. Without --salvage a restore of damaged data stops as before and points at the option.	2026-11-04 04:08:20 +01:00
dillard	f0293e6d01	Roadmap 1.0.2: the trash of restore points on a tab of its own (Bin)	2026-11-01 17:13:50 +01:00
dillard	f7c9894b78	PLAN: 1.0.1 released; new screenshots in 1.0.2	2026-11-01 12:59:49 +01:00
Dillard Blom	be3746e1c3	Merge pull request #26 from dillardblom/release-1.0.1 ... Release 1.0.1 v1.0.1	2026-11-01 12:58:34 +01:00
dillard	f5473495ce	App description and README: ask for a star in the app store and on GitHub	2026-11-01 12:08:01 +01:00