278 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dillard	6ae1b2277f	Version 1.1.0: release date	2026-11-09 17:02:30 +01:00
dillard	2a72acb70b	Review fixes: the derived key is wiped also when a key is damaged, each tidy-up step after a removal on its own ... - The key derived from the passphrase is wiped on every path, also when the key or nonce turns out damaged - After a removal the pending flag and the cached index are tidied up independently; a failing log cannot stop the removal event - Tests: a short salt or nonce gives the damaged-key error	2026-11-09 02:33:50 +01:00
dillard	fb8e8d0694	Review fixes: a malformed key reads as damaged, removing a location always reports it, a key from before 1.1.0 in the recovery guide ... - Invalid base64 or a wrong salt/nonce length in a key gives the damaged-key error, so the recovery options are shown - Tidying up after a removed location cannot fail the removal anymore, so the removal event is always sent - Recovery guide: a key made before 1.1.0 counts as used and cannot make way	2026-11-09 00:55:20 +01:00
dillard	042fdae686	Review fixes: the longest deletion delay when the location's settings cannot be read, a damaged passphrase slot does not stop the next one ... - Unreadable settings or an invalid delay on the location: 365 days, so nothing leaves the Bin sooner than on the old server - A passphrase change that still cannot reach a location is logged; removing a location clears its flag - A damaged slot (missing parts, invalid base64) is skipped and the next slot is tried - Tests: damaged settings scenario, damaged slot, upgrade test checks the migration and the time zone	2026-11-08 18:29:31 +01:00
dillard	99f1655d04	Review fixes: a missed passphrase change follows at the next backup, a key from before 1.1.0 counts as used, schedules edited in their own time zone ... - A location that a passphrase change could not reach gets it at its next backup - A key from before 1.1.0 is never replaced as an unused one - An unreadable settings file on a location is told apart from a missing one - A schedule is shown and saved in the zone it was saved in - Saving an unchanged schedule keeps its moment; --timezone needs a schedule - Schedule coverage uses the time the backup began reading its data - Tests: catch-up scenario, upgrade test restores the new file, autumn clock change	2026-11-08 17:18:13 +01:00
dillard	373d7e0424	Screenshots of 1.1.0: a schedule per location, a pinned restore point with a note, passphrase slots, the Bin	2026-11-08 16:12:07 +01:00
dillard	c5024d3ddc	Version 1.1.0: changelog, README status and roadmap	2026-11-08 13:02:28 +01:00
dillard	345b1d545e	Keep the list of backups within the page: long text wraps instead of scrolling	2026-11-08 12:28:23 +01:00
dillard	98067f7746	Roadmap: what 1.1.0 brings, parallel uploads in 1.2.0	2026-11-08 12:11:15 +01:00
dillard	cd951af1a1	Test the update from a released version on a server that was backing up	2026-11-08 11:58:36 +01:00
dillard	cf9cf50286	Replace a backup key that was never used by the key of existing backups ... A key made by mistake on a server that is being restored left no way back: key:import and the passphrase route refused because a key existed, and nothing removed it. As long as the key never protected anything (no location created or adopted with it, no location or run in the database), occ backup:key:import --replace-unused-key and occ backup:target:add --replace-unused-key replace it, after the new key was opened. A key that was used is never replaced, also after its location was removed. The refusal names the option, and all administrators are notified of a replacement.	2026-11-08 11:53:55 +01:00
dillard	2b21596b3a	Plan: speed measurements, parallel uploads planned for 1.2.0	2026-11-08 11:37:13 +01:00
dillard	953312bf44	Read files for a backup in pieces of 4 MiB instead of 8 KiB ... PHP reads a file 8 KiB at a time by default. On a disk where every request waits, such as a network or cloud volume, that made reading five times slower: on the test server 5 MiB/s instead of 27 MiB/s. Without PHP's read buffer every read of a blob is one request to the disk. The same for packs on a local location.	2026-11-07 12:44:57 +01:00
dillard	569077fa69	Restore without the recovery kit: the key from the location, with a passphrase ... On a server without a backup key, backup:target:add finds the backups at the location and takes their key from its config with one of the old server's passphrases. Backups then wait until a new recovery kit is confirmed. The deletion delay and the repository id are now stored on the location, sealed with the key, so the delay survives a restore without a kit and --repair-config can write a lost config again from the kit. The key page warns against creating a new key when restoring a server. New test: tests/integration/recovery-scenarios.sh, and a no-kit mode in full-restore-roundtrip.sh.	2026-11-07 12:22:38 +01:00
dillard	51a4a67bc4	Heavy background jobs of other apps pause while a backup runs ... Reserved in Nextcloud's job list, so no app is turned off and a job that is running goes on. At most 6 hours in a row, then they get an hour; Nextcloud ends a reservation by itself after 12 hours, should the end of a pause never come. On by default, with a setting to turn it off.	2026-11-07 01:56:44 +01:00
dillard	a8b5e85c9b	All folders get their original modification time back on a restore ... The backup records the time of the folders with something in them, packed in the meta record, so an older version still reads the snapshot. A restore through Nextcloud leaves a folder that was already there alone. The round trip checks what Nextcloud shows, not only the disk.	2026-11-07 01:07:09 +01:00
dillard	36f2427e9a	Tell administrators about a location without a schedule ... A notification when a schedule is turned off, then a reminder: once a week while another location is backed up, every day when none is. The round trip now also checks the text of these notifications and of the one about names, and that one location never gets two runs.	2026-11-07 00:23:28 +01:00
dillard	e58b86fc48	A schedule per location, and locations backed up one after the other ... Off, once a day or every N hours from a time, each in its own time zone. Existing locations keep the time that held for all of them. A backup that has to wait shows as queued, and its restore point has the time it really began.	2026-11-07 00:04:00 +01:00
dillard	193f65940b	Pin a restore point and keep a note on it ... Both live in the catalog on the location, so a new server sees them; the database holds a local copy for the lists. The retention policy leaves a pinned restore point alone and does not count it. Available on the restore page and as occ backup:pin and backup:note.	2026-11-06 23:49:50 +01:00
dillard	6540a9314a	A name a snapshot cannot hold gets a notification of its own ... Reported apart from unreadable files, with example paths and the advice to rename them, instead of advice about permissions. Start of the 1.1.0 plan in PLAN.md.	2026-11-06 23:38:40 +01:00
Dillard Blom	f81c5e13e9	Merge pull request #30 from dillardblom/1.0.3 ... Release 1.0.3 v1.0.3	2026-11-06 20:29:09 +01:00
dillard	1b621b7149	Backup state stays valid JSON with a name that is not UTF-8; Bin reads on from an id, not a position	2026-11-06 20:02:09 +01:00
dillard	1736005648	Review of 1.0.3: names a snapshot cannot hold, restore-directory errors under salvage, browse speed ... - Leave out and report a file or folder whose name a snapshot cannot hold - A staging write failure stops a full restore under --salvage - Browse reuses its repository; empty folders read along with the entries - Keep leases alive while empty folders are created; exact-prefix empty folder is the target - Clean up tables that were partly created or before the transaction started - Bin: no empty message after a read error; longer pages	2026-11-06 19:44:52 +01:00
dillard	c7a651f269	Version 1.0.3: changelog, README status and roadmap	2026-11-06 19:05:54 +01:00
dillard	8897d8074f	Restore through Nextcloud: also a restored folder that is itself empty gets its time back	2026-11-06 18:40:10 +01:00
dillard	0b9fa1ae92	Restore through Nextcloud: empty folders get their time back	2026-11-06 18:36:15 +01:00
dillard	ee406c481e	Restore: go on after a partial write; created tables stay once the database is committed	2026-11-06 18:27:35 +01:00
dillard	dd2e1af997	Roadmap: original modification time for all folders; guide: folders reported as updated	2026-11-06 18:08:34 +01:00
dillard	d96229f370	Release the locks of a killed restore when it runs again; legacy test without empty folders ... A killed restore left its leases for up to two hours, so the first backup of the recovered server was refused. A backup from before 1.0.3 records no empty folders: the test now expects that.	2026-11-06 17:34:33 +01:00
dillard	b14b16c66c	Empty folders in backup and restore, a full disk stops a restore, progress in the Bin ... - Record folders without files in the meta record of a snapshot and restore them (#27) - Errors writing to the restore directory stop the restore, also with --salvage - Resume: check the folder before skipping, clean temporaries in folders that are complete - Clean only read-ahead folders of our own; refresh the lease on skipped files - Drop tables a failed database restore created; keep --salvage from calling a restored database fresh - Retry a pack that fails to open on the storage (SMB) - Without cron, running backups take turns from the open page - Bin: read in parts with a counter and progress bar, show read errors, refresh when a run ends	2026-11-06 15:50:02 +01:00
dillard	ce40053cec	Merge remote-tracking branch 'github/main' into 1.0.3	2026-11-06 12:23:09 +01:00
Dillard Blom	46e0186f35	Merge pull request #29 from dillardblom/polish ... Docs: --salvage after a stopped restore; a full disk during a restore	2026-11-06 12:22:58 +01:00
dillard	d5eb7abbff	Guide: after a full disk, run the restore again as in step 8	2026-11-06 12:02:38 +01:00
dillard	4c973d5cd8	Docs: a full disk during a restore looks like a damaged backup, to be fixed in 1.0.3	2026-11-06 12:00:52 +01:00
dillard	e9592cb6c6	PLAN: findings from the reviews of 1.0.2, for 1.0.3	2026-11-06 11:57:08 +01:00
dillard	b86b07a49c	Disaster recovery guide: --salvage after a restore that stopped halfway	2026-11-06 11:56:10 +01:00
dillard	ecae6e9cdd	PLAN: 1.0.2 released; a branch per version	2026-11-06 11:29:57 +01:00
Dillard Blom	d36c40e77b	Merge pull request #28 from dillardblom/release-1.0.2 ... Release 1.0.2 v1.0.2	2026-11-06 11:29:14 +01:00
dillard	50c2636926	Version 1.0.2: changelog, README status and roadmap	2026-11-06 11:03:14 +01:00
dillard	d1e9a53ce5	App store text: the line asking for a star in bold	2026-11-06 10:42:16 +01:00
dillard	b625fe7666	A restore that stops at a file gone from the location points at --salvage; test a full restore with a damaged and a missing pack ... The DR test of 1.0.2 over SMB is done: result in PLAN, empty folders as a known issue (#27).	2026-11-06 10:27:41 +01:00
dillard	74bfa7052f	PLAN: DR test 1.0.2 over SMB, findings so far	2026-11-05 08:35:46 +01:00
dillard	bf48d339e4	Remove the read-ahead folder of a killed restore with what it kept; changelog and DR guide: a stopped full restore goes on where it was	2026-11-05 08:35:28 +01:00
dillard	029b3b9cc7	Full restore goes on where it stopped; SMB recovers from a failed read ... The DR test of 1.0.2 over SMB stopped after 392 GiB of 983: the connection to the Storage Box dropped, and the smbclient extension then returns a string of length -1 for a failed read (it stores smbc_read()'s -1 in an unsigned length). The read loop counted that as data and went backwards forever at 100% CPU. Reads over SMB are now checked; a failed one is read again the usual way over a new connection. backup:restore:full that runs again (--force) leaves the data files already in place with the snapshot's size and modification time, as a backup takes them as unchanged, and reads ahead only what is still missing: after 392 GiB it goes on instead of starting over.	2026-11-05 08:18:50 +01:00
dillard	dce93aa75e	Over SMB, read index files in fewer round trips ... files_external's fopen() first stats the file and icewind's stream reads 1 MiB at a time. Open, read and close straight through the native extension's state instead: from Canada to a Hetzner Storage Box (130 ms) 0.66 instead of 1.05 s per index file, about 5 instead of 8 hours for the 27,000 index files of a 1 TB repository in backup:target:add. Anything unexpected falls back to the usual way.	2026-11-04 20:18:46 +01:00
dillard	3ec23a3072	PLAN: DR test for 1.0.2 over SMB; screenshots approved	2026-11-04 13:28:04 +01:00
dillard	4a395c75f2	PLAN: 1.0.2 status, DR test planned before the release	2026-11-04 04:22:40 +01:00
dillard	36a30732ad	Without system cron: warning and a backup that moves on while the page is open (#13); progress of the database phase; Bin tab ... - The admin page and backup:status warn when background jobs do not run on a system cron, or the cron stopped, with a link to how to set one up. - Without a working system cron, the open admin page moves a running backup forward through /api/tick, a short step per poll. - The database phase shows the tables dumped ("db: 47/183 tables"); a backup that has not moved for 15 minutes says it is waiting for background jobs. - The trash of restore points is on a tab of its own, Bin. - New screenshots; changelog and plan for 1.0.2.	2026-11-04 04:08:20 +01:00
dillard	d7c7ca093b	target:add: estimate the time left only once enough of the index is in ... The first index file also carries the time to connect, which made an S3 location say "about 642 min left" for something done within a minute.	2026-11-04 04:08:20 +01:00
dillard	05d02f3c70	Faster full restores from S3 and SMB ... S3: the packs are read ahead, eight ranges at a time, straight into files in the spool. SMB: ranges are read in 8 MiB reads through libsmbclient instead of icewind's 1 MiB reads; over a line with a 130 ms round trip, reading was four times faster in a test (18 instead of 4.5 MiB/s).	2026-11-04 04:08:20 +01:00