259 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dillard	6540a9314a	A name a snapshot cannot hold gets a notification of its own ... Reported apart from unreadable files, with example paths and the advice to rename them, instead of advice about permissions. Start of the 1.1.0 plan in PLAN.md.	2026-11-06 23:38:40 +01:00
Dillard Blom	f81c5e13e9	Merge pull request #30 from dillardblom/1.0.3 ... Release 1.0.3 v1.0.3	2026-11-06 20:29:09 +01:00
dillard	1b621b7149	Backup state stays valid JSON with a name that is not UTF-8; Bin reads on from an id, not a position	2026-11-06 20:02:09 +01:00
dillard	1736005648	Review of 1.0.3: names a snapshot cannot hold, restore-directory errors under salvage, browse speed ... - Leave out and report a file or folder whose name a snapshot cannot hold - A staging write failure stops a full restore under --salvage - Browse reuses its repository; empty folders read along with the entries - Keep leases alive while empty folders are created; exact-prefix empty folder is the target - Clean up tables that were partly created or before the transaction started - Bin: no empty message after a read error; longer pages	2026-11-06 19:44:52 +01:00
dillard	c7a651f269	Version 1.0.3: changelog, README status and roadmap	2026-11-06 19:05:54 +01:00
dillard	8897d8074f	Restore through Nextcloud: also a restored folder that is itself empty gets its time back	2026-11-06 18:40:10 +01:00
dillard	0b9fa1ae92	Restore through Nextcloud: empty folders get their time back	2026-11-06 18:36:15 +01:00
dillard	ee406c481e	Restore: go on after a partial write; created tables stay once the database is committed	2026-11-06 18:27:35 +01:00
dillard	dd2e1af997	Roadmap: original modification time for all folders; guide: folders reported as updated	2026-11-06 18:08:34 +01:00
dillard	d96229f370	Release the locks of a killed restore when it runs again; legacy test without empty folders ... A killed restore left its leases for up to two hours, so the first backup of the recovered server was refused. A backup from before 1.0.3 records no empty folders: the test now expects that.	2026-11-06 17:34:33 +01:00
dillard	b14b16c66c	Empty folders in backup and restore, a full disk stops a restore, progress in the Bin ... - Record folders without files in the meta record of a snapshot and restore them (#27) - Errors writing to the restore directory stop the restore, also with --salvage - Resume: check the folder before skipping, clean temporaries in folders that are complete - Clean only read-ahead folders of our own; refresh the lease on skipped files - Drop tables a failed database restore created; keep --salvage from calling a restored database fresh - Retry a pack that fails to open on the storage (SMB) - Without cron, running backups take turns from the open page - Bin: read in parts with a counter and progress bar, show read errors, refresh when a run ends	2026-11-06 15:50:02 +01:00
dillard	ce40053cec	Merge remote-tracking branch 'github/main' into 1.0.3	2026-11-06 12:23:09 +01:00
Dillard Blom	46e0186f35	Merge pull request #29 from dillardblom/polish ... Docs: --salvage after a stopped restore; a full disk during a restore	2026-11-06 12:22:58 +01:00
dillard	d5eb7abbff	Guide: after a full disk, run the restore again as in step 8	2026-11-06 12:02:38 +01:00
dillard	4c973d5cd8	Docs: a full disk during a restore looks like a damaged backup, to be fixed in 1.0.3	2026-11-06 12:00:52 +01:00
dillard	e9592cb6c6	PLAN: findings from the reviews of 1.0.2, for 1.0.3	2026-11-06 11:57:08 +01:00
dillard	b86b07a49c	Disaster recovery guide: --salvage after a restore that stopped halfway	2026-11-06 11:56:10 +01:00
dillard	ecae6e9cdd	PLAN: 1.0.2 released; a branch per version	2026-11-06 11:29:57 +01:00
Dillard Blom	d36c40e77b	Merge pull request #28 from dillardblom/release-1.0.2 ... Release 1.0.2 v1.0.2	2026-11-06 11:29:14 +01:00
dillard	50c2636926	Version 1.0.2: changelog, README status and roadmap	2026-11-06 11:03:14 +01:00
dillard	d1e9a53ce5	App store text: the line asking for a star in bold	2026-11-06 10:42:16 +01:00
dillard	b625fe7666	A restore that stops at a file gone from the location points at --salvage; test a full restore with a damaged and a missing pack ... The DR test of 1.0.2 over SMB is done: result in PLAN, empty folders as a known issue (#27).	2026-11-06 10:27:41 +01:00
dillard	74bfa7052f	PLAN: DR test 1.0.2 over SMB, findings so far	2026-11-05 08:35:46 +01:00
dillard	bf48d339e4	Remove the read-ahead folder of a killed restore with what it kept; changelog and DR guide: a stopped full restore goes on where it was	2026-11-05 08:35:28 +01:00
dillard	029b3b9cc7	Full restore goes on where it stopped; SMB recovers from a failed read ... The DR test of 1.0.2 over SMB stopped after 392 GiB of 983: the connection to the Storage Box dropped, and the smbclient extension then returns a string of length -1 for a failed read (it stores smbc_read()'s -1 in an unsigned length). The read loop counted that as data and went backwards forever at 100% CPU. Reads over SMB are now checked; a failed one is read again the usual way over a new connection. backup:restore:full that runs again (--force) leaves the data files already in place with the snapshot's size and modification time, as a backup takes them as unchanged, and reads ahead only what is still missing: after 392 GiB it goes on instead of starting over.	2026-11-05 08:18:50 +01:00
dillard	dce93aa75e	Over SMB, read index files in fewer round trips ... files_external's fopen() first stats the file and icewind's stream reads 1 MiB at a time. Open, read and close straight through the native extension's state instead: from Canada to a Hetzner Storage Box (130 ms) 0.66 instead of 1.05 s per index file, about 5 instead of 8 hours for the 27,000 index files of a 1 TB repository in backup:target:add. Anything unexpected falls back to the usual way.	2026-11-04 20:18:46 +01:00
dillard	3ec23a3072	PLAN: DR test for 1.0.2 over SMB; screenshots approved	2026-11-04 13:28:04 +01:00
dillard	4a395c75f2	PLAN: 1.0.2 status, DR test planned before the release	2026-11-04 04:22:40 +01:00
dillard	36a30732ad	Without system cron: warning and a backup that moves on while the page is open (#13); progress of the database phase; Bin tab ... - The admin page and backup:status warn when background jobs do not run on a system cron, or the cron stopped, with a link to how to set one up. - Without a working system cron, the open admin page moves a running backup forward through /api/tick, a short step per poll. - The database phase shows the tables dumped ("db: 47/183 tables"); a backup that has not moved for 15 minutes says it is waiting for background jobs. - The trash of restore points is on a tab of its own, Bin. - New screenshots; changelog and plan for 1.0.2.	2026-11-04 04:08:20 +01:00
dillard	d7c7ca093b	target:add: estimate the time left only once enough of the index is in ... The first index file also carries the time to connect, which made an S3 location say "about 642 min left" for something done within a minute.	2026-11-04 04:08:20 +01:00
dillard	05d02f3c70	Faster full restores from S3 and SMB ... S3: the packs are read ahead, eight ranges at a time, straight into files in the spool. SMB: ranges are read in 8 MiB reads through libsmbclient instead of icewind's 1 MiB reads; over a line with a 130 ms round trip, reading was four times faster in a test (18 instead of 4.5 MiB/s).	2026-11-04 04:08:20 +01:00
dillard	dfe1fe9aad	--salvage for backup:restore and backup:restore:full: restore what can be read from a damaged backup ... A file whose data cannot be read or decrypted is left out and listed, and the command exits with code 2. When the database cannot be restored (its dump is damaged, or it does not fit this installation), backup:restore:full leaves the fresh database as it is and puts the data directory and config of the backup in ngb-salvage-<snapshot> in the data directory. Without --salvage a restore of damaged data stops as before and points at the option.	2026-11-04 04:08:20 +01:00
dillard	f0293e6d01	Roadmap 1.0.2: the trash of restore points on a tab of its own (Bin)	2026-11-01 17:13:50 +01:00
dillard	f7c9894b78	PLAN: 1.0.1 released; new screenshots in 1.0.2	2026-11-01 12:59:49 +01:00
Dillard Blom	be3746e1c3	Merge pull request #26 from dillardblom/release-1.0.1 ... Release 1.0.1 v1.0.1	2026-11-01 12:58:34 +01:00
dillard	f5473495ce	App description and README: ask for a star in the app store and on GitHub	2026-11-01 12:08:01 +01:00
dillard	3694184674	Version 1.0.1: changelog, README status and roadmap	2026-11-01 12:03:19 +01:00
dillard	a0fc91a700	Integration tests: grep the whole occ output instead of grep -q, which stopped reading and failed the pipe under pipefail	2026-10-31 23:13:27 +01:00
dillard	a11e6c7254	1.0.1: label for "Back up now", retention explained in the trash, occ times in the schedule's zone, single-file restore to a directory, leftovers of a stopped restore removed ... - Admin page: an optional label (at most 100 characters) for a backup started with "Back up now"; an (i) by "removed by retention policy" in the trash that explains what it means and until when the restore point can be brought back. backup:trash says the same below its list. - backup:status, backup:list, backup:trash and backup:browse show times in the zone of the daily schedule and name it, instead of the server's UTC. - backup:restore --to-directory of a file restored nothing: the path got a "/" appended. - A restore removes "<name>.ngb-restore-<hex>" temporaries that a stopped restore left in the folders it writes to. - Roadmap: the cron warning (#13), reading ahead from S3/SMB and --salvage move to 1.0.2.	2026-10-30 19:32:53 +01:00
dillard	6f1a37f4f4	Changelog: release candidate headings carry their app version, so the app store no longer shows all of them under 1.0.0	2026-10-29 13:07:25 +01:00
dillard	5cde57bc1b	Changelog: 1.0.0 says what the app is instead of release bookkeeping; rc.6 drops its outdated outlook	2026-10-29 12:42:44 +01:00
dillard	3eec2e9554	Full restore on SQLite: take the write lock first, so another writer is waited for instead of "database is locked" ... SQLite in WAL mode starts a transaction as a reader; a write from another process between our first read and our first write makes our first write fail at once, whatever the busy timeout. The SQLite full-restore test now runs a second writer alongside the restore.	2026-10-29 03:06:05 +01:00
dillard	cdf0b70e25	Changelog: the admin page's single-line key field as the cause of #24; PLAN 1.0.1: #24, database locked at the start of a full restore on SQLite	2026-10-29 02:40:03 +01:00
dillard	66615c2a88	Merge branch 'main' into polish ... # Conflicts: # CHANGELOG.md	2026-10-29 02:27:01 +01:00
Dillard Blom	cc75c7128c	Merge pull request #25 from dillardblom/release-1.0.0 ... Release 1.0.0 v1.0.0	2026-10-29 02:26:04 +01:00
dillard	a0c4abb60a	Version 1.0.0: same code as 1.0.0-rc.6; changelog, README status	2026-10-29 01:32:12 +01:00
dillard	4847ce293d	SFTP: restore the line breaks of an OpenSSH private key before use (#24) ... Nextcloud 35 reads keys with phpseclib 3, which refuses an OpenSSH-format key whose line breaks were lost ("Unable to read key"); phpseclib 2 on Nextcloud 34 read it. The key is fixed when used, so stored locations work again without re-entering it.	2026-10-28 01:41:58 +01:00
dillard	90dd97977a	CI: build the admin page on every push	2026-10-27 11:34:53 +01:00
dillard	08534975d7	Admin page: Docs tab with links to the how-to, disaster recovery guide and roadmap; drop "in beta" from the feedback line ... PLAN: 1.0.1 item 11 (Docs tab, done); 1.1.0 item 9: remove the feedback request in 1.1.0 or 1.2.0.	2026-10-27 11:23:34 +01:00
dillard	95e3dfcf6d	User restore by user id, backup:user:list (#22) ... backup:user:restore <location> <uid> only said "is not a recorded user export": the argument is the manifest path, which only backup:user:backup printed once. backup:user:list shows the exports again; a bare user id works when that user has exactly one export, otherwise the error lists them. The export is resolved before the replace confirmation. Fixes #22	2026-10-27 00:10:59 +01:00