11 Commits

Author	SHA1	Message	Date
dillard	b14b16c66c	Empty folders in backup and restore, a full disk stops a restore, progress in the Bin ... - Record folders without files in the meta record of a snapshot and restore them (#27) - Errors writing to the restore directory stop the restore, also with --salvage - Resume: check the folder before skipping, clean temporaries in folders that are complete - Clean only read-ahead folders of our own; refresh the lease on skipped files - Drop tables a failed database restore created; keep --salvage from calling a restored database fresh - Retry a pack that fails to open on the storage (SMB) - Without cron, running backups take turns from the open page - Bin: read in parts with a counter and progress bar, show read errors, refresh when a run ends	2026-11-06 15:50:02 +01:00
dillard	029b3b9cc7	Full restore goes on where it stopped; SMB recovers from a failed read ... The DR test of 1.0.2 over SMB stopped after 392 GiB of 983: the connection to the Storage Box dropped, and the smbclient extension then returns a string of length -1 for a failed read (it stores smbc_read()'s -1 in an unsigned length). The read loop counted that as data and went backwards forever at 100% CPU. Reads over SMB are now checked; a failed one is read again the usual way over a new connection. backup:restore:full that runs again (--force) leaves the data files already in place with the snapshot's size and modification time, as a backup takes them as unchanged, and reads ahead only what is still missing: after 392 GiB it goes on instead of starting over.	2026-11-05 08:18:50 +01:00
dillard	dfe1fe9aad	--salvage for backup:restore and backup:restore:full: restore what can be read from a damaged backup ... A file whose data cannot be read or decrypted is left out and listed, and the command exits with code 2. When the database cannot be restored (its dump is damaged, or it does not fit this installation), backup:restore:full leaves the fresh database as it is and puts the data directory and config of the backup in ngb-salvage-<snapshot> in the data directory. Without --salvage a restore of damaged data stops as before and points at the option.	2026-11-04 04:08:20 +01:00
dillard	a11e6c7254	1.0.1: label for "Back up now", retention explained in the trash, occ times in the schedule's zone, single-file restore to a directory, leftovers of a stopped restore removed ... - Admin page: an optional label (at most 100 characters) for a backup started with "Back up now"; an (i) by "removed by retention policy" in the trash that explains what it means and until when the restore point can be brought back. backup:trash says the same below its list. - backup:status, backup:list, backup:trash and backup:browse show times in the zone of the daily schedule and name it, instead of the server's UTC. - backup:restore --to-directory of a file restored nothing: the path got a "/" appended. - A restore removes "<name>.ngb-restore-<hex>" temporaries that a stopped restore left in the folders it writes to. - Roadmap: the cron warning (#13), reading ahead from S3/SMB and --salvage move to 1.0.2.	2026-10-30 19:32:53 +01:00
dillard	70965dd4bd	Keep leases alive during long restores and exports; PR review fixes ... Full restore, raw directory restore, per-user export and the safety export held a fixed-TTL lease on the location that nothing refreshed. A restore or export running longer than that TTL let a cleanup take over and delete data still in use. They now refresh the lease while they work (per blob, per row batch, per written chunk), throttled to once a minute. Also: - backup:prune still applies snapshot retention when user-export retention fails for a location. - The recovery kit carries the deletion delay, so a disaster recovery keeps the original protection instead of falling back to 7 days. Older kits still import with the default. - CI checkouts no longer persist the token in the git config. - recoveryKit() has its real return type; the stale baseline entry is gone.	2026-10-06 22:38:47 +02:00
dillard	f18b4f0862	Phase 2: disaster recovery (occ backup:restore:full) ... New commands, per PLAN.md 3.7: - occ backup:key:import: adopt the master key from a recovery kit on a fresh installation (instead of occ backup:key:init generating a new one), so occ backup:target:add opens the existing repository on the location rather than creating an empty one. - occ backup:restore:full: replace the database (consistent, one transaction), restore the data directory in place, and merge config.php -- refuses up front and lists what's missing when the Nextcloud major version or any app enabled at backup time doesn't match here (their tables must exist before the database restore can use them), rather than guessing or partially applying. Repository::restore() gained an optional $stripPrefix (write "data/<uid>/..." straight into the real data directory, not a "data/" subdirectory of it) -- restoring "data/" and "config/" this way needs no temporary copy of what can be the largest part of an instance. occ backup:target:add now backfills the local snapshot cache (ngb_snapshots, what occ backup:list actually reads) when it adopts an existing repository: without this, a freshly connected location showed no restore points at all, even though the repository itself had them. One unreadable snapshot is skipped rather than aborting the backfill or the target add itself. Three self-referential problems found and fixed by testing this against two real, separate Nextcloud instances sharing only the backup location (tests/integration/full-restore-roundtrip.sh covers this; the per-process nature of these bugs meant they were invisible from inside a single occ invocation, including the one used to write the fixes, which is exactly why a second, fresh process re-running occ backup:status etc. was necessary to catch them): - The full database dump includes ng_backup's own tables and appconfig rows; restoring them verbatim would overwrite the key/target that is actively performing the restore, on THIS installation, with the backed-up instance's. Both are now preserved: ng_backup's own tables (and, on PostgreSQL, their sequences -- excluding a table does not exclude its sequence) are excluded from what gets restored, and its appconfig rows are captured before the database restore and written back after, in their own transaction. - config.php's 'secret' is merged in from the backup (needed so already-encrypted content in the restored database, e.g. share tokens, stays decryptable), but Target::options, the master key, the wrapped key and the recovery kit code are all encrypted with that same instance secret (OCP's ICrypto, or Nextcloud AppConfig's own "sensitive" flag underneath), not with ng_backup's own key. Changing it without re-encrypting them first left all four permanently undecryptable the moment any later occ command ran. Fixed with two helpers (TargetService::reencryptOptionsAround(), KeyService::reencryptForSecretRotation()) that decrypt before and re-encrypt after whatever callback actually changes the secret; config.php is applied dead last in restoreFull(), after every other repository access. Not crash-safe (a process kill mid-rotation would leave a key missing, not just stale) -- disaster recovery is run once by hand, right after occ maintenance:install, a window too narrow yet to justify a two-phase write. - A subtler variant of the same bug: Nextcloud's own AppConfig additionally encrypts a "sensitive" value itself, tied to the same secret, underneath our own encryption. Overwriting an existing sensitive key makes it read the old value first to compare, which then also fails against the new secret. Deleting the key before setting the new value avoids that read entirely. Roadmap (PLAN.md 9): phase 3 (in-place full restore, user self-service) moved out to v1.3.0; phase 2 continues toward v1.2.0 with FTP/SMB, a verification/checksum-audit command, user-export retention, and deciding whether per-user export joins the automatic schedule.	2026-10-05 23:31:08 +02:00
dillard	807353cbe4	Wire Psalm into CI, fix a real bug it found in user-migration import ... RepositoryImportSource::copyToFolder() called putContent() on the generic return type of Node::get(), which doesn't declare that method -- only File does. Existing-file overwrite during a per-user restore would have fatally errored at runtime; added an instanceof check and a resource leak fix (finally-close) while there. Also guarded getFileAsStream() against fopen() returning false. Everything else Psalm (strict level) currently flags is mechanical (missing `final` on leaf classes -- applied everywhere safe) or pre-existing array-shape/type gaps, captured in psalm-baseline.xml so CI fails only on new issues from here on, not the whole backlog at once.	2026-10-05 11:45:37 +02:00
dillard	2610c1ad2f	Expiring leases instead of long-held locks ... A backup step killed by a time limit, OOM or kill -9 kept Nextcloud's lock on its location for up to an hour, blocking every new start and the run itself. Locations and runs are now guarded by leases in ngb_leases with an expiry: a backup step holds them for its duration plus two minutes, restores refresh theirs every minute, prune holds 30 minutes, untrash 5. Nextcloud's lock is only used for the milliseconds in which a lease is checked and written. Also: starting a backup while one is already running simply continues it (no exclusive lease needed), and running to completion waits instead of spinning while a crashed step's lease has not expired. Tested on NC34: a backup killed four times at random moments resumes on its own after the lease expires (about two minutes), and restores bit-identically.	2026-10-02 14:44:52 +02:00
dillard	131bb6178d	Close remaining review gaps ... - restore runs from the web are created under the location's exclusive lock, like backup runs - raw restore writes to an unpredictable temporary name opened with exclusive create (never follows a planted symlink) and validates the path prefix; the restore service rejects non-canonical paths instead of trimming them (occ normalises user input before calling it) - restore jobs report progress every minute, so a long restore is not taken for a dead one - README: Object Lock turns deletions into delete markers	2026-10-02 11:25:22 +02:00
dillard	ac82b38e37	Fixes from the phase-1 code review ... - one locking protocol per location: backup steps and restores hold a shared lock, prune and untrash an exclusive one and refuse while any run (backup, restore, export) exists; starting a backup checks and creates the run under the exclusive lock, closing the race with prune - snapshot entries are validated when read (relative path without '.', '..', NUL, backslash or control characters); raw restore also checks that the resolved directory stays below the target; the restore API and service accept canonical paths only - recovery kit confirmation requires a recent password confirmation and the confirmation code printed in the current kit file (new code per kit version), proving the kit was actually downloaded - prune decides on the authenticated index files on the location and refreshes the local index cache from them - restore job marks the run failed on any error; restore runs without progress for 6 hours are failed so they cannot block cleanup forever - tests for safe paths and a malicious snapshot path	2026-10-02 11:21:40 +02:00
dillard	c601a02b2b	M3: restore files and folders (occ backup:browse, backup:restore) ... User files are restored through the Files API so the file cache, versions and trash stay consistent: modes new-folder (default, into 'Restored <date>/<path>'), merge (overwritten files keep their old content as a version) and replace (extra files go to the trash). Each file is staged under a temporary name and moved into place only after all its blobs were verified. Any snapshot path can also be restored raw to a local directory (disaster recovery). Tested on NC34: all three modes restore checksum-identical files, the overwritten content is kept as a version, the extra file is in the trash, no temporary files remain.	2026-10-02 00:58:18 +02:00