Commit Graph

7 Commits

Author SHA1 Message Date
dillard 029b3b9cc7 Full restore goes on where it stopped; SMB recovers from a failed read
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
The DR test of 1.0.2 over SMB stopped after 392 GiB of 983: the connection to the Storage Box
dropped, and the smbclient extension then returns a string of length -1 for a failed read (it
stores smbc_read()'s -1 in an unsigned length). The read loop counted that as data and went
backwards forever at 100% CPU. Reads over SMB are now checked; a failed one is read again the
usual way over a new connection.

backup:restore:full that runs again (--force) leaves the data files already in place with the
snapshot's size and modification time, as a backup takes them as unchanged, and reads ahead only
what is still missing: after 392 GiB it goes on instead of starting over.
2026-11-05 08:18:50 +01:00
dillard a11e6c7254 1.0.1: label for "Back up now", retention explained in the trash, occ times in the schedule's zone, single-file restore to a directory, leftovers of a stopped restore removed
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
- Admin page: an optional label (at most 100 characters) for a backup started with "Back up now";
  an (i) by "removed by retention policy" in the trash that explains what it means and until when
  the restore point can be brought back. backup:trash says the same below its list.
- backup:status, backup:list, backup:trash and backup:browse show times in the zone of the daily
  schedule and name it, instead of the server's UTC.
- backup:restore --to-directory of a file restored nothing: the path got a "/" appended.
- A restore removes "<name>.ngb-restore-<hex>" temporaries that a stopped restore left in the
  folders it writes to.
- Roadmap: the cron warning (#13), reading ahead from S3/SMB and --salvage move to 1.0.2.
2026-10-30 19:32:53 +01:00
dillard 9bc4e4a4ab Never create a repository where backup data lies but no config could be read 2026-10-21 22:25:48 +02:00
dillard 144097d159 Run the backup job at every cron run; skip deflate on compressed data 2026-10-15 20:59:04 +02:00
dillard 532b23482e Storage limit per location
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
S3 has no bucket quota, so NG Backup enforces one: before a pack is
uploaded, the stored size plus the pack must stay under the location's
limit, otherwise the backup stops with an error (and administrators are
notified through the failed run). Repacking during prune is exempt, as it
is how space is freed. occ backup:target:limit NAME [SIZE|none].

Also ignore credential files in the repository.
2026-10-02 14:23:25 +02:00
dillard ac82b38e37 Fixes from the phase-1 code review
- one locking protocol per location: backup steps and restores hold a
  shared lock, prune and untrash an exclusive one and refuse while any
  run (backup, restore, export) exists; starting a backup checks and
  creates the run under the exclusive lock, closing the race with prune
- snapshot entries are validated when read (relative path without '.',
  '..', NUL, backslash or control characters); raw restore also checks
  that the resolved directory stays below the target; the restore API
  and service accept canonical paths only
- recovery kit confirmation requires a recent password confirmation and
  the confirmation code printed in the current kit file (new code per
  kit version), proving the kit was actually downloaded
- prune decides on the authenticated index files on the location and
  refreshes the local index cache from them
- restore job marks the run failed on any error; restore runs without
  progress for 6 hours are failed so they cannot block cleanup forever
- tests for safe paths and a malicious snapshot path
2026-10-02 11:21:40 +02:00
dillard 9bfb6ae334 M1: PHPUnit test suite for crypto, repository and resumable runs
Dev tooling via composer (phpunit, psalm, nextcloud/ocp); vendor/ is
mounted from outside the repository by a helper script. 28 unit tests
ported from the phase-0 spikes (round trips, corruption, wrong keys,
dedup, incremental runs, partial restore, tampering, object binding,
time-boxed steps, file changed between steps).
2026-10-02 00:32:56 +02:00