Compare commits

..

33 Commits

Author SHA1 Message Date
dillard e7229626f8 Prepare the 0.1.0-alpha.1 release
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
info.xml's version field is restricted by Nextcloud's own info.xsd to plain
numeric SemVer, no pre-release suffix, so the alpha/beta/rc stage from
PLAN.md's release table lives in the git tag and CHANGELOG instead; noted
that in PLAN.md so it isn't rediscovered later. Bumped the version, wrote
an honest description for an alpha (feature-complete for phase 1, not for
production), and added CHANGELOG.md.

README: new "Storage backend experiences" section, seeded with the Hetzner
Storage Box sub-account root-path quirk found while testing a real SFTP
target (root must be "home", not "/"), open for others to extend via PR.
2026-10-05 11:45:48 +02:00
dillard 807353cbe4 Wire Psalm into CI, fix a real bug it found in user-migration import
RepositoryImportSource::copyToFolder() called putContent() on the generic
return type of Node::get(), which doesn't declare that method -- only
File does. Existing-file overwrite during a per-user restore would have
fatally errored at runtime; added an instanceof check and a resource leak
fix (finally-close) while there. Also guarded getFileAsStream() against
fopen() returning false.

Everything else Psalm (strict level) currently flags is mechanical
(missing `final` on leaf classes -- applied everywhere safe) or pre-existing
array-shape/type gaps, captured in psalm-baseline.xml so CI fails only on
new issues from here on, not the whole backlog at once.
2026-10-05 11:45:37 +02:00
dillard 48ea6bb33d CI: round trip against real S3 and SFTP backends, not just a local path
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
Added tests/integration/targets-matrix.sh, which runs the full backup/restore
round trip through files_external's AmazonS3 and SFTP backend classes (S3
via a disposable s3mock container, SFTP via atmoz/sftp), since the existing
matrix only ever exercised the local backend.

This caught a real bug: moving a snapshot to the trash read it from the
location and streamed it straight back into a write on the same connection,
which SFTP can't do reliably (phpseclib fails to interleave a read and a
write on one connection). Backends now expose a native move() and the
id-bound snapshot object is moved with that instead of a read+write copy;
the path-bound database dump still goes through decrypt/re-encrypt, since
its AEAD authentication is tied to its path.
2026-10-05 03:15:34 +02:00
dillard 9d97f9c1f1 Admin UI: locations under Settings, fix the missing restore-point date
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
Move backup locations onto the Settings tab next to the key setup,
since adding one is a one-time step like the key, not daily use. The
"Backup and restore" tab points there when no location exists yet.

The restore-point dropdown relied on NcSelect's get-option-label,
which it does not render for the selected value, so no date showed.
Precompute a displayLabel per snapshot and select on that instead,
the same way the location dropdown already does.

Signed-off-by: Dillard Blom <dillard.blom@ensembia.com>
2026-10-04 11:15:45 +02:00
dillard b6ba09b078 Admin UI: restore point dates and a settings/daily-use split
Show the date of the restore point a restore run used, both in the
backup history table and when confirming a restore. Split the admin
page into a "Backup and restore" tab (locations, backups, restore)
and a "Settings" tab (key, slots, recovery kit), so daily work no
longer sits behind the key setup at the top of the page.

Signed-off-by: Dillard Blom <dillard.blom@ensembia.com>
2026-10-02 17:11:34 +02:00
dillard 85e835bfc2 Storage limit: 80% warning and limit in the web interface
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
- after each successful backup the stored size is recorded; at 80% of a
  location's limit administrators get a notification (once a day)
- locations list shows stored size and limit; the add form has a
  'maximum size' field that explains that empty means unlimited and
  billed by the provider; PUT /api/targets/{id}/limit (password
  confirmation)

Tested on NC34 with the OVH test bucket: a backup above the limit stops,
a successful one at 81% sends the warning.
2026-10-02 14:50:16 +02:00
dillard 2610c1ad2f Expiring leases instead of long-held locks
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
A backup step killed by a time limit, OOM or kill -9 kept Nextcloud's
lock on its location for up to an hour, blocking every new start and the
run itself. Locations and runs are now guarded by leases in ngb_leases
with an expiry: a backup step holds them for its duration plus two
minutes, restores refresh theirs every minute, prune holds 30 minutes,
untrash 5. Nextcloud's lock is only used for the milliseconds in which a
lease is checked and written.

Also: starting a backup while one is already running simply continues
it (no exclusive lease needed), and running to completion waits instead
of spinning while a crashed step's lease has not expired.

Tested on NC34: a backup killed four times at random moments resumes on
its own after the lease expires (about two minutes), and restores
bit-identically.
2026-10-02 14:44:52 +02:00
dillard 532b23482e Storage limit per location
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
S3 has no bucket quota, so NG Backup enforces one: before a pack is
uploaded, the stored size plus the pack must stay under the location's
limit, otherwise the backup stops with an error (and administrators are
notified through the failed run). Repacking during prune is exempt, as it
is how space is freed. occ backup:target:limit NAME [SIZE|none].

Also ignore credential files in the repository.
2026-10-02 14:23:25 +02:00
dillard 066873a818 Plan: release stages and version numbers
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
2026-10-02 13:16:54 +02:00
dillard c54b2cebed M5: end-to-end test matrix and CI
- tests/integration/roundtrip.sh: key, kit code and confirmation,
  location, full and unchanged incremental backup, restore new-folder and
  replace (files identical, extra file in the trash), database dump and
  restore, retention, trash and restore from trash, no files_external
  errors in the log
- tests/integration/matrix.sh: fresh Nextcloud per version and database
  (SQLite, MariaDB, PostgreSQL), removes only its own containers and
  their volumes; locally green on NC32 and NC35 with all three databases
- GitHub Actions: unit tests on PHP 8.2-8.4, round trip on NC32-35 x
  SQLite/MariaDB/PostgreSQL

Fixes found by the matrix:
- files_external is no longer booted when it is disabled: only its
  classes are loaded and a private BackendService gets its backends from
  its Application class (booting registered its mount provider, which
  failed on installations where its tables were never created)
- SQLite: the database file is excluded from the file backup (it is
  covered by the logical dump and copying the live file is inconsistent)
- occ backup:key:confirm accepts --code and --confirm for scripts
2026-10-02 11:56:57 +02:00
dillard 48bb829d45 M5: resumable database dump and faster database restore
- DbDumper::dumpStep(): tables are dumped until the step's deadline in one
  REPEATABLE READ transaction and resumed at the last primary key in the
  next step; the running table checksum is carried over. A dump that fits
  in one step is fully consistent, otherwise the snapshot records
  dbConsistent=false. BackupService uses it, so the database phase no
  longer needs one long request (hosted Nextcloud, web cron)
- restore inserts 100 rows per statement and leaves tables whose checksum
  already matches the dump untouched
- spike C: 30-step dump gives the same checksums as the one-shot dump;
  restore rewrote only the 3 damaged tables of 147; 300k-row table
  restored in 14.6 s instead of 46 s
2026-10-02 11:34:59 +02:00
dillard f75ed3aed6 M5: chained catalog against rollback and hidden snapshots
Every change to the set of snapshots or trash entries writes an
encrypted catalog generation bound to the repository and its position,
with the hash of the previous generation. The server anchors the newest
generation it has seen in its app config; on open, the location must
show at least that generation and chain back to it, and every snapshot
listed must exist. A rolled-back or tampered location is refused (the
run fails and administrators are notified). Opening with the passphrase
on a new server accepts the newest valid chain.

occ backup:target:trust-current accepts a legitimately older state
(e.g. restored from the provider's snapshot) after typing the location
name; audited and announced to all administrators.

Tests: chain advances, rollback, hidden snapshot, swapped generation,
disaster recovery without anchor. Tested on NC34 by deleting the newest
catalog generation on the SFTP location.
2026-10-02 11:30:42 +02:00
dillard 131bb6178d Close remaining review gaps
- restore runs from the web are created under the location's exclusive
  lock, like backup runs
- raw restore writes to an unpredictable temporary name opened with
  exclusive create (never follows a planted symlink) and validates the
  path prefix; the restore service rejects non-canonical paths instead
  of trimming them (occ normalises user input before calling it)
- restore jobs report progress every minute, so a long restore is not
  taken for a dead one
- README: Object Lock turns deletions into delete markers
2026-10-02 11:25:22 +02:00
dillard ac82b38e37 Fixes from the phase-1 code review
- one locking protocol per location: backup steps and restores hold a
  shared lock, prune and untrash an exclusive one and refuse while any
  run (backup, restore, export) exists; starting a backup checks and
  creates the run under the exclusive lock, closing the race with prune
- snapshot entries are validated when read (relative path without '.',
  '..', NUL, backslash or control characters); raw restore also checks
  that the resolved directory stays below the target; the restore API
  and service accept canonical paths only
- recovery kit confirmation requires a recent password confirmation and
  the confirmation code printed in the current kit file (new code per
  kit version), proving the kit was actually downloaded
- prune decides on the authenticated index files on the location and
  refreshes the local index cache from them
- restore job marks the run failed on any error; restore runs without
  progress for 6 hours are failed so they cannot block cleanup forever
- tests for safe paths and a malicious snapshot path
2026-10-02 11:21:40 +02:00
dillard d3aa07b106 M5: deletion delay with trash, passphrase slots, ransomware documentation
- removed snapshots go to a trash on the location and their data is only
  freed after the deletion delay, which is chosen when the key is created
  (default 7 days, 1-365) and cannot be changed afterwards; any admin can
  bring them back (occ backup:trash, backup:trash:restore, web UI)
- up to three passphrase slots open the same master key (like LUKS), so
  one person leaving or dying does not lock the company out; slot changes
  are written to every location, announced to all admins and require a
  new recovery kit confirmation, while backups keep running
- occ backup:key:slots, backup:key:slot:add|replace|remove; key init takes
  --label and --delete-delay-days
- web UI: deletion delay and first slot at key creation, slot management,
  trash per location with restore; all behind password confirmation
- README: key, slots (safe / CTO / head of IT), and protection against
  ransomware: object lock in compliance mode (e.g. 90 days), credentials
  without delete rights, provider snapshots, off-site copy
2026-10-02 03:01:44 +02:00
dillard b9e8858d76 M5: notifications, setup check, audit log and password confirmation
- notifications to administrators for failed backup/restore runs and
  locations without a successful backup for two days (once a day)
- setup check in the admin overview: error when the key/kit is not set
  up, warning for missing, old or failed backups
- security events (recovery kit downloaded, location removed,
  append-only turned off, retention changed, snapshots forgotten) go to
  the audit log and are announced to all administrators, from the web
  and from occ; the scheduled weekly cleanup is logged only
- fix: the recovery-kit download link failed the CSRF check (412); it
  now requires a recent password confirmation instead, like creating the
  key, removing a location and restoring
2026-10-02 02:29:46 +02:00
dillard 38654317a2 M5: retention policy, prune with repacking, append-only locations
- RetentionPolicy (last/daily/weekly/monthly) and PruneService: forget
  snapshots the policy drops, delete packs nothing uses and repack mostly
  unused ones by copying encrypted blobs as-is; new packs are indexed
  before old ones are removed
- a location lock keeps prune and backups apart; append-only locations
  are never pruned (cleanup via the location's lifecycle rules)
- weekly prune from the background job; occ backup:prune [--dry-run],
  backup:retention, backup:target:append-only
- tests: forget+prune frees space and the remaining snapshot restores
  identically, repack of mixed packs, dry-run preview, retention buckets
2026-10-02 02:21:48 +02:00
dillard fabdfa6786 Ignore built CSS and the PHPUnit result cache 2026-10-02 01:10:03 +02:00
dillard 36e6a3fde7 M4: admin settings page and JSON API
- AdminApiController (admin only, CSRF-protected): status, create key,
  download recovery kit, confirm (statement accepted + exact phrase),
  backends with their usable auth mechanisms, add/test/remove locations,
  start backups, schedule, snapshots, browse, queue restores
- RestoreJob runs web-initiated restores in the background; backup and
  restore runs are kept apart in the run table
- admin section 'NG Backup' with a Vue 3 / @nextcloud/vue 9 page: key and
  recovery-kit flow (download, tick statement, 'Yes, I confirm'), location
  form built from files_external's parameter definitions, schedule, run
  list with polling, snapshot browser and restore with the three modes
- auth mechanisms that depend on a user session or on files_external's
  own stored credentials are not offered

API tested with curl on NC34 (status, backends, schedule validation,
confirmation refusal, browse, queued restore and backup completed by
cron); non-admins get 403, anonymous 401.
2026-10-02 01:09:47 +02:00
dillard c601a02b2b M3: restore files and folders (occ backup:browse, backup:restore)
User files are restored through the Files API so the file cache,
versions and trash stay consistent: modes new-folder (default, into
'Restored <date>/<path>'), merge (overwritten files keep their old
content as a version) and replace (extra files go to the trash). Each
file is staged under a temporary name and moved into place only after
all its blobs were verified. Any snapshot path can also be restored raw
to a local directory (disaster recovery).

Tested on NC34: all three modes restore checksum-identical files, the
overwritten content is kept as a version, the extra file is in the
trash, no temporary files remain.
2026-10-02 00:58:18 +02:00
dillard 370f3c55fa M2: local cache of pack indexes (ngb_index)
Opening a repository lists index/ once and downloads only index files
that are not cached; packs written by this server are cached on write,
stale cache rows are dropped. Each resumable step therefore no longer
downloads every index file from the location.
2026-10-02 00:55:46 +02:00
dillard 3f6785bcc0 M2: full backup runs (database, config, data) with occ and cron
BackupService starts a run per location (database dump in one
transaction, then config + data directory as a resumable file run),
stores the state in ngb_runs after every step, locks a run while a step
executes and records the snapshot. Previews, logs, updater backups and
per-user cache/uploads are excluded (TreeWalker now supports patterns).
BackupJob (every 5 min) starts the daily schedule and steps running runs
with a short budget under web cron and a long one under system cron.
occ: backup:run, backup:status, backup:list.

Tested on NC34/PostgreSQL: first run 128 files / 1 GiB in 64 s (443 MiB
uploaded to S3), unchanged second run 2 s / 0 bytes; a background run to
SFTP completed through cron.php.
2026-10-02 00:53:50 +02:00
dillard 4a28bf8790 M2: cursor-based resumable runs with merge-join against the parent
TreeWalker walks one or more roots in a stable segment-wise order with
exclusions and resumes after a cursor; ParentCursor merge-joins the
previous snapshot in the same order. BackupRun no longer keeps a file
list or the parent's entries in memory or in its state.
2026-10-02 00:46:31 +02:00
dillard 506854d597 M1: key management, backup locations and occ commands
- migration: ngb_targets, ngb_runs, ngb_snapshots
- KeyService: master key stored encrypted with the instance secret
  (unattended backups), passphrase-wrapped copy in every repository and
  in the recovery kit, recorded admin confirmation per key fingerprint
- TargetService: locations built from files_external backends without a
  mount; files_external is loaded through IAppManager::loadApp() and its
  enabled state is left alone; credentials encrypted with ICrypto;
  existing repositories are adopted only if made with this key, and a
  repository cannot be registered twice
- Repository: initWithKey/openWithKey with an encrypted key check
- occ: backup:key:init|kit|confirm, backup:target:backends|add|list|remove
2026-10-02 00:37:18 +02:00
dillard 9bfb6ae334 M1: PHPUnit test suite for crypto, repository and resumable runs
Dev tooling via composer (phpunit, psalm, nextcloud/ocp); vendor/ is
mounted from outside the repository by a helper script. 28 unit tests
ported from the phase-0 spikes (round trips, corruption, wrong keys,
dedup, incremental runs, partial restore, tampering, object binding,
time-boxed steps, file changed between steps).
2026-10-02 00:32:56 +02:00
dillard 7b33347dbc Phase 0: fixes from review (torn files, pending blobs, exact sequences, UTF-8, storage checks)
- detect files changing during reading or between resumable steps and
  re-read them (size/mtime/ctime/inode), spike F overwrites mid-run
- drop pending blob ids when a pack or index upload fails
- dump and restore PostgreSQL sequence state exactly, bound parameters,
  validated catalog names; spike C checks all sequences exactly
- keep invalid UTF-8 byte-exact (tagged base64)
- refuse large tables without a primary key
- StorageBackend: check write/flush/close results, validate base path
- user_migration import/export: root paths, intermediate folders,
  partial writes, read errors
- spikes: never touch an existing account, own fixtures, 512M note
2026-10-01 18:58:51 +02:00
dillard 06e55b08ea Plan: decisions after phase 0 (files_external state, per-user restore, versions) 2026-10-01 18:50:57 +02:00
dillard 93fec0e8f3 Phase 0: resumable time-boxed backup steps (spike F) and results
BackupRun is a resumable state machine: each step works until a deadline,
uploads its pack and tree segment, and returns the state; large files are
resumed at the next blob. Snapshot is written last.

Spike F: 200 MiB file resumed across steps in separate processes; with 6
random kill -9 the run completes and restores bit-identically.
spikes/RESULTS.md summarises phase 0 and the findings for the plan.
2026-10-01 18:39:26 +02:00
dillard d474050be0 Phase 0: per-user backup/restore through user_migration (spike E)
RepositoryExportDestination / RepositoryImportSource implement the
public OCP\UserMigration interfaces on top of the repository, so
user_migration's migrators (calendar, contacts, trashbin, account,
files) export straight into the encrypted repository.

Spike E (NC34): export 0.8s, unchanged second export stores nothing;
after deleting the user, import restores account, files (bit-identical),
calendar event and contact. Findings: calendars/address books come back
as 'migrated-*' with new object URIs; importing over an existing user
duplicates contacts and adds a migrated birthday calendar; creating the
user on import needs >128 MB (password_policy word list).
2026-10-01 18:36:57 +02:00
dillard bc16a186d5 Phase 0: repository on files_external storages without a mount (spike D)
ExternalStorageFactory builds storages with files_external's own
backends and auth mechanisms from an unsaved StorageConfig (as its
ConfigAdapter does), and StorageBackend adapts any IStorage to IBackend
(writeStream, seekable range reads, temp name + rename except on S3).
IBackend::put() now owns and always closes the stream, because
files_external's S3 writeStream closes it.

Spike D (NC34): S3 (SeaweedFS) and SFTP: 120 MiB backup ~6s, unchanged
run uploads nothing, restore bit-identical, no files_external mount
created, no temp files (max 12 KiB in /tmp). Works with files_external
disabled when its classes are loaded. Backup peak memory 85-89 MiB on
these targets (to investigate).
2026-10-01 18:33:15 +02:00
dillard 915e534ab2 Phase 0: logical DB dump/restore without external tools (spike C)
DbDumper reads every prefixed table through the Nextcloud connection in
one REPEATABLE READ (read-only) transaction, with keyset pagination on
the primary key, and streams each table as JSON lines into its own blob
stream (BlobWriter), so unchanged tables are deduplicated. Restore
replaces table contents in one transaction and resets PostgreSQL
sequences.

Spike C (NC34, PostgreSQL, standard image without pg_dump/pgsql ext):
300k-row table with bytea: dump 6.3s / 50 MiB peak, 94 MiB JSON -> 20 MiB
stored; unchanged second dump stores nothing; restore makes every table
checksum-identical and all 110 sequences continue above the maximum.
2026-10-01 18:26:27 +02:00
dillard 1839997972 Phase 0: repository format with packs, dedup and incremental runs (spike B)
Content-addressed repository on an IBackend (local implementation):
4 MiB blobs, deflate when it helps, each blob encrypted with its id as
associated data, ~32 MiB packs with an encrypted header, one index file
per pack, snapshots as encrypted JSON lines written last.

Spike B: 390 MiB tree, peak 54 MiB; unchanged run reads nothing; a 1 MiB
change in a 300 MiB file uploads one blob; full and partial restore are
bit-identical; tampering is detected.
2026-10-01 18:18:38 +02:00
dillard 07e3d263b4 Phase 0: skeleton, streaming encryption (spike A)
Repository skeleton (app id ng_backup, AGPL-3.0-or-later), KeyRing with
KDF subkeys, keyed blob ids and Argon2id key wrapping, and StreamCipher
(secretstream XChaCha20-Poly1305, framed, bounded memory).

Spike A: 1 GiB round trip with memory_limit=64M, peak < 2 MiB, ~160/122
MiB/s; tampering, truncation, wrong key/passphrase are rejected.
2026-10-01 18:15:24 +02:00
93 changed files with 4030 additions and 221 deletions
+62
View File
@@ -0,0 +1,62 @@
# SPDX-FileCopyrightText: 2026 Dillard Blom
# SPDX-License-Identifier: AGPL-3.0-or-later
name: CI
on:
push:
pull_request:
permissions:
contents: read
jobs:
unit:
runs-on: ubuntu-latest
strategy:
matrix:
php: ['8.2', '8.3', '8.4']
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: sodium, zip
coverage: none
- run: composer install --no-interaction --no-progress
- run: vendor/bin/phpunit -c tests/phpunit.xml
psalm:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: '8.2'
extensions: sodium, zip
coverage: none
- run: composer install --no-interaction --no-progress
- run: composer psalm
integration:
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
# restore, retention, trash) on a fresh Nextcloud per version and database.
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
nextcloud: ['32', '33', '34', '35']
database: ['sqlite', 'mariadb', 'pgsql']
steps:
- uses: actions/checkout@v4
- name: Round trip on Nextcloud ${{ matrix.nextcloud }} / ${{ matrix.database }}
run: tests/integration/matrix.sh "${{ matrix.nextcloud }}" "${{ matrix.database }}"
integration-targets:
# Same round trip, but through the real S3 (s3mock) and SFTP backends instead of a local
# path, so files_external's AmazonS3/SFTP backend and auth classes are actually exercised.
# Backend correctness does not depend on the Nextcloud version or database, so this runs once.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Round trip against S3 and SFTP
run: tests/integration/targets-matrix.sh
+3
View File
@@ -1,7 +1,10 @@
/build/ /build/
cred-*
*credentials*
/css/ /css/
/js/ /js/
/node_modules/ /node_modules/
.phpunit.result.cache .phpunit.result.cache
*.secret
/spikes/out/ /spikes/out/
/vendor/ /vendor/
+44
View File
@@ -0,0 +1,44 @@
# Changelog
All notable changes to NG Backup are documented here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning follows the stages in
`PLAN.md` section 9. The `<version>` in `appinfo/info.xml` is plain SemVer-numeric (Nextcloud's
own `info.xsd` doesn't allow a `-alpha`/`-beta`/`-rc` suffix there); the stability stage is instead
carried by the git tag, the release, and this file.
## [0.1.0-alpha.1] - 2026-10-05
First public alpha. Feature-complete for phase 1: scheduled, encrypted, incremental backups, and
restore of files and folders. **Not for production use** — test on a copy of production (dev,
test, staging). Full disaster recovery (`backup:restore:full` on a fresh installation) is phase 2
and not implemented yet.
### Added
- Encrypted (XChaCha20-Poly1305), content-addressed, deduplicated, incremental backups, streamed
from source to destination without a local copy of the data.
- Backup locations via files_external's own backend/auth classes, without ever mounting the
location: S3 (and compatible services), SFTP, WebDAV, FTP, Swift, or a local path.
- A recovery kit with up to three passphrase slots, mandatory download-and-confirm before the
first backup runs, and an optional encrypted copy by email.
- Resumable, time-boxed background jobs (database dump, file walk, upload), so a run survives
short PHP time limits and a crash is resumed, never restarted, without leaving the instance in
maintenance mode.
- Database dump and restore without external tools, via Nextcloud's own `IDBConnection`/DBAL,
streamed through the same pipeline as file data.
- Restore of files and folders via the web interface and `occ`, into a new folder, merged, or
replacing the original (with the previous content going to the trash).
- Retention policy (last/daily/weekly/monthly), prune with pack repacking, and a snapshot trash
with a configurable deletion delay.
- Append-only locations and documented recommendations against ransomware and a compromised
server (object lock, delete-less credentials, off-site copies).
- Admin settings page (locations, schedule, retention, key management) and a JSON API.
- Notifications and an admin setup check for a failed or overdue backup.
- CI: unit tests on PHP 8.2–8.4, and an end-to-end backup→restore round trip on Nextcloud 32–35 ×
SQLite/MariaDB/PostgreSQL, and against real S3 and SFTP backends (not just a local path).
### Known limitations
- No full/disaster-recovery restore yet (phase 2).
- No per-user backup/restore via user_migration yet (phase 2).
- No FTP/SMB coverage in CI yet, and no verification/checksum-audit command yet (phase 2).
- See the "Storage backend experiences" section of the README for provider-specific quirks found
so far (e.g. Hetzner Storage Box sub-accounts).
+22 -4
View File
@@ -214,8 +214,8 @@ Daarom:
### 3.8 Backup en restore per gebruiker: via user_migration ### 3.8 Backup en restore per gebruiker: via user_migration
In plaats van zelf te bepalen welke gegevens van een gebruiker meegaan, hergebruiken we In plaats van zelf te bepalen welke gegevens van een gebruiker meegaan, hergebruiken we
**user_migration** (officiële Nextcloud-app, actief onderhouden, NC29–36; staat ook op nc01). **user_migration** (officiële Nextcloud-app, actief onderhouden, NC29–36; draait ook op onze eigen productie-instantie).
Die app werkt met *migrators* die andere apps zelf aanleveren. Op nc01 hebben die er al: Die app werkt met *migrators* die andere apps zelf aanleveren. Op onze productie-instantie zijn dat er al:
calendar, dav (agenda's en contactpersonen), deck, files_trashbin, mail, settings (profiel, calendar, dav (agenda's en contactpersonen), deck, files_trashbin, mail, settings (profiel,
account, app-instellingen) en de bestanden zelf. Elke app die later een migrator toevoegt, gaat account, app-instellingen) en de bestanden zelf. Elke app die later een migrator toevoegt, gaat
automatisch mee. automatisch mee.
@@ -308,14 +308,14 @@ niet met de hunne, maar bij installatie waarschuwen als `backup` ook actief is.
| 2 | Backup/restore per gebruiker via user_migration, volledige restore op verse installatie, FTP/SMB, verificatie | 3–4 weken | | 2 | Backup/restore per gebruiker via user_migration, volledige restore op verse installatie, FTP/SMB, verificatie | 3–4 weken |
| 3 | In-place volledige restore (restore-modus), zelfbediening voor gebruikers, extra doelen | open | | 3 | In-place volledige restore (restore-modus), zelfbediening voor gebruikers, extra doelen | open |
Release in de app store pas na fase 1 en een periode van eigen gebruik op nc01 (net als occweb): Release in de app store pas na fase 1 en een periode van eigen gebruik op onze productie-instantie (net als occweb):
een backup-app die niet betrouwbaar is, is erger dan geen backup-app. een backup-app die niet betrouwbaar is, is erger dan geen backup-app.
## 8. Besluiten (2026-10-01) ## 8. Besluiten (2026-10-01)
1. **Doelgroep:** alle varianten: AIO, losse installaties én gehoste Nextcloud zonder SSH. 1. **Doelgroep:** alle varianten: AIO, losse installaties én gehoste Nextcloud zonder SSH.
2. **Licentie:** AGPL-3.0-or-later. 2. **Licentie:** AGPL-3.0-or-later.
3. **Repository:** Ensembia op Gitea + GitHub (dillardblom), zoals occweb. 3. **Repository:** eigen Gitea + GitHub (dillardblom), zoals occweb.
4. **Naam:** "NG Backup", app-id `ng_backup`, occ-voorvoegsel `backup:`. 4. **Naam:** "NG Backup", app-id `ng_backup`, occ-voorvoegsel `backup:`.
5. **Herstelkit:** verplicht downloaden + expliciete bevestiging door de beheerder (vinkje + 5. **Herstelkit:** verplicht downloaden + expliciete bevestiging door de beheerder (vinkje +
"Yes, I confirm"), zie 3.2; versleuteld per e-mail als optionele extra kopie. "Yes, I confirm"), zie 3.2; versleuteld per e-mail als optionele extra kopie.
@@ -325,6 +325,24 @@ een backup-app die niet betrouwbaar is, is erger dan geen backup-app.
export, verwijderen, opnieuw aanmaken), óf terugzetten als `<gebruiker>-bak`, zie 3.8. export, verwijderen, opnieuw aanmaken), óf terugzetten als `<gebruiker>-bak`, zie 3.8.
8. **Fase 0 afgerond (2026-10-01),** resultaten in `spikes/RESULTS.md`. 8. **Fase 0 afgerond (2026-10-01),** resultaten in `spikes/RESULTS.md`.
## 9. Releases en versies (besluit 2026-10-02)
| Versie | Betekenis |
|---|---|
| 0.1.0-alpha.N | publiek; meetesten gewaardeerd op een kopie van productie (dev, test, staging) |
| 0.9.0-beta.N | mag meedraaien op productie, naast de reguliere backup; volledige restore aanwezig en getest |
| 1.0.0-rc.N | releasekandidaten, alleen bugfixes |
| 1.0.0 | stabiel: vervanging van de reguliere backup, noodherstel bewezen |
| 1.2.0 e.v. | functies van fase 2 (restore per gebruiker, FTP/SMB, verificatie, ...) |
Volgorde: fase 1 grondig testen, reviewen en debuggen; dan nadenken over 0.1.0-alpha.1. Een
minimale volledige restore (`occ backup:restore:full` op een verse installatie) komt vóór de beta.
**Let op (ontdekt 2026-10-05):** Nextcloud's eigen `info.xsd` staat voor `<version>` alleen
`[0-9]+(\.[0-9]+){0,2}` toe — geen `-alpha`/`-beta`/`-rc`-achtervoegsel. De stabiliteitsaanduiding
uit deze tabel gaat dus niet in `info.xml` (dat blijft gewoon oplopend, bijv. `0.1.0`), maar in de
git-tag (`v0.1.0-alpha.1`), de GitHub/Gitea release (gemarkeerd als pre-release) en de CHANGELOG.
## Bijlage: lessen uit nextcloud/backup (test NC34, 2026-10-01) ## Bijlage: lessen uit nextcloud/backup (test NC34, 2026-10-01)
- Externe programma's (`pg_dump`) en extensies (`pgsql`, `mysqli`) → onbruikbaar op standaard/hosted. - Externe programma's (`pg_dump`) en extensies (`pgsql`, `mysqli`) → onbruikbaar op standaard/hosted.
+126 -3
View File
@@ -1,8 +1,131 @@
# NG Backup # NG Backup
Encrypted, incremental backups of Nextcloud with restore from the web interface. Encrypted, incremental backups of Nextcloud (database, configuration and data) with restore
Works without shell access: no external programs, no extra PHP extensions beyond sodium. from the web interface. Works without shell access: no external programs, no extra PHP
extensions beyond sodium. Runs on Nextcloud All-in-One, regular installations and hosted
Nextcloud.
**Status: phase 0 (prototype). Not usable yet.** See `PLAN.md` for the design. **Status: in development (phase 1). Not ready for production use.** See `PLAN.md` for the design.
License: AGPL-3.0-or-later. License: AGPL-3.0-or-later.
## How it works
- Backups go to a **location**: S3 (and S3-compatible storage), SFTP, WebDAV, FTP, Swift or a
local path. NG Backup uses the connection code of Nextcloud's External storage app, but never
mounts the location: it is only connected while a backup or restore runs, and it never
appears in Files, for anyone.
- Data is cut into blocks, deduplicated, compressed and encrypted (XChaCha20-Poly1305) before it
leaves the server, and streamed straight to the location: no local copy, no double disk space.
Unchanged files are not read again; only new data is uploaded.
- The database is dumped through Nextcloud's own connection, inside one consistent transaction.
- Restores go through Nextcloud, so overwritten files keep their old content as a version and
removed files go to the trash.
## Quick start
```
occ backup:key:init --label="Safe" # first passphrase, deletion delay (default 7 days)
occ backup:key:slot:add --label="CTO" # optional second and third passphrase
occ backup:key:slot:add --label="Head of IT"
occ backup:key:kit --output=/root/ng_backup-kit.json
occ backup:key:confirm
occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket=... -o key=... -o secret=...
occ backup:run
```
Or use **Administration settings → NG Backup**.
## The backup key, passphrases and the recovery kit
All backups of an installation are encrypted with one master key.
- The **server** keeps the master key, encrypted with the instance secret, so scheduled backups
and restores work without anyone typing a passphrase. Any administrator can restore from the
web interface. Consequence: whoever fully controls the server can read the backups; whoever
only gets hold of the backup location cannot.
- For a **restore on a new server** (disaster recovery) you need the **recovery kit** and **one
passphrase**. The kit contains the master key, encrypted separately with each passphrase.
- Up to **three passphrases** ("slots") can open the key, so one person leaving or dying does not
lock the company out. A suggested setup:
| Slot | Kept by |
|---|---|
| 1 | in the safe (sealed envelope) |
| 2 | the CTO |
| 3 | the head of IT |
Every holder keeps a copy of the same recovery kit and only knows their own passphrase. Using
a single slot is possible, but then that one passphrase is a single point of failure.
- When a holder leaves, give their slot a new passphrase (`occ backup:key:slot:replace`), download
the new kit and confirm it. Note: an old kit together with the old passphrase still opens the
same master key; to revoke it completely, start new locations with a new key.
- Downloading the kit, and every change to the slots, requires the administrator's password, is
written to the audit log and is announced to all administrators.
## Protection against ransomware and a compromised server
A backup is only useful if an attacker cannot destroy it. An attacker with administrator access
to Nextcloud could try to remove old restore points so that only infected data remains.
NG Backup makes this hard and visible, but **software alone can never guarantee it**: the
location must refuse deletions itself.
What NG Backup does:
- **Deletion delay.** Removed restore points first go to a trash on the location and are only
deleted after the delay chosen at installation (default 7 days, cannot be changed afterwards).
Within that time any administrator can bring them back (`occ backup:trash:restore`, or in the
web interface).
- **Notifications to all administrators** when restore points are removed, the retention is
changed, append-only is turned off, a location is removed or the recovery kit is downloaded,
and an entry in the audit log (admin_audit).
- **Append-only locations** (`occ backup:target:append-only NAME on`): NG Backup never deletes
anything there; old data is cleaned up by the location's own lifecycle rules.
- Removing a location in NG Backup only removes the connection. The backups on the location are
not touched and nothing keeps cleaning up in the background.
What you should do on the location (strongly recommended):
- **S3: enable Object Lock** on the bucket, in **compliance mode**, with a default retention of
for example **90 days**, plus versioning. Then nobody, not even with the server's credentials,
can delete or overwrite backups younger than 90 days. Combine with an append-only location in
NG Backup and a lifecycle rule that removes old versions after the lock expires.
With Object Lock, a deletion by NG Backup (or by an attacker through it) only adds a *delete
marker*: the locked data stays and can be brought back from the bucket's object versions until
the lock expires. Tested with an S3-compatible server; still to be verified on AWS itself, which
requires a checksum on uploads to locked buckets.
- Use **credentials without delete rights** for the backup location where the storage supports it.
- **SFTP / storage boxes:** enable the provider's own snapshots (e.g. daily, kept 30-90 days) and
give the backup user access to its own folder only, without a shell.
- Keep at least one location **off-site**, on a different network than the Nextcloud server.
## Restore
- **Files and folders** of a user: in the web interface or `occ backup:restore`, into a new folder
("Restored <date>"), merged into the original place, or replacing it.
- **Raw restore** of any path (for example `config/`) to a local directory: `occ backup:restore ... --to-directory=DIR`.
- **Whole server** (disaster recovery onto a fresh installation): planned for phase 2.
## Storage backend experiences
Notes from actually running a backend, beyond what's in files_external's own documentation.
Contributions welcome via PR — add what you ran into with your own provider.
- **Hetzner Storage Box, sub-account:** the SFTP backend's `root` option must be `home`, not `/`.
A sub-account's SFTP login lands in `/home` (confirmed via `pwd()` after login); writing to the
literal chroot root (`/`) fails with a generic `NET_SFTP_STATUS_FAILURE` from the SFTP server,
while the same write under `home` (or a path below it) succeeds. This affects any backend option
form that lets you set a path relative to the account's root, not just ng_backup. Not yet
verified whether this also applies to a Hetzner Storage Box **main** account (only tested with a
sub-account so far) — if you've tried a main account, please say so in a PR.
## Commands
| Command | Purpose |
|---|---|
| `backup:key:init`, `backup:key:kit`, `backup:key:confirm` | create the key, recovery kit, confirmation |
| `backup:key:slots`, `backup:key:slot:add/replace/remove` | passphrase slots |
| `backup:target:backends/add/list/remove/append-only` | backup locations |
| `backup:run`, `backup:status`, `backup:list` | back up, status, restore points |
| `backup:browse`, `backup:restore` | browse and restore |
| `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash |
+17 -2
View File
@@ -4,8 +4,12 @@
<id>ng_backup</id> <id>ng_backup</id>
<name>NG Backup</name> <name>NG Backup</name>
<summary>Encrypted, incremental backups of your Nextcloud with restore from the web interface</summary> <summary>Encrypted, incremental backups of your Nextcloud with restore from the web interface</summary>
<description><![CDATA[Work in progress, not ready for use.]]></description> <description><![CDATA[Alpha: feature-complete for scheduled, encrypted, incremental backups to
<version>0.0.5</version> S3/SFTP/WebDAV/local, with restore of files and folders from the web interface and occ. Meant for
testing on a copy of production (dev, test, staging) -- not yet for production use, and full
disaster recovery is not implemented yet. See the recovery kit section of the README before
relying on this for anything.]]></description>
<version>0.1.0</version>
<licence>agpl</licence> <licence>agpl</licence>
<author>Dillard Blom</author> <author>Dillard Blom</author>
<namespace>NgBackup</namespace> <namespace>NgBackup</namespace>
@@ -32,6 +36,17 @@
<command>OCA\NgBackup\Command\ListSnapshots</command> <command>OCA\NgBackup\Command\ListSnapshots</command>
<command>OCA\NgBackup\Command\Browse</command> <command>OCA\NgBackup\Command\Browse</command>
<command>OCA\NgBackup\Command\Restore</command> <command>OCA\NgBackup\Command\Restore</command>
<command>OCA\NgBackup\Command\Prune</command>
<command>OCA\NgBackup\Command\Retention</command>
<command>OCA\NgBackup\Command\TargetAppendOnly</command>
<command>OCA\NgBackup\Command\TargetTrustCurrent</command>
<command>OCA\NgBackup\Command\TargetLimit</command>
<command>OCA\NgBackup\Command\Trash</command>
<command>OCA\NgBackup\Command\TrashRestore</command>
<command>OCA\NgBackup\Command\KeySlots</command>
<command>OCA\NgBackup\Command\KeySlotAdd</command>
<command>OCA\NgBackup\Command\KeySlotReplace</command>
<command>OCA\NgBackup\Command\KeySlotRemove</command>
</commands> </commands>
<settings> <settings>
<admin>OCA\NgBackup\Settings\Admin</admin> <admin>OCA\NgBackup\Settings\Admin</admin>
+6
View File
@@ -11,10 +11,16 @@ return [
['name' => 'AdminApi#initKey', 'url' => '/api/key', 'verb' => 'POST'], ['name' => 'AdminApi#initKey', 'url' => '/api/key', 'verb' => 'POST'],
['name' => 'AdminApi#downloadKit', 'url' => '/api/key/kit', 'verb' => 'GET'], ['name' => 'AdminApi#downloadKit', 'url' => '/api/key/kit', 'verb' => 'GET'],
['name' => 'AdminApi#confirmKit', 'url' => '/api/key/confirm', 'verb' => 'POST'], ['name' => 'AdminApi#confirmKit', 'url' => '/api/key/confirm', 'verb' => 'POST'],
['name' => 'AdminApi#addSlot', 'url' => '/api/key/slots', 'verb' => 'POST'],
['name' => 'AdminApi#replaceSlot', 'url' => '/api/key/slots/{slot}', 'verb' => 'PUT'],
['name' => 'AdminApi#removeSlot', 'url' => '/api/key/slots/{slot}', 'verb' => 'DELETE'],
['name' => 'AdminApi#trash', 'url' => '/api/targets/{targetId}/trash', 'verb' => 'GET'],
['name' => 'AdminApi#untrash', 'url' => '/api/targets/{targetId}/trash/{snapshotId}', 'verb' => 'POST'],
['name' => 'AdminApi#backends', 'url' => '/api/backends', 'verb' => 'GET'], ['name' => 'AdminApi#backends', 'url' => '/api/backends', 'verb' => 'GET'],
['name' => 'AdminApi#addTarget', 'url' => '/api/targets', 'verb' => 'POST'], ['name' => 'AdminApi#addTarget', 'url' => '/api/targets', 'verb' => 'POST'],
['name' => 'AdminApi#removeTarget', 'url' => '/api/targets/{id}', 'verb' => 'DELETE'], ['name' => 'AdminApi#removeTarget', 'url' => '/api/targets/{id}', 'verb' => 'DELETE'],
['name' => 'AdminApi#testTarget', 'url' => '/api/targets/{id}/test', 'verb' => 'POST'], ['name' => 'AdminApi#testTarget', 'url' => '/api/targets/{id}/test', 'verb' => 'POST'],
['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'],
['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'], ['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'],
['name' => 'AdminApi#setSchedule', 'url' => '/api/schedule', 'verb' => 'PUT'], ['name' => 'AdminApi#setSchedule', 'url' => '/api/schedule', 'verb' => 'PUT'],
['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'], ['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'],
+3 -1
View File
@@ -12,7 +12,7 @@ use OCP\AppFramework\Bootstrap\IBootContext;
use OCP\AppFramework\Bootstrap\IBootstrap; use OCP\AppFramework\Bootstrap\IBootstrap;
use OCP\AppFramework\Bootstrap\IRegistrationContext; use OCP\AppFramework\Bootstrap\IRegistrationContext;
class Application extends App implements IBootstrap { final class Application extends App implements IBootstrap {
public const APP_ID = 'ng_backup'; public const APP_ID = 'ng_backup';
public function __construct() { public function __construct() {
@@ -20,6 +20,8 @@ class Application extends App implements IBootstrap {
} }
public function register(IRegistrationContext $context): void { public function register(IRegistrationContext $context): void {
$context->registerNotifierService(\OCA\NgBackup\Notification\Notifier::class);
$context->registerSetupCheck(\OCA\NgBackup\SetupCheck\BackupStatusCheck::class);
} }
public function boot(IBootContext $context): void { public function boot(IBootContext $context): void {
+1 -1
View File
@@ -7,5 +7,5 @@ declare(strict_types=1);
namespace OCA\NgBackup\Backend; namespace OCA\NgBackup\Backend;
class BackendException extends \RuntimeException { final class BackendException extends \RuntimeException {
} }
+7
View File
@@ -27,6 +27,13 @@ interface IBackend {
public function exists(string $path): bool; public function exists(string $path): bool;
/**
* Move an object within the same location (used for the trash). Backends must use a native
* rename, not a read+write copy: some storages (e.g. SFTP) cannot reliably read and write on
* the same connection at once, since the read side is a lazily-pulled stream.
*/
public function move(string $from, string $to): void;
/** @return list<string> object paths below $prefix (recursive) */ /** @return list<string> object paths below $prefix (recursive) */
public function list(string $prefix): array; public function list(string $prefix): array;
+11
View File
@@ -106,6 +106,17 @@ final class LocalBackend implements IBackend {
@unlink($this->abs($path)); @unlink($this->abs($path));
} }
public function move(string $from, string $to): void {
$target = $this->abs($to);
$dir = dirname($target);
if (!is_dir($dir) && !mkdir($dir, 0700, true) && !is_dir($dir)) {
throw new BackendException('Cannot create ' . $dir);
}
if (!rename($this->abs($from), $target)) {
throw new BackendException("Cannot move $from to $to");
}
}
private function abs(string $path): string { private function abs(string $path): string {
if (str_contains($path, '..') || str_starts_with($path, '/')) { if (str_contains($path, '..') || str_starts_with($path, '/')) {
throw new BackendException('Invalid path: ' . $path); throw new BackendException('Invalid path: ' . $path);
+8
View File
@@ -115,6 +115,14 @@ final class StorageBackend implements IBackend {
$this->storage->unlink($this->abs($path)); $this->storage->unlink($this->abs($path));
} }
public function move(string $from, string $to): void {
$target = $this->abs($to);
$this->mkdirs(dirname($target));
if (!$this->storage->rename($this->abs($from), $target)) {
throw new BackendException("Cannot move $from to $to");
}
}
private function listInto(string $dir, array &$result): void { private function listInto(string $dir, array &$result): void {
if (!$this->storage->is_dir($dir)) { if (!$this->storage->is_dir($dir)) {
return; return;
+40 -1
View File
@@ -22,10 +22,12 @@ use Psr\Log\LoggerInterface;
* With system cron a step may take minutes; with webcron/AJAX cron it is kept short so it fits * With system cron a step may take minutes; with webcron/AJAX cron it is kept short so it fits
* in a normal web request (hosted Nextcloud). * in a normal web request (hosted Nextcloud).
*/ */
class BackupJob extends TimedJob { final class BackupJob extends TimedJob {
public function __construct( public function __construct(
ITimeFactory $time, ITimeFactory $time,
private BackupService $backups, private BackupService $backups,
private \OCA\NgBackup\Service\PruneService $prune,
private \OCA\NgBackup\Service\AlertService $alerts,
private TargetService $targets, private TargetService $targets,
private RunMapper $runs, private RunMapper $runs,
private IAppConfig $appConfig, private IAppConfig $appConfig,
@@ -45,6 +47,43 @@ class BackupJob extends TimedJob {
$run = $this->backups->step($run, min($deadline, microtime(true) + $budget)); $run = $this->backups->step($run, min($deadline, microtime(true) + $budget));
} }
} }
$this->failStaleRestores();
$this->weeklyPrune();
foreach ($this->targets->list() as $t) {
$this->alerts->checkStale($t);
}
}
/** A restore run that has not moved for 6 hours is considered dead (it would block cleanup forever). */
private function failStaleRestores(): void {
foreach ($this->runs->findRunning(null, 'restore') as $run) {
if (time() - $run->getUpdatedAt() > 6 * 3600) {
$run->setStatus(\OCA\NgBackup\Db\Run::FAILED);
$run->setError('No progress for 6 hours; marked as failed');
$run->setFinishedAt(time());
$this->runs->update($run);
$this->alerts->runFailed($run, '#' . $run->getTargetId());
}
}
}
/** Once a week (after Sunday's backup), apply the retention policy to non-append-only locations. */
private function weeklyPrune(): void {
$lastWeek = $this->appConfig->getValueString(Application::APP_ID, 'last_prune_week', '');
if (date('N') !== '7' || $lastWeek === date('o-W') || $this->runs->findRunning(null, BackupService::KIND_FULL) !== []) {
return;
}
foreach ($this->targets->list() as $t) {
if ($t->getAppendOnly()) {
continue;
}
try {
$this->prune->apply($t, false, true);
} catch (\Throwable $e) {
$this->logger->warning('NG Backup: cleanup of ' . $t->getName() . ' failed: ' . $e->getMessage());
}
}
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', date('o-W'));
} }
/** Daily schedule: appconfig ng_backup/schedule_time "HH:MM" (server time); empty = no schedule. */ /** Daily schedule: appconfig ng_backup/schedule_time "HH:MM" (server time); empty = no schedule. */
+29 -8
View File
@@ -16,29 +16,46 @@ use OCP\BackgroundJob\QueuedJob;
use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface;
/** Runs a restore requested from the web interface, outside the web request. */ /** Runs a restore requested from the web interface, outside the web request. */
class RestoreJob extends QueuedJob { final class RestoreJob extends QueuedJob {
public function __construct( public function __construct(
ITimeFactory $time, ITimeFactory $time,
private RunMapper $runs, private RunMapper $runs,
private TargetService $targets, private TargetService $targets,
private RestoreService $restore, private RestoreService $restore,
private LoggerInterface $logger, private LoggerInterface $logger,
private \OCA\NgBackup\Service\AlertService $alerts,
) { ) {
parent::__construct($time); parent::__construct($time);
} }
protected function run($argument): void { protected function run($argument): void {
$run = $this->runs->find((int)$argument['run']); $runId = (int)($argument['run'] ?? 0);
try {
$run = $this->runs->find($runId);
} catch (\Throwable $e) {
$this->logger->error('NG Backup restore job: run ' . $runId . ' not found', ['exception' => $e]);
return;
}
if ($run->getStatus() !== Run::RUNNING) { if ($run->getStatus() !== Run::RUNNING) {
return; return;
} }
$req = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
$run->setPhase('restoring');
$run->setUpdatedAt(time());
$this->runs->update($run);
@set_time_limit(0);
try { try {
$result = $this->restore->restoreUserFiles($this->targets->get((string)$run->getTargetId()), $req['snapshot'], $req['path'], $req['mode']); $req = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
$run->setPhase('restoring');
$run->setUpdatedAt(time());
$run = $this->runs->update($run);
@set_time_limit(0);
$lastBeat = time();
$result = $this->restore->restoreUserFiles($this->targets->get((string)$run->getTargetId()), $req['snapshot'], $req['path'], $req['mode'],
function (int $done, int $total) use (&$run, &$lastBeat): void {
// Heartbeat, so a long restore is not mistaken for a dead one.
if (time() - $lastBeat >= 60) {
$lastBeat = time();
$run->setPhase("restoring $done/$total");
$run->setUpdatedAt(time());
$run = $this->runs->update($run);
}
});
$run->setStatus(Run::DONE); $run->setStatus(Run::DONE);
$run->setPhase('done'); $run->setPhase('done');
$run->setStats(json_encode($result, JSON_THROW_ON_ERROR)); $run->setStats(json_encode($result, JSON_THROW_ON_ERROR));
@@ -46,6 +63,10 @@ class RestoreJob extends QueuedJob {
$this->logger->error('NG Backup restore ' . $run->getId() . ' failed: ' . $e->getMessage(), ['exception' => $e]); $this->logger->error('NG Backup restore ' . $run->getId() . ' failed: ' . $e->getMessage(), ['exception' => $e]);
$run->setStatus(Run::FAILED); $run->setStatus(Run::FAILED);
$run->setError($e->getMessage()); $run->setError($e->getMessage());
try {
$this->alerts->runFailed($run, '#' . $run->getTargetId());
} catch (\Throwable) {
}
} }
$run->setFinishedAt(time()); $run->setFinishedAt(time());
$run->setUpdatedAt(time()); $run->setUpdatedAt(time());
+1 -1
View File
@@ -15,7 +15,7 @@ use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface; use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class Browse extends Command { final class Browse extends Command {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
private RestoreService $restore, private RestoreService $restore,
+10 -3
View File
@@ -14,7 +14,7 @@ use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Question\Question; use Symfony\Component\Console\Question\Question;
class KeyConfirm extends Command { final class KeyConfirm extends Command {
public const STATEMENT = 'I have downloaded the recovery kit and stored it safely, outside this server. ' public const STATEMENT = 'I have downloaded the recovery kit and stored it safely, outside this server. '
. 'I understand that keeping these keys safe is my own responsibility and that backups cannot be restored without them.'; . 'I understand that keeping these keys safe is my own responsibility and that backups cannot be restored without them.';
public const CONFIRMATION = 'Yes, I confirm'; public const CONFIRMATION = 'Yes, I confirm';
@@ -28,7 +28,9 @@ class KeyConfirm extends Command {
protected function configure(): void { protected function configure(): void {
$this->setName('backup:key:confirm') $this->setName('backup:key:confirm')
->setDescription('Confirm that the recovery kit is stored safely (required before the first backup)') ->setDescription('Confirm that the recovery kit is stored safely (required before the first backup)')
->addOption('user', 'u', InputOption::VALUE_REQUIRED, 'Admin user id recorded with the confirmation', 'occ'); ->addOption('user', 'u', InputOption::VALUE_REQUIRED, 'Admin user id recorded with the confirmation', 'occ')
->addOption('code', null, InputOption::VALUE_REQUIRED, 'Confirmation code from the kit file (for scripts; asked otherwise)')
->addOption('confirm', null, InputOption::VALUE_REQUIRED, 'The confirmation phrase "' . self::CONFIRMATION . '" (for scripts; asked otherwise)');
} }
protected function execute(InputInterface $input, OutputInterface $output): int { protected function execute(InputInterface $input, OutputInterface $output): int {
@@ -40,7 +42,12 @@ class KeyConfirm extends Command {
$output->writeln(''); $output->writeln('');
$output->writeln(wordwrap(self::STATEMENT, 78)); $output->writeln(wordwrap(self::STATEMENT, 78));
$output->writeln(''); $output->writeln('');
$answer = (string)$this->getHelper('question')->ask($input, $output, new Question('Type "' . self::CONFIRMATION . '" to proceed: ')); $code = $input->getOption('code') ?? (string)$this->getHelper('question')->ask($input, $output, new Question('Confirmation code from the recovery kit file ("confirmation_code"): '));
if (!$this->keys->checkKitCode($code)) {
$output->writeln('<error>That code does not match the current recovery kit (download it with occ backup:key:kit first).</error>');
return 1;
}
$answer = $input->getOption('confirm') ?? (string)$this->getHelper('question')->ask($input, $output, new Question('Type "' . self::CONFIRMATION . '" to proceed: '));
if (trim($answer) !== self::CONFIRMATION) { if (trim($answer) !== self::CONFIRMATION) {
$output->writeln('<error>Not confirmed.</error>'); $output->writeln('<error>Not confirmed.</error>');
return 1; return 1;
+6 -2
View File
@@ -14,7 +14,7 @@ use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Question\Question; use Symfony\Component\Console\Question\Question;
class KeyInit extends Command { final class KeyInit extends Command {
public function __construct( public function __construct(
private KeyService $keys, private KeyService $keys,
) { ) {
@@ -25,6 +25,8 @@ class KeyInit extends Command {
$this->setName('backup:key:init') $this->setName('backup:key:init')
->setDescription('Create the backup key, protected by a passphrase you keep safe') ->setDescription('Create the backup key, protected by a passphrase you keep safe')
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Read the passphrase from this file instead of asking') ->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Read the passphrase from this file instead of asking')
->addOption('label', 'l', InputOption::VALUE_REQUIRED, 'Label of the first passphrase slot (who/where), e.g. "Safe"', 'Slot 1')
->addOption('delete-delay-days', null, InputOption::VALUE_REQUIRED, 'Days a removed snapshot stays in the trash before its data is deleted (1-365, fixed after this)', (string)KeyService::DEFAULT_DELETE_DELAY)
->setHelp(<<<'HELP' ->setHelp(<<<'HELP'
Creates the master key that encrypts all backups of this installation. Creates the master key that encrypts all backups of this installation.
@@ -57,12 +59,14 @@ HELP);
} }
} }
try { try {
$this->keys->initialize($pass); $this->keys->initialize($pass, (int)$input->getOption('delete-delay-days'), (string)$input->getOption('label'));
} catch (\InvalidArgumentException $e) { } catch (\InvalidArgumentException $e) {
$output->writeln('<error>' . $e->getMessage() . '</error>'); $output->writeln('<error>' . $e->getMessage() . '</error>');
return 1; return 1;
} }
$output->writeln('Backup key created, fingerprint <info>' . $this->keys->fingerprint() . '</info>.'); $output->writeln('Backup key created, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
$output->writeln('Removed snapshots stay in the trash on the location for <info>' . $this->keys->deleteDelayDays() . ' days</info> (cannot be changed later).');
$output->writeln('Tip: add up to two more passphrases for other people with occ backup:key:slot:add (e.g. CTO, head of IT).');
$output->writeln('Next: save the recovery kit with <info>occ backup:key:kit --output=FILE</info> and confirm with <info>occ backup:key:confirm</info>.'); $output->writeln('Next: save the recovery kit with <info>occ backup:key:kit --output=FILE</info> and confirm with <info>occ backup:key:confirm</info>.');
return 0; return 0;
} }
+3 -1
View File
@@ -13,9 +13,10 @@ use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption; use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class KeyKit extends Command { final class KeyKit extends Command {
public function __construct( public function __construct(
private KeyService $keys, private KeyService $keys,
private \OCA\NgBackup\Service\AlertService $alerts,
) { ) {
parent::__construct(); parent::__construct();
} }
@@ -31,6 +32,7 @@ class KeyKit extends Command {
$output->writeln('<error>No backup key yet: run occ backup:key:init first.</error>'); $output->writeln('<error>No backup key yet: run occ backup:key:init first.</error>');
return 1; return 1;
} }
$this->alerts->securityEvent('kit_downloaded');
$json = json_encode($this->keys->recoveryKit(), JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR) . "\n"; $json = json_encode($this->keys->recoveryKit(), JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR) . "\n";
$file = $input->getOption('output'); $file = $input->getOption('output');
if ($file === null) { if ($file === null) {
+17
View File
@@ -0,0 +1,17 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\KeyService;
use OCA\NgBackup\Service\KeySlotService;
final class KeySlotAdd extends KeySlots {
public function __construct(KeyService $keys, KeySlotService $slots) {
parent::__construct($keys, $slots, 'add');
}
}
+17
View File
@@ -0,0 +1,17 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\KeyService;
use OCA\NgBackup\Service\KeySlotService;
final class KeySlotRemove extends KeySlots {
public function __construct(KeyService $keys, KeySlotService $slots) {
parent::__construct($keys, $slots, 'remove');
}
}
+17
View File
@@ -0,0 +1,17 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\KeyService;
use OCA\NgBackup\Service\KeySlotService;
final class KeySlotReplace extends KeySlots {
public function __construct(KeyService $keys, KeySlotService $slots) {
parent::__construct($keys, $slots, 'replace');
}
}
+97
View File
@@ -0,0 +1,97 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\KeyService;
use OCA\NgBackup\Service\KeySlotService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Helper\Table;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Question\Question;
/** backup:key:slots, backup:key:slot:add, backup:key:slot:replace, backup:key:slot:remove */
class KeySlots extends Command {
public function __construct(
private KeyService $keys,
private KeySlotService $slots,
private string $action = 'list',
) {
// list is the default; subclasses pick another action
parent::__construct();
}
protected function configure(): void {
$help = <<<'HELP'
Up to three passphrases ("slots") can open the backup key, so that one person leaving or dying
does not lock you out. Example: one passphrase in the safe, one with the CTO, one with the head
of IT. Each holder keeps a copy of the same recovery kit and only knows their own passphrase.
After every change: occ backup:key:kit and occ backup:key:confirm.
HELP;
match ($this->action) {
'add' => $this->setName('backup:key:slot:add')->setDescription('Add a passphrase slot')
->addOption('label', 'l', InputOption::VALUE_REQUIRED, 'Who or where, e.g. "CTO" or "Safe"', 'Slot'),
'replace' => $this->setName('backup:key:slot:replace')->setDescription('Set a new passphrase for a slot (e.g. its holder left)')
->addArgument('slot', InputArgument::REQUIRED, 'Slot number (see backup:key:slots)')
->addOption('label', 'l', InputOption::VALUE_REQUIRED, 'New label'),
'remove' => $this->setName('backup:key:slot:remove')->setDescription('Remove a passphrase slot (not the last one)')
->addArgument('slot', InputArgument::REQUIRED, 'Slot number (see backup:key:slots)'),
default => $this->setName('backup:key:slots')->setDescription('List the passphrase slots of the backup key'),
};
if ($this->action === 'add' || $this->action === 'replace') {
$this->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Read the passphrase from this file');
}
$this->setHelp($help);
}
protected function execute(InputInterface $input, OutputInterface $output): int {
if (!$this->keys->isInitialized()) {
$output->writeln('<error>No backup key yet: run occ backup:key:init first.</error>');
return 1;
}
try {
switch ($this->action) {
case 'add':
$n = $this->slots->add($this->passphrase($input, $output), (string)$input->getOption('label'));
$output->writeln("Slot $n added.");
break;
case 'replace':
$this->slots->replace((int)$input->getArgument('slot'), $this->passphrase($input, $output), $input->getOption('label'));
$output->writeln('Slot ' . (int)$input->getArgument('slot') . ' has a new passphrase.');
break;
case 'remove':
$this->slots->remove((int)$input->getArgument('slot'));
$output->writeln('Slot ' . (int)$input->getArgument('slot') . ' removed.');
break;
default:
(new Table($output))->setHeaders(['slot', 'label', 'set on'])
->setRows(array_map(fn ($s) => [$s['slot'], $s['label'], $s['created'] ? date('Y-m-d H:i', $s['created']) : ''], $this->keys->slots()))->render();
return 0;
}
} catch (\Throwable $e) {
$output->writeln('<error>' . $e->getMessage() . '</error>');
return 1;
}
$output->writeln('Now download and confirm the new recovery kit: occ backup:key:kit -o FILE && occ backup:key:confirm');
return 0;
}
private function passphrase(InputInterface $input, OutputInterface $output): string {
if ($file = $input->getOption('passphrase-file')) {
return rtrim((string)@file_get_contents($file), "\r\n");
}
$h = $this->getHelper('question');
$p = (string)$h->ask($input, $output, (new Question('Passphrase (min. 12 characters): '))->setHidden(true)->setHiddenFallback(false));
if ($p !== (string)$h->ask($input, $output, (new Question('Repeat: '))->setHidden(true)->setHiddenFallback(false))) {
throw new \InvalidArgumentException('The passphrases do not match');
}
return $p;
}
}
+1 -1
View File
@@ -15,7 +15,7 @@ use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface; use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class ListSnapshots extends Command { final class ListSnapshots extends Command {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
private SnapshotMapper $snapshots, private SnapshotMapper $snapshots,
+54
View File
@@ -0,0 +1,54 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\PruneService;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
final class Prune extends Command {
public function __construct(
private TargetService $targets,
private PruneService $prune,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:prune')
->setDescription('Apply the retention policy: remove old snapshots and the data only they used')
->addArgument('target', InputArgument::OPTIONAL, 'Location name or id (default: all that are not append-only)')
->addOption('dry-run', null, InputOption::VALUE_NONE, 'Only show what would be removed');
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$name = $input->getArgument('target');
$targets = $name === null ? array_filter($this->targets->list(), fn ($t) => !$t->getAppendOnly()) : [$this->targets->get($name)];
$dry = (bool)$input->getOption('dry-run');
$output->writeln('Policy: ' . json_encode($this->prune->policy()->toArray()) . ($dry ? ' (dry run)' : ''));
$rc = 0;
foreach ($targets as $t) {
try {
$r = $this->prune->apply($t, $dry);
} catch (\Throwable $e) {
$output->writeln('<error>' . $t->getName() . ': ' . $e->getMessage() . '</error>');
$rc = 1;
continue;
}
$p = $r['prune'];
$output->writeln(sprintf('<info>%s</info>: keep %d, %s %d snapshots; packs: %d, %s %d, repack %d, frees %.1f MiB',
$t->getName(), count($r['kept']), $dry ? 'would forget' : 'forgot', count($r['forgotten']), $p['packs'],
$dry ? 'would delete' : 'deleted', $p['deleted'], $p['repacked'], $p['freedBytes'] / 1048576));
}
return $rc;
}
}
+2 -2
View File
@@ -15,7 +15,7 @@ use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption; use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class Restore extends Command { final class Restore extends Command {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
private RestoreService $restore, private RestoreService $restore,
@@ -49,7 +49,7 @@ HELP);
try { try {
$target = $this->targets->get($input->getArgument('target')); $target = $this->targets->get($input->getArgument('target'));
$snapshot = $input->getArgument('snapshot'); $snapshot = $input->getArgument('snapshot');
$path = $input->getArgument('path'); $path = trim((string)$input->getArgument('path'), '/'); // CLI convenience; the service itself only accepts canonical paths
if ($dir = $input->getOption('to-directory')) { if ($dir = $input->getOption('to-directory')) {
$n = $this->restore->restoreToDirectory($target, $snapshot, $path, $dir); $n = $this->restore->restoreToDirectory($target, $snapshot, $path, $dir);
$output->writeln("Restored <info>$n</info> files to $dir."); $output->writeln("Restored <info>$n</info> files to $dir.");
+54
View File
@@ -0,0 +1,54 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\PruneService;
use OCA\NgBackup\Service\RetentionPolicy;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
final class Retention extends Command {
public function __construct(
private PruneService $prune,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:retention')
->setDescription('Show or set how many snapshots are kept')
->addOption('last', null, InputOption::VALUE_REQUIRED, 'Always keep the newest N')
->addOption('daily', null, InputOption::VALUE_REQUIRED, 'Keep one per day for the last N days with a backup')
->addOption('weekly', null, InputOption::VALUE_REQUIRED, 'Keep one per week for N weeks')
->addOption('monthly', null, InputOption::VALUE_REQUIRED, 'Keep one per month for N months')
->setHelp("Example: occ backup:retention --last=3 --daily=7 --weekly=4 --monthly=12");
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$p = $this->prune->policy()->toArray();
$changed = false;
foreach (['last', 'daily', 'weekly', 'monthly'] as $k) {
if ($input->getOption($k) !== null) {
$p[$k] = max(0, (int)$input->getOption($k));
$changed = true;
}
}
if ($changed) {
try {
$this->prune->setPolicy(RetentionPolicy::fromArray($p));
} catch (\InvalidArgumentException $e) {
$output->writeln('<error>' . $e->getMessage() . '</error>');
return 1;
}
}
$output->writeln(sprintf('Keep the last %d, plus one per day for %d days, per week for %d weeks, per month for %d months.', $p['last'], $p['daily'], $p['weekly'], $p['monthly']));
return 0;
}
}
+1 -1
View File
@@ -16,7 +16,7 @@ use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption; use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class RunBackup extends Command { final class RunBackup extends Command {
public function __construct( public function __construct(
private BackupService $backups, private BackupService $backups,
private TargetService $targets, private TargetService $targets,
+1 -1
View File
@@ -15,7 +15,7 @@ use Symfony\Component\Console\Helper\Table;
use Symfony\Component\Console\Input\InputInterface; use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class Status extends Command { final class Status extends Command {
public function __construct( public function __construct(
private KeyService $keys, private KeyService $keys,
private TargetService $targets, private TargetService $targets,
+1 -1
View File
@@ -15,7 +15,7 @@ use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption; use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class TargetAdd extends Command { final class TargetAdd extends Command {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
private KeyService $keys, private KeyService $keys,
+45
View File
@@ -0,0 +1,45 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
final class TargetAppendOnly extends Command {
public function __construct(
private TargetService $targets,
private \OCA\NgBackup\Service\AlertService $alerts,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:target:append-only')
->setDescription('Make a location append-only (NG Backup never deletes there) or normal again')
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
->addArgument('state', InputArgument::REQUIRED, 'on or off')
->setHelp("Append-only protects backups if this server is compromised: combine it with credentials\nwithout delete rights and a lifecycle rule or object lock on the location itself.");
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$state = strtolower((string)$input->getArgument('state'));
if (!in_array($state, ['on', 'off'], true)) {
$output->writeln('<error>state must be on or off</error>');
return 1;
}
$t = $this->targets->setAppendOnly($this->targets->get($input->getArgument('target')), $state === 'on');
if ($state === 'off') {
$this->alerts->securityEvent('append_only_off', ['target' => $t->getName()]);
}
$output->writeln($t->getName() . ' is ' . ($t->getAppendOnly() ? 'append-only' : 'normal') . '.');
return 0;
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Input\InputInterface; use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class TargetBackends extends Base { final class TargetBackends extends Base {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
) { ) {
+52
View File
@@ -0,0 +1,52 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
final class TargetLimit extends Command {
public function __construct(
private TargetService $targets,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:target:limit')
->setDescription('Set a maximum size for the backups stored on a location (e.g. 100G), or "none"')
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
->addArgument('size', InputArgument::OPTIONAL, 'Size like 500M, 20G, 2T, or "none"; omit to show the current limit')
->setHelp("A backup that would exceed the limit stops with an error and administrators are notified.\n"
. "Note: with versioning or Object Lock on the location, removed data keeps using space there\n"
. "until it expires; keep the limit below what you are willing to pay for.");
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$t = $this->targets->get($input->getArgument('target'));
$size = $input->getArgument('size');
if ($size !== null) {
if (strtolower($size) === 'none') {
$t = $this->targets->setMaxBytes($t, null);
} elseif (preg_match('/^(\d+(?:\.\d+)?)\s*([KMGT]?)i?B?$/i', $size, $m)) {
$factor = ['' => 1, 'K' => 1024, 'M' => 1024 ** 2, 'G' => 1024 ** 3, 'T' => 1024 ** 4][strtoupper($m[2])];
$t = $this->targets->setMaxBytes($t, (int)round((float)$m[1] * $factor));
} else {
$output->writeln('<error>Use a size like 500M, 20G or 2T, or "none".</error>');
return 1;
}
}
$used = $this->targets->repository($t)->storedBytes();
$output->writeln(sprintf('%s: %.2f GiB stored, limit %s', $t->getName(), $used / 1073741824,
$t->getMaxBytes() === null ? 'none' : sprintf('%.2f GiB', $t->getMaxBytes() / 1073741824)));
return 0;
}
}
+1 -1
View File
@@ -13,7 +13,7 @@ use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption; use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class TargetList extends Base { final class TargetList extends Base {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
) { ) {
+3 -1
View File
@@ -13,9 +13,10 @@ use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface; use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface; use Symfony\Component\Console\Output\OutputInterface;
class TargetRemove extends Command { final class TargetRemove extends Command {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
private \OCA\NgBackup\Service\AlertService $alerts,
) { ) {
parent::__construct(); parent::__construct();
} }
@@ -34,6 +35,7 @@ class TargetRemove extends Command {
return 1; return 1;
} }
$this->targets->remove($t); $this->targets->remove($t);
$this->alerts->securityEvent('target_removed', ['target' => $t->getName()]);
$output->writeln('Removed <info>' . $t->getName() . '</info>. The backups on the location itself were not touched; add it again to use them.'); $output->writeln('Removed <info>' . $t->getName() . '</info>. The backups on the location itself were not touched; add it again to use them.');
return 0; return 0;
} }
+57
View File
@@ -0,0 +1,57 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Db\AppConfigCatalogAnchor;
use OCA\NgBackup\Service\AlertService;
use OCA\NgBackup\Service\TargetService;
use OCP\IAppConfig;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Question\Question;
/**
* Accept the state a location shows now, after NG Backup refused it as rolled back. Only for
* legitimate cases (e.g. the location was restored from the provider's own snapshot), only from
* occ, audited and announced to all administrators.
*/
final class TargetTrustCurrent extends Command {
public function __construct(
private TargetService $targets,
private IAppConfig $appConfig,
private AlertService $alerts,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:target:trust-current')
->setDescription('Accept the current state of a location that was refused as rolled back')
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
->setHelp("NG Backup refuses a location whose backup catalog is older than what this server saw\n"
. "before, because an attacker could have restored an old state. If you restored the location\n"
. "yourself (e.g. from the provider's snapshot), accept it with this command.");
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$t = $this->targets->get($input->getArgument('target'));
$output->writeln('<comment>Only continue if you know why the location shows an older state.</comment>');
$answer = (string)$this->getHelper('question')->ask($input, $output, new Question('Type the location name to accept its current state: '));
if ($answer !== $t->getName()) {
$output->writeln('<error>Not confirmed.</error>');
return 1;
}
$this->appConfig->deleteKey('ng_backup', 'catalog_' . preg_replace('/[^a-f0-9]/', '', (string)$t->getRepositoryId()));
$repo = $this->targets->repository($t); // re-anchors on the newest valid catalog
$this->alerts->securityEvent('catalog_trusted', ['target' => $t->getName(), 'generation' => (string)$repo->catalogGeneration()]);
$output->writeln(sprintf('Accepted generation %d of %s (%d snapshots).', $repo->catalogGeneration(), $t->getName(), count($repo->listSnapshots())));
return 0;
}
}
+49
View File
@@ -0,0 +1,49 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\KeyService;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Helper\Table;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
final class Trash extends Command {
public function __construct(
private TargetService $targets,
private KeyService $keys,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:trash')
->setDescription('List removed snapshots that can still be restored (backup:trash:restore)')
->addArgument('target', InputArgument::OPTIONAL, 'Location name or id (default: all)');
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$name = $input->getArgument('target');
$delay = $this->keys->deleteDelayDays() * 86400;
$rows = [];
foreach ($name === null ? $this->targets->list() : [$this->targets->get($name)] as $t) {
foreach ($this->targets->repository($t)->trash() as $id => $info) {
$rows[] = [$t->getName(), $id, (string)($info['time'] ?? ''), $info['label'], date('Y-m-d H:i', $info['forgottenAt']), $info['by'],
date('Y-m-d H:i', $info['forgottenAt'] + $delay)];
}
}
if ($rows === []) {
$output->writeln('The trash is empty.');
return 0;
}
(new Table($output))->setHeaders(['location', 'snapshot', 'made', 'label', 'removed', 'by', 'deleted after'])->setRows($rows)->render();
return 0;
}
}
+42
View File
@@ -0,0 +1,42 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\PruneService;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
final class TrashRestore extends Command {
public function __construct(
private TargetService $targets,
private PruneService $prune,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:trash:restore')
->setDescription('Bring a removed snapshot back from the trash')
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:trash)');
}
protected function execute(InputInterface $input, OutputInterface $output): int {
try {
$this->prune->untrash($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'));
} catch (\Throwable $e) {
$output->writeln('<error>' . $e->getMessage() . '</error>');
return 1;
}
$output->writeln('Snapshot restored from the trash.');
return 0;
}
}
+119 -13
View File
@@ -19,6 +19,8 @@ use OCA\NgBackup\Service\RestoreService;
use OCA\NgBackup\Service\TargetService; use OCA\NgBackup\Service\TargetService;
use OCP\AppFramework\Controller; use OCP\AppFramework\Controller;
use OCP\AppFramework\Http; use OCP\AppFramework\Http;
use OCP\AppFramework\Http\Attribute\NoCSRFRequired;
use OCP\AppFramework\Http\Attribute\PasswordConfirmationRequired;
use OCP\AppFramework\Http\DataDownloadResponse; use OCP\AppFramework\Http\DataDownloadResponse;
use OCP\AppFramework\Http\JSONResponse; use OCP\AppFramework\Http\JSONResponse;
use OCP\BackgroundJob\IJobList; use OCP\BackgroundJob\IJobList;
@@ -30,7 +32,7 @@ use OCP\IUserSession;
* JSON API for the admin settings page. Every method requires an admin (the AppFramework * JSON API for the admin settings page. Every method requires an admin (the AppFramework
* default for controllers without NoAdminRequired) and a valid CSRF token. * default for controllers without NoAdminRequired) and a valid CSRF token.
*/ */
class AdminApiController extends Controller { final class AdminApiController extends Controller {
public function __construct( public function __construct(
IRequest $request, IRequest $request,
private KeyService $keys, private KeyService $keys,
@@ -42,6 +44,9 @@ class AdminApiController extends Controller {
private IAppConfig $appConfig, private IAppConfig $appConfig,
private IJobList $jobs, private IJobList $jobs,
private IUserSession $session, private IUserSession $session,
private \OCA\NgBackup\Service\AlertService $alerts,
private \OCA\NgBackup\Service\KeySlotService $slots,
private \OCA\NgBackup\Service\PruneService $prune,
) { ) {
parent::__construct(Application::APP_ID, $request); parent::__construct(Application::APP_ID, $request);
} }
@@ -50,12 +55,15 @@ class AdminApiController extends Controller {
$targets = []; $targets = [];
foreach ($this->targets->list() as $t) { foreach ($this->targets->list() as $t) {
$targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(), $targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt()]; 'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt(), 'appendOnly' => (bool)$t->getAppendOnly(),
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t)];
} }
$runs = array_map(fn (Run $r) => $this->runJson($r), $this->runs->findRecent(15)); $runs = array_map(fn (Run $r) => $this->runJson($r), $this->runs->findRecent(15));
return new JSONResponse([ return new JSONResponse([
'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(), 'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(),
'confirmation' => $this->keys->recoveryKitConfirmation()], 'confirmation' => $this->keys->recoveryKitConfirmation(), 'everConfirmed' => $this->keys->isInitialized() && $this->keys->everConfirmed(),
'slots' => $this->keys->isInitialized() ? $this->keys->slots() : [], 'maxSlots' => \OCA\NgBackup\Service\KeyService::MAX_SLOTS,
'deleteDelayDays' => $this->keys->isInitialized() ? $this->keys->deleteDelayDays() : \OCA\NgBackup\Service\KeyService::DEFAULT_DELETE_DELAY],
'statement' => KeyConfirm::STATEMENT, 'statement' => KeyConfirm::STATEMENT,
'confirmationPhrase' => KeyConfirm::CONFIRMATION, 'confirmationPhrase' => KeyConfirm::CONFIRMATION,
'targets' => $targets, 'targets' => $targets,
@@ -64,47 +72,126 @@ class AdminApiController extends Controller {
]); ]);
} }
public function initKey(string $passphrase): JSONResponse { #[PasswordConfirmationRequired]
public function initKey(string $passphrase, int $deleteDelayDays = \OCA\NgBackup\Service\KeyService::DEFAULT_DELETE_DELAY, string $label = 'Slot 1'): JSONResponse {
try { try {
$this->keys->initialize($passphrase); $this->keys->initialize($passphrase, $deleteDelayDays, $label);
} catch (\Throwable $e) { } catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST); return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
} }
return new JSONResponse(['fingerprint' => $this->keys->fingerprint()]); return new JSONResponse(['fingerprint' => $this->keys->fingerprint()]);
} }
/**
* Plain download link (no CSRF token on a navigation), therefore guarded by a recent password
* confirmation instead, logged to the audit log and announced to all administrators. The kit
* only holds the passphrase-wrapped key: useless without the passphrase.
*/
#[NoCSRFRequired]
#[PasswordConfirmationRequired]
public function downloadKit(): DataDownloadResponse|JSONResponse { public function downloadKit(): DataDownloadResponse|JSONResponse {
if (!$this->keys->isInitialized()) { if (!$this->keys->isInitialized()) {
return new JSONResponse(['error' => 'No backup key yet'], Http::STATUS_BAD_REQUEST); return new JSONResponse(['error' => 'No backup key yet'], Http::STATUS_BAD_REQUEST);
} }
$this->alerts->securityEvent('kit_downloaded');
$json = json_encode($this->keys->recoveryKit(), JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR); $json = json_encode($this->keys->recoveryKit(), JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
return new DataDownloadResponse($json, 'ng_backup-recovery-kit-' . $this->keys->fingerprint() . '.json', 'application/json'); return new DataDownloadResponse($json, 'ng_backup-recovery-kit-' . $this->keys->fingerprint() . '.json', 'application/json');
} }
/** The admin must tick the statement and send the exact confirmation phrase. */ /** The admin must tick the statement, send the exact phrase and the code from the kit file. */
public function confirmKit(bool $accepted, string $phrase): JSONResponse { #[PasswordConfirmationRequired]
public function confirmKit(bool $accepted, string $phrase, string $code = ''): JSONResponse {
if (!$accepted || $phrase !== KeyConfirm::CONFIRMATION) { if (!$accepted || $phrase !== KeyConfirm::CONFIRMATION) {
return new JSONResponse(['error' => 'Not confirmed'], Http::STATUS_BAD_REQUEST); return new JSONResponse(['error' => 'Not confirmed'], Http::STATUS_BAD_REQUEST);
} }
if (!$this->keys->checkKitCode($code)) {
return new JSONResponse(['error' => 'The confirmation code does not match the current recovery kit'], Http::STATUS_BAD_REQUEST);
}
$this->keys->confirmRecoveryKit($this->session->getUser()?->getUID() ?? 'unknown'); $this->keys->confirmRecoveryKit($this->session->getUser()?->getUID() ?? 'unknown');
return new JSONResponse(['confirmation' => $this->keys->recoveryKitConfirmation()]); return new JSONResponse(['confirmation' => $this->keys->recoveryKitConfirmation()]);
} }
#[PasswordConfirmationRequired]
public function addSlot(string $passphrase, string $label): JSONResponse {
try {
return new JSONResponse(['slot' => $this->slots->add($passphrase, $label)]);
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
}
#[PasswordConfirmationRequired]
public function replaceSlot(int $slot, string $passphrase, ?string $label = null): JSONResponse {
try {
$this->slots->replace($slot, $passphrase, $label);
return new JSONResponse([]);
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
}
#[PasswordConfirmationRequired]
public function removeSlot(int $slot): JSONResponse {
try {
$this->slots->remove($slot);
return new JSONResponse([]);
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
}
public function trash(int $targetId): JSONResponse {
try {
$delay = $this->keys->deleteDelayDays() * 86400;
$out = [];
foreach ($this->targets->repository($this->targets->get((string)$targetId))->trash() as $id => $info) {
$out[] = ['id' => $id, 'label' => $info['label'], 'time' => $info['time'], 'removedAt' => $info['forgottenAt'],
'by' => $info['by'], 'deleteAfter' => $info['forgottenAt'] + $delay];
}
return new JSONResponse($out);
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
}
#[PasswordConfirmationRequired]
public function untrash(int $targetId, string $snapshotId): JSONResponse {
try {
$this->prune->untrash($this->targets->get((string)$targetId), $snapshotId);
return new JSONResponse([]);
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
}
public function backends(): JSONResponse { public function backends(): JSONResponse {
return new JSONResponse($this->targets->availableBackends()); return new JSONResponse($this->targets->availableBackends());
} }
public function addTarget(string $name, string $backend, string $auth, array $options = [], string $path = 'ng_backup'): JSONResponse { public function addTarget(string $name, string $backend, string $auth, array $options = [], string $path = 'ng_backup', ?float $maxGb = null): JSONResponse {
try { try {
$r = $this->targets->add($name, $backend, $auth, $options, $path); $r = $this->targets->add($name, $backend, $auth, $options, $path);
if ($maxGb !== null && $maxGb > 0) {
$this->targets->setMaxBytes($r['target'], (int)round($maxGb * 1073741824));
}
} catch (\Throwable $e) { } catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST); return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
} }
return new JSONResponse(['id' => $r['target']->getId(), 'created' => $r['created']]); return new JSONResponse(['id' => $r['target']->getId(), 'created' => $r['created']]);
} }
/** Size limit in GiB; null or 0 = unlimited. */
#[PasswordConfirmationRequired]
public function setTargetLimit(int $id, ?float $maxGb = null): JSONResponse {
$t = $this->targets->setMaxBytes($this->targets->get((string)$id), ($maxGb !== null && $maxGb > 0) ? (int)round($maxGb * 1073741824) : null);
return new JSONResponse(['maxBytes' => $t->getMaxBytes()]);
}
#[PasswordConfirmationRequired]
public function removeTarget(int $id): JSONResponse { public function removeTarget(int $id): JSONResponse {
$this->targets->remove($this->targets->get((string)$id)); $target = $this->targets->get((string)$id);
$this->targets->remove($target);
$this->alerts->securityEvent('target_removed', ['target' => $target->getName()]);
return new JSONResponse([]); return new JSONResponse([]);
} }
@@ -152,13 +239,19 @@ class AdminApiController extends Controller {
} }
/** Queue a restore of user files; it runs as a background job and shows up in the runs. */ /** Queue a restore of user files; it runs as a background job and shows up in the runs. */
#[PasswordConfirmationRequired]
public function startRestore(int $targetId, string $snapshotId, string $path, string $mode = RestoreService::MODE_NEW_FOLDER): JSONResponse { public function startRestore(int $targetId, string $snapshotId, string $path, string $mode = RestoreService::MODE_NEW_FOLDER): JSONResponse {
if (!in_array($mode, [RestoreService::MODE_NEW_FOLDER, RestoreService::MODE_MERGE, RestoreService::MODE_REPLACE], true) if (!in_array($mode, [RestoreService::MODE_NEW_FOLDER, RestoreService::MODE_MERGE, RestoreService::MODE_REPLACE], true)
|| !preg_match('#^data/[^/]+/files(/.*)?$#', $path)) { || !\OCA\NgBackup\Repository\Repository::isSafePath($path) || !preg_match('#^data/[^/]+/files(/.*)?$#', $path)) {
return new JSONResponse(['error' => 'Invalid path or mode'], Http::STATUS_BAD_REQUEST); return new JSONResponse(['error' => 'Invalid path or mode'], Http::STATUS_BAD_REQUEST);
} }
try {
$target = $this->targets->get((string)$targetId);
} catch (\Throwable) {
return new JSONResponse(['error' => 'No such location'], Http::STATUS_NOT_FOUND);
}
$run = new Run(); $run = new Run();
$run->setTargetId($targetId); $run->setTargetId($target->getId());
$run->setKind('restore'); $run->setKind('restore');
$run->setStatus(Run::RUNNING); $run->setStatus(Run::RUNNING);
$run->setPhase('queued'); $run->setPhase('queued');
@@ -167,14 +260,27 @@ class AdminApiController extends Controller {
$run->setSnapshotId($snapshotId); $run->setSnapshotId($snapshotId);
$run->setStartedAt(time()); $run->setStartedAt(time());
$run->setUpdatedAt(time()); $run->setUpdatedAt(time());
$run = $this->runs->insert($run); try {
$run = $this->prune->underStartLock($target, fn () => $this->runs->insert($run));
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_CONFLICT);
}
$this->jobs->add(RestoreJob::class, ['run' => $run->getId()]); $this->jobs->add(RestoreJob::class, ['run' => $run->getId()]);
return new JSONResponse($this->runJson($run)); return new JSONResponse($this->runJson($run));
} }
private function runJson(Run $r): array { private function runJson(Run $r): array {
$snapshot = null;
if ($r->getSnapshotId() !== null) {
try {
$s = $this->snapshots->findOne($r->getTargetId(), $r->getSnapshotId());
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel()];
} catch (\Throwable) {
// Snapshot already pruned or trashed: the run keeps its id, just not the date.
}
}
return ['id' => $r->getId(), 'targetId' => $r->getTargetId(), 'kind' => $r->getKind(), 'status' => $r->getStatus(), return ['id' => $r->getId(), 'targetId' => $r->getTargetId(), 'kind' => $r->getKind(), 'status' => $r->getStatus(),
'phase' => $r->getPhase(), 'snapshotId' => $r->getSnapshotId(), 'startedAt' => $r->getStartedAt(), 'phase' => $r->getPhase(), 'snapshotId' => $r->getSnapshotId(), 'snapshot' => $snapshot, 'startedAt' => $r->getStartedAt(),
'updatedAt' => $r->getUpdatedAt(), 'finishedAt' => $r->getFinishedAt(), 'error' => $r->getError(), 'updatedAt' => $r->getUpdatedAt(), 'finishedAt' => $r->getFinishedAt(), 'error' => $r->getError(),
'stats' => json_decode($r->getStats() ?? 'null', true)]; 'stats' => json_decode($r->getStats() ?? 'null', true)];
} }
+1 -1
View File
@@ -7,5 +7,5 @@ declare(strict_types=1);
namespace OCA\NgBackup\Crypto; namespace OCA\NgBackup\Crypto;
class CryptoException extends \RuntimeException { final class CryptoException extends \RuntimeException {
} }
+17
View File
@@ -76,6 +76,23 @@ final class KeyRing {
]; ];
} }
/**
* Open a key field that is either one wrapped key or {"slots": [{slot, label, key}, ...]}:
* every slot holds the same master key under its own passphrase.
*/
public static function unwrapAny(array $field, #[\SensitiveParameter] string $passphrase): self {
if (!isset($field['slots'])) {
return self::unwrap($field, $passphrase);
}
foreach ($field['slots'] as $slot) {
try {
return self::unwrap($slot['key'], $passphrase);
} catch (CryptoException) {
}
}
throw new CryptoException('Wrong passphrase or damaged key');
}
/** @param array{v:int, salt:string, ops:int, mem:int, nonce:string, key:string} $wrapped */ /** @param array{v:int, salt:string, ops:int, mem:int, nonce:string, key:string} $wrapped */
public static function unwrap(array $wrapped, #[\SensitiveParameter] string $passphrase): self { public static function unwrap(array $wrapped, #[\SensitiveParameter] string $passphrase): self {
$b64 = static fn (string $s): string => sodium_base642bin($s, SODIUM_BASE64_VARIANT_ORIGINAL); $b64 = static fn (string $s): string => sodium_base642bin($s, SODIUM_BASE64_VARIANT_ORIGINAL);
+34
View File
@@ -0,0 +1,34 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Db;
use OCA\NgBackup\AppInfo\Application;
use OCA\NgBackup\Repository\ICatalogAnchor;
use OCP\IAppConfig;
/** Keeps the newest catalog generation seen per repository in the app config (outside the location). */
final class AppConfigCatalogAnchor implements ICatalogAnchor {
public function __construct(
private IAppConfig $appConfig,
) {
}
public function get(string $repositoryId): ?array {
$raw = $this->appConfig->getValueString(Application::APP_ID, $this->key($repositoryId), '', true);
$data = $raw === '' ? null : json_decode($raw, true);
return is_array($data) && isset($data['gen'], $data['hash']) ? ['gen' => (int)$data['gen'], 'hash' => (string)$data['hash']] : null;
}
public function set(string $repositoryId, int $gen, string $hash): void {
$this->appConfig->setValueString(Application::APP_ID, $this->key($repositoryId), json_encode(['gen' => $gen, 'hash' => $hash]), true);
}
private function key(string $repositoryId): string {
return 'catalog_' . preg_replace('/[^a-f0-9]/', '', $repositoryId);
}
}
+121 -45
View File
@@ -31,6 +31,8 @@ use OCA\NgBackup\Repository\Repository;
*/ */
final class DbDumper { final class DbDumper {
private const BATCH = 2000; private const BATCH = 2000;
/** Rows per INSERT on restore; keeps the number of bound parameters well below database limits. */
private const INSERT_ROWS = 100;
/** Tables without a primary key are read in one query; refuse that above this size. */ /** Tables without a primary key are read in one query; refuse that above this size. */
private const MAX_ROWS_WITHOUT_PK = 50000; private const MAX_ROWS_WITHOUT_PK = 50000;
@@ -41,39 +43,87 @@ final class DbDumper {
} }
/** /**
* @return array{tables: array<string, array{blobs: list<string>, rows: int, bytes: int, sha256: string}>, provider: string} * Dump everything in one go (one transaction, fully consistent).
*
* @return array{tables: array<string, array{blobs: list<string>, rows: int, bytes: int, sha256: string}>, sequences: array, provider: string, consistent: bool}
*/ */
public function dump(Repository $repo, array &$stats): array { public function dump(Repository $repo, array &$stats): array {
$sm = $this->db->createSchemaManager(); $state = $this->startDump();
$tables = array_values(array_filter($sm->listTableNames(), fn (string $t) => str_starts_with($t, $this->prefix))); while (!$state['done']) {
sort($tables); $state = $this->dumpStep($repo, $state, PHP_FLOAT_MAX, $stats);
}
return $this->manifest($state);
}
/** State for a resumable dump (see dumpStep()). */
public function startDump(): array {
$tables = array_values(array_filter($this->db->createSchemaManager()->listTableNames(), fn (string $t) => str_starts_with($t, $this->prefix)));
sort($tables);
return ['tables' => $tables, 'i' => 0, 'after' => null, 'segments' => [], 'rows' => 0, 'bytes' => 0, 'hash' => null,
'result' => [], 'sequences' => [], 'steps' => 0, 'done' => false];
}
/**
* Dump tables until $deadline, inside one REPEATABLE READ transaction for this step. A table
* that does not fit is resumed in the next step at the last primary key written. If the whole
* dump fits in one step it is fully consistent; otherwise each step is consistent on its own
* ('consistent' => false in the manifest).
*/
public function dumpStep(Repository $repo, array $state, float $deadline, array &$stats): array {
$state['steps']++;
$sm = $this->db->createSchemaManager();
$this->db->setTransactionIsolation(TransactionIsolationLevel::REPEATABLE_READ); $this->db->setTransactionIsolation(TransactionIsolationLevel::REPEATABLE_READ);
$this->db->beginTransaction(); $this->db->beginTransaction();
try { try {
if ($this->db->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\PostgreSQLPlatform) { if ($this->db->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\PostgreSQLPlatform) {
$this->db->executeStatement('SET TRANSACTION READ ONLY'); $this->db->executeStatement('SET TRANSACTION READ ONLY');
} }
$result = []; while ($state['i'] < count($state['tables'])) {
foreach ($tables as $table) { $table = $state['tables'][$state['i']];
$schema = $sm->introspectTable($table);
$writer = $repo->blobWriter($stats); $writer = $repo->blobWriter($stats);
$hash = hash_init('sha256'); $hash = $state['hash'] !== null ? unserialize(base64_decode($state['hash']), ['allowed_classes' => [\HashContext::class]]) : hash_init('sha256');
$rows = 0; if ($state['after'] === null && $state['segments'] === []) {
foreach ($this->tableLines($sm->introspectTable($table)) as $kind => $line) { $writer->write($this->header($schema) . "\n");
}
$complete = true;
foreach ($this->rowLines($schema, $state['after']) as [$key, $line]) {
$writer->write($line . "\n"); $writer->write($line . "\n");
if ($kind === 'row') { hash_update($hash, $line . "\n");
hash_update($hash, $line . "\n"); $state['rows']++;
$rows++; $state['after'] = $key;
if (microtime(true) >= $deadline) {
$complete = false;
break;
} }
} }
$result[$table] = ['blobs' => $writer->finish(), 'rows' => $rows, 'bytes' => $writer->bytes(), 'sha256' => hash_final($hash)]; $state['segments'][] = $writer->finish();
$state['bytes'] += $writer->bytes();
if (!$complete) {
$state['hash'] = base64_encode(serialize($hash));
break;
}
$state['result'][$table] = ['blobs' => array_merge(...$state['segments']), 'rows' => $state['rows'],
'bytes' => $state['bytes'], 'sha256' => hash_final($hash)];
$state = array_merge($state, ['i' => $state['i'] + 1, 'after' => null, 'segments' => [], 'rows' => 0, 'bytes' => 0, 'hash' => null]);
if (microtime(true) >= $deadline && $state['i'] < count($state['tables'])) {
break;
}
}
if ($state['i'] >= count($state['tables'])) {
$state['sequences'] = $this->sequenceStates($state['tables']);
$state['done'] = true;
} }
$sequences = $this->sequenceStates($tables);
} finally { } finally {
$this->db->rollBack(); $this->db->rollBack();
} }
$repo->flushPacks(); $repo->flushPacks();
return ['tables' => $result, 'sequences' => $sequences, 'provider' => get_class($this->db->getDatabasePlatform())]; return $state;
}
public function manifest(array $state): array {
return ['tables' => $state['result'], 'sequences' => $state['sequences'],
'provider' => get_class($this->db->getDatabasePlatform()), 'consistent' => $state['steps'] === 1];
} }
/** /**
@@ -82,16 +132,23 @@ final class DbDumper {
* *
* @param array{tables: array<string, array{blobs: list<string>, rows: int}>} $manifest * @param array{tables: array<string, array{blobs: list<string>, rows: int}>} $manifest
*/ */
public function restore(Repository $repo, array $manifest): int { public function restore(Repository $repo, array $manifest, ?array &$report = null): int {
$sm = $this->db->createSchemaManager(); $sm = $this->db->createSchemaManager();
$existing = array_flip($sm->listTableNames()); $existing = array_flip($sm->listTableNames());
$total = 0; $total = 0;
$report = ['skipped' => [], 'restored' => []];
$this->db->beginTransaction(); $this->db->beginTransaction();
try { try {
foreach ($manifest['tables'] as $table => $info) { foreach ($manifest['tables'] as $table => $info) {
if (!isset($existing[$table])) { if (!isset($existing[$table])) {
throw new \RuntimeException("Table $table from the backup does not exist here (different app versions?)"); throw new \RuntimeException("Table $table from the backup does not exist here (different app versions?)");
} }
// Unchanged since the backup: leave it alone (much faster than delete + insert).
$live = $this->checksum($table);
if ($live['rows'] === $info['rows'] && hash_equals($info['sha256'], $live['sha256'])) {
$report['skipped'][] = $table;
continue;
}
$this->db->executeStatement('DELETE FROM ' . $this->db->quoteIdentifier($table)); $this->db->executeStatement('DELETE FROM ' . $this->db->quoteIdentifier($table));
$columns = null; $columns = null;
$binary = []; $binary = [];
@@ -112,17 +169,18 @@ final class DbDumper {
if (is_array($value) && isset($value['$b'])) { if (is_array($value) && isset($value['$b'])) {
$value = base64_decode($value['$b'], true); // text that was not valid UTF-8 $value = base64_decode($value['$b'], true); // text that was not valid UTF-8
} }
$row[$col] = ($value !== null && isset($binary[$col])) ? base64_decode($value, true) : $value; $row[] = ($value !== null && isset($binary[$col])) ? base64_decode($value, true) : $value;
} }
$batch[] = $row; $batch[] = $row;
if (count($batch) >= 500) { if (count($batch) >= self::INSERT_ROWS) {
$total += $this->insertBatch($table, $batch, $binary); $total += $this->insertBatch($table, $columns, $batch, $binary);
$batch = []; $batch = [];
} }
} }
if ($batch !== []) { if ($batch !== []) {
$total += $this->insertBatch($table, $batch, $binary); $total += $this->insertBatch($table, $columns, $batch, $binary);
} }
$report['restored'][] = $table;
} }
$this->restoreSequences($manifest['sequences'] ?? []); $this->restoreSequences($manifest['sequences'] ?? []);
$this->db->commit(); $this->db->commit();
@@ -135,20 +193,22 @@ final class DbDumper {
/** Same checksum as dump(): sha256 over the row lines in primary-key order. */ /** Same checksum as dump(): sha256 over the row lines in primary-key order. */
public function checksum(string $table): array { public function checksum(string $table): array {
$sm = $this->db->createSchemaManager();
$hash = hash_init('sha256'); $hash = hash_init('sha256');
$rows = 0; $rows = 0;
foreach ($this->tableLines($sm->introspectTable($table)) as $kind => $line) { foreach ($this->rowLines($this->db->createSchemaManager()->introspectTable($table)) as [, $line]) {
if ($kind === 'row') { hash_update($hash, $line . "\n");
hash_update($hash, $line . "\n"); $rows++;
$rows++;
}
} }
return ['rows' => $rows, 'sha256' => hash_final($hash)]; return ['rows' => $rows, 'sha256' => hash_final($hash)];
} }
/** @return \Generator<string, string> 'header' => line, then 'row' => line ... */ private function header(\Doctrine\DBAL\Schema\Table $table): string {
private function tableLines(\Doctrine\DBAL\Schema\Table $table): \Generator { [$columns, $binary, $pk] = $this->describe($table);
return json_encode(['table' => $table->getName(), 'columns' => $columns, 'binary' => $binary, 'pk' => $pk], JSON_THROW_ON_ERROR);
}
/** @return array{0: list<string>, 1: list<string>, 2: list<string>} columns, binary columns, primary key */
private function describe(\Doctrine\DBAL\Schema\Table $table): array {
$columns = array_values(array_map(fn ($c) => $c->getName(), $table->getColumns())); $columns = array_values(array_map(fn ($c) => $c->getName(), $table->getColumns()));
$binary = []; $binary = [];
foreach ($table->getColumns() as $c) { foreach ($table->getColumns() as $c) {
@@ -156,25 +216,34 @@ final class DbDumper {
$binary[] = $c->getName(); $binary[] = $c->getName();
} }
} }
$pk = $table->getPrimaryKey()?->getColumns() ?? []; return [$columns, $binary, $table->getPrimaryKey()?->getColumns() ?? []];
yield 'header' => json_encode(['table' => $table->getName(), 'columns' => array_values($columns), 'binary' => $binary, 'pk' => $pk], JSON_THROW_ON_ERROR); }
/**
* Rows in primary-key order, after the key $after (resume), as [key, JSON line].
*
* @return \Generator<array{0: list<mixed>|null, 1: string}>
*/
private function rowLines(\Doctrine\DBAL\Schema\Table $table, ?array $after = null): \Generator {
[$columns, $binary, $pk] = $this->describe($table);
$binaryFlip = array_flip($binary); $binaryFlip = array_flip($binary);
$q = fn (string $c) => $this->db->quoteIdentifier($c); $q = fn (string $c) => $this->db->quoteIdentifier($c);
$select = 'SELECT ' . implode(', ', array_map($q, $columns)) . ' FROM ' . $q($table->getName()); $select = 'SELECT ' . implode(', ', array_map($q, $columns)) . ' FROM ' . $q($table->getName());
if ($pk === []) { if ($pk === []) {
// Nextcloud requires primary keys; small odd tables are read in one query. // Nextcloud requires primary keys; small odd tables are read in one query (not resumable).
$count = (int)$this->db->fetchOne('SELECT COUNT(*) FROM ' . $q($table->getName())); $count = (int)$this->db->fetchOne('SELECT COUNT(*) FROM ' . $q($table->getName()));
if ($count > self::MAX_ROWS_WITHOUT_PK) { if ($count > self::MAX_ROWS_WITHOUT_PK) {
throw new \RuntimeException("Table {$table->getName()} has no primary key and $count rows; cannot dump it with bounded memory"); throw new \RuntimeException("Table {$table->getName()} has no primary key and $count rows; cannot dump it with bounded memory");
} }
$batches = [$this->db->fetchAllNumeric($select)]; $batches = [$this->db->fetchAllNumeric($select)];
$pkIdx = [];
} else { } else {
$batches = $this->keysetBatches($select, $pk, $columns); $batches = $this->keysetBatches($select, $pk, $columns, $after);
$pkIdx = array_map(fn ($c) => array_search($c, $columns, true), $pk);
} }
foreach ($batches as $rows) { foreach ($batches as $rows) {
foreach ($rows as $row) { foreach ($rows as $row) {
$key = $pkIdx === [] ? null : array_map(fn ($i) => $row[$i], $pkIdx);
foreach ($columns as $i => $col) { foreach ($columns as $i => $col) {
if (is_resource($row[$i])) { if (is_resource($row[$i])) {
$row[$i] = stream_get_contents($row[$i]); $row[$i] = stream_get_contents($row[$i]);
@@ -189,23 +258,23 @@ final class DbDumper {
$row[$i] = (string)(is_bool($row[$i]) ? (int)$row[$i] : $row[$i]); $row[$i] = (string)(is_bool($row[$i]) ? (int)$row[$i] : $row[$i]);
} }
} }
yield 'row' => json_encode($row, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_THROW_ON_ERROR); yield [$key, json_encode($row, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_THROW_ON_ERROR)];
} }
} }
} }
/** @return \Generator<list<list<mixed>>> */ /** @return \Generator<list<list<mixed>>> */
private function keysetBatches(string $select, array $pk, array $columns): \Generator { private function keysetBatches(string $select, array $pk, array $columns, ?array $after): \Generator {
$q = fn (string $c) => $this->db->quoteIdentifier($c); $q = fn (string $c) => $this->db->quoteIdentifier($c);
$order = ' ORDER BY ' . implode(', ', array_map($q, $pk)); $order = ' ORDER BY ' . implode(', ', array_map($q, $pk));
$pkIdx = array_map(fn ($c) => array_search($c, $columns, true), $pk); $cols = array_map(fn ($c) => $q($c), $pk);
$last = null; $last = $after;
while (true) { while (true) {
$sql = $select; $sql = $select;
$params = []; $params = [];
if ($last !== null) { if ($last !== null) {
// Row-value comparison (a, b) > (?, ?), supported by PostgreSQL, MySQL/MariaDB and SQLite. // Row-value comparison (a, b) > (?, ?), supported by PostgreSQL, MySQL/MariaDB and SQLite.
$sql .= ' WHERE (' . implode(', ', array_map($q, $pk)) . ') > (' . implode(', ', array_fill(0, count($pk), '?')) . ')'; $sql .= ' WHERE (' . implode(', ', $cols) . ') > (' . implode(', ', array_fill(0, count($pk), '?')) . ')';
$params = $last; $params = $last;
} }
$rows = $this->db->fetchAllNumeric($sql . $order . ' LIMIT ' . self::BATCH, $params); $rows = $this->db->fetchAllNumeric($sql . $order . ' LIMIT ' . self::BATCH, $params);
@@ -213,22 +282,29 @@ final class DbDumper {
return; return;
} }
yield $rows; yield $rows;
$end = end($rows);
$last = array_map(fn ($i) => $end[$i], $pkIdx);
if (count($rows) < self::BATCH) { if (count($rows) < self::BATCH) {
return; return;
} }
$end = end($rows);
$last = array_map(fn ($c) => $end[array_search($c, $columns, true)], $pk);
} }
} }
private function insertBatch(string $table, array $rows, array $binary): int { /** One INSERT with several rows (portable multi-row VALUES). */
private function insertBatch(string $table, array $columns, array $rows, array $binary): int {
$q = fn (string $c) => $this->db->quoteIdentifier($c);
$placeholders = '(' . implode(', ', array_fill(0, count($columns), '?')) . ')';
$sql = 'INSERT INTO ' . $q($table) . ' (' . implode(', ', array_map($q, $columns)) . ') VALUES '
. implode(', ', array_fill(0, count($rows), $placeholders));
$params = [];
$types = [];
foreach ($rows as $row) { foreach ($rows as $row) {
$types = []; foreach ($columns as $i => $col) {
foreach ($row as $col => $_) { $params[] = $row[$i];
$types[$col] = isset($binary[$col]) ? \Doctrine\DBAL\ParameterType::LARGE_OBJECT : \Doctrine\DBAL\ParameterType::STRING; $types[] = isset($binary[$col]) ? \Doctrine\DBAL\ParameterType::LARGE_OBJECT : ($row[$i] === null ? \Doctrine\DBAL\ParameterType::NULL : \Doctrine\DBAL\ParameterType::STRING);
} }
$this->db->insert($this->db->quoteIdentifier($table), array_combine(array_map(fn ($c) => $this->db->quoteIdentifier($c), array_keys($row)), $row), array_values($types));
} }
$this->db->executeStatement($sql, $params, $types);
return count($rows); return count($rows);
} }
+1 -1
View File
@@ -11,7 +11,7 @@ use OCA\NgBackup\Repository\IIndexCache;
use OCP\DB\Exception; use OCP\DB\Exception;
use OCP\IDBConnection; use OCP\IDBConnection;
class DbIndexCache implements IIndexCache { final class DbIndexCache implements IIndexCache {
public function __construct( public function __construct(
private IDBConnection $db, private IDBConnection $db,
) { ) {
+1 -1
View File
@@ -33,7 +33,7 @@ use OCP\AppFramework\Db\Entity;
* @method int|null getFinishedAt() * @method int|null getFinishedAt()
* @method void setFinishedAt(?int $v) * @method void setFinishedAt(?int $v)
*/ */
class Run extends Entity { final class Run extends Entity {
public const RUNNING = 'running'; public const RUNNING = 'running';
public const DONE = 'done'; public const DONE = 'done';
public const FAILED = 'failed'; public const FAILED = 'failed';
+1 -1
View File
@@ -12,7 +12,7 @@ use OCP\DB\QueryBuilder\IQueryBuilder;
use OCP\IDBConnection; use OCP\IDBConnection;
/** @template-extends QBMapper<Run> */ /** @template-extends QBMapper<Run> */
class RunMapper extends QBMapper { final class RunMapper extends QBMapper {
public function __construct(IDBConnection $db) { public function __construct(IDBConnection $db) {
parent::__construct($db, 'ngb_runs', Run::class); parent::__construct($db, 'ngb_runs', Run::class);
} }
+1 -1
View File
@@ -25,7 +25,7 @@ use OCP\AppFramework\Db\Entity;
* @method int|null getBytes() * @method int|null getBytes()
* @method void setBytes(?int $v) * @method void setBytes(?int $v)
*/ */
class Snapshot extends Entity { final class Snapshot extends Entity {
protected int $targetId = 0; protected int $targetId = 0;
protected string $snapshotId = ''; protected string $snapshotId = '';
protected string $kind = ''; protected string $kind = '';
+1 -1
View File
@@ -12,7 +12,7 @@ use OCP\DB\QueryBuilder\IQueryBuilder;
use OCP\IDBConnection; use OCP\IDBConnection;
/** @template-extends QBMapper<Snapshot> */ /** @template-extends QBMapper<Snapshot> */
class SnapshotMapper extends QBMapper { final class SnapshotMapper extends QBMapper {
public function __construct(IDBConnection $db) { public function __construct(IDBConnection $db) {
parent::__construct($db, 'ngb_snapshots', Snapshot::class); parent::__construct($db, 'ngb_snapshots', Snapshot::class);
} }
+9 -1
View File
@@ -22,10 +22,14 @@ use OCP\AppFramework\Db\Entity;
* @method void setBasePath(string $v) * @method void setBasePath(string $v)
* @method string|null getRepositoryId() * @method string|null getRepositoryId()
* @method void setRepositoryId(?string $v) * @method void setRepositoryId(?string $v)
* @method bool|null getAppendOnly()
* @method void setAppendOnly(bool $v)
* @method int|null getMaxBytes()
* @method void setMaxBytes(?int $v)
* @method int getCreatedAt() * @method int getCreatedAt()
* @method void setCreatedAt(int $v) * @method void setCreatedAt(int $v)
*/ */
class Target extends Entity { final class Target extends Entity {
protected string $name = ''; protected string $name = '';
protected string $backend = ''; protected string $backend = '';
protected string $auth = ''; protected string $auth = '';
@@ -33,8 +37,12 @@ class Target extends Entity {
protected string $basePath = ''; protected string $basePath = '';
protected ?string $repositoryId = null; protected ?string $repositoryId = null;
protected int $createdAt = 0; protected int $createdAt = 0;
protected ?bool $appendOnly = null;
protected ?int $maxBytes = null;
public function __construct() { public function __construct() {
$this->addType('createdAt', 'integer'); $this->addType('createdAt', 'integer');
$this->addType('appendOnly', 'boolean');
$this->addType('maxBytes', 'integer');
} }
} }
+1 -1
View File
@@ -11,7 +11,7 @@ use OCP\AppFramework\Db\QBMapper;
use OCP\IDBConnection; use OCP\IDBConnection;
/** @template-extends QBMapper<Target> */ /** @template-extends QBMapper<Target> */
class TargetMapper extends QBMapper { final class TargetMapper extends QBMapper {
public function __construct(IDBConnection $db) { public function __construct(IDBConnection $db) {
parent::__construct($db, 'ngb_targets', Target::class); parent::__construct($db, 'ngb_targets', Target::class);
} }
@@ -13,7 +13,7 @@ use OCP\DB\Types;
use OCP\Migration\IOutput; use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep; use OCP\Migration\SimpleMigrationStep;
class Version000001Date20261002000000 extends SimpleMigrationStep { final class Version000001Date20261002000000 extends SimpleMigrationStep {
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper { public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */ /** @var ISchemaWrapper $schema */
$schema = $schemaClosure(); $schema = $schemaClosure();
@@ -14,7 +14,7 @@ use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep; use OCP\Migration\SimpleMigrationStep;
/** Local cache of the per-pack index files, so opening a repository does not download them all. */ /** Local cache of the per-pack index files, so opening a repository does not download them all. */
class Version000002Date20261002010000 extends SimpleMigrationStep { final class Version000002Date20261002010000 extends SimpleMigrationStep {
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper { public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */ /** @var ISchemaWrapper $schema */
$schema = $schemaClosure(); $schema = $schemaClosure();
@@ -0,0 +1,28 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Migration;
use Closure;
use OCP\DB\ISchemaWrapper;
use OCP\DB\Types;
use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep;
/** Append-only locations: NG Backup never deletes anything there (cleanup by the location's own lifecycle rules). */
final class Version000003Date20261002020000 extends SimpleMigrationStep {
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */
$schema = $schemaClosure();
$t = $schema->getTable('ngb_targets');
if (!$t->hasColumn('append_only')) {
$t->addColumn('append_only', Types::BOOLEAN, ['notnull' => false, 'default' => false]);
return $schema;
}
return null;
}
}
@@ -0,0 +1,28 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Migration;
use Closure;
use OCP\DB\ISchemaWrapper;
use OCP\DB\Types;
use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep;
/** Optional size limit per location (cost control; S3 has no bucket quota). */
final class Version000004Date20261002030000 extends SimpleMigrationStep {
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */
$schema = $schemaClosure();
$t = $schema->getTable('ngb_targets');
if (!$t->hasColumn('max_bytes')) {
$t->addColumn('max_bytes', Types::BIGINT, ['notnull' => false, 'unsigned' => true]);
return $schema;
}
return null;
}
}
@@ -0,0 +1,33 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Migration;
use Closure;
use OCP\DB\ISchemaWrapper;
use OCP\DB\Types;
use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep;
/** Leases with an expiry: a killed process cannot block a location for longer than its lease. */
final class Version000005Date20261002040000 extends SimpleMigrationStep {
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */
$schema = $schemaClosure();
if ($schema->hasTable('ngb_leases')) {
return null;
}
$t = $schema->createTable('ngb_leases');
$t->addColumn('id', Types::STRING, ['notnull' => true, 'length' => 32]);
$t->addColumn('name', Types::STRING, ['notnull' => true, 'length' => 128]);
$t->addColumn('mode', Types::STRING, ['notnull' => true, 'length' => 1]);
$t->addColumn('expires_at', Types::BIGINT, ['notnull' => true, 'unsigned' => true]);
$t->setPrimaryKey(['id']);
$t->addIndex(['name'], 'ngb_leases_name');
return $schema;
}
}
+77
View File
@@ -0,0 +1,77 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Notification;
use OCA\NgBackup\AppInfo\Application;
use OCP\IURLGenerator;
use OCP\L10N\IFactory;
use OCP\Notification\INotification;
use OCP\Notification\INotifier;
use OCP\Notification\UnknownNotificationException;
final class Notifier implements INotifier {
public function __construct(
private IFactory $l10n,
private IURLGenerator $url,
) {
}
public function getID(): string {
return Application::APP_ID;
}
public function getName(): string {
return $this->l10n->get(Application::APP_ID)->t('NG Backup');
}
public function prepare(INotification $notification, string $languageCode): INotification {
if ($notification->getApp() !== Application::APP_ID) {
throw new UnknownNotificationException();
}
$l = $this->l10n->get(Application::APP_ID, $languageCode);
$p = $notification->getSubjectParameters();
switch ($notification->getSubject()) {
case 'run_failed':
$notification->setParsedSubject($p['kind'] === 'restore'
? $l->t('Restore from %s failed', [$p['target']])
: $l->t('Backup to %s failed', [$p['target']]));
$notification->setParsedMessage((string)($p['error'] ?? ''));
break;
case 'quota_warning':
$notification->setParsedSubject($l->t('Backup location %1$s is at %2$s%% of its limit (%3$s of %4$s)', [$p['target'], $p['percent'], $p['used'], $p['limit']]));
$notification->setParsedMessage($l->t('Backups stop when the limit is reached. Raise the limit, or let the retention policy remove older backups.'));
break;
case 'backup_stale':
$notification->setParsedSubject($l->t('No successful backup to %1$s since %2$s', [$p['target'], $p['since']]));
$notification->setParsedMessage($l->t('Check the backup settings and the server log.'));
break;
case 'security_event':
$who = (string)($p['actor'] ?? '');
$notification->setParsedSubject(match ($p['event'] ?? '') {
'kit_downloaded' => $l->t('%s downloaded the backup recovery kit', [$who]),
'target_removed' => $l->t('%1$s removed backup location %2$s', [$who, $p['target'] ?? '']),
'append_only_off' => $l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
'retention_changed' => $l->t('%1$s changed the backup retention to %2$s', [$who, $p['policy'] ?? '']),
'snapshots_forgotten' => $l->t('%1$s moved %2$s snapshots of %3$s to the trash; they are deleted permanently after %4$s unless restored', [$who, $p['count'] ?? '', $p['target'] ?? '', $p['until'] ?? '']),
'snapshot_untrashed' => $l->t('%1$s restored snapshot %2$s of %3$s from the trash', [$who, $p['snapshot'] ?? '', $p['target'] ?? '']),
'catalog_trusted' => $l->t('%1$s accepted an older state of backup location %2$s (catalog generation %3$s)', [$who, $p['target'] ?? '', $p['generation'] ?? '']),
'slot_added' => $l->t('%1$s added backup passphrase slot %2$s (%3$s); download and confirm the new recovery kit', [$who, $p['slot'] ?? '', $p['label'] ?? '']),
'slot_replaced' => $l->t('%1$s set a new passphrase for backup key slot %2$s (%3$s); download and confirm the new recovery kit', [$who, $p['slot'] ?? '', $p['label'] ?? '']),
'slot_removed' => $l->t('%1$s removed backup passphrase slot %2$s; download and confirm the new recovery kit', [$who, $p['slot'] ?? '']),
default => $l->t('NG Backup security event'),
});
$notification->setParsedMessage($l->t('If this was not you or another administrator, check who has admin access.'));
break;
default:
throw new UnknownNotificationException();
}
$notification->setLink($this->url->linkToRouteAbsolute('settings.AdminSettings.index', ['section' => 'ng_backup']));
$notification->setIcon($this->url->getAbsoluteURL($this->url->imagePath(Application::APP_ID, 'app-dark.svg')));
return $notification;
}
}
+46 -2
View File
@@ -45,8 +45,9 @@ final class BlobIndex {
* and only the index files that are not cached yet are downloaded. Cached packs that no * and only the index files that are not cached yet are downloaded. Cached packs that no
* longer exist on the backend are dropped from the cache. * longer exist on the backend are dropped from the cache.
*/ */
public function load(): void { public function load(bool $verify = false): void {
$cached = $this->cache?->all($this->repositoryId) ?? []; $this->blobs = [];
$cached = $verify ? [] : ($this->cache?->all($this->repositoryId) ?? []);
$present = []; $present = [];
$this->downloaded = 0; $this->downloaded = 0;
foreach ($this->backend->list('index') as $path) { foreach ($this->backend->list('index') as $path) {
@@ -65,6 +66,20 @@ final class BlobIndex {
$this->blobs[$e['id']] = [$packId, $e['offset'], $e['length'], $e['raw'], $e['flags']]; $this->blobs[$e['id']] = [$packId, $e['offset'], $e['length'], $e['raw'], $e['flags']];
} }
} }
if ($verify && $this->cache !== null) {
// Refresh the cache from what was just verified.
$this->cache->remove($this->repositoryId, array_keys($this->cache->all($this->repositoryId)));
foreach ($this->packs() as $packId => $_) {
$entries = [];
foreach ($this->blobs as $id => [$pack, $offset, $length, $raw, $flags]) {
if ($pack === $packId) {
$entries[] = ['id' => $id, 'offset' => $offset, 'length' => $length, 'raw' => $raw, 'flags' => $flags];
}
}
$this->cache->put($this->repositoryId, $packId, $entries);
}
return;
}
$stale = array_keys(array_diff_key($cached, $present)); $stale = array_keys(array_diff_key($cached, $present));
if ($stale !== []) { if ($stale !== []) {
$this->cache?->remove($this->repositoryId, $stale); $this->cache?->remove($this->repositoryId, $stale);
@@ -104,6 +119,35 @@ final class BlobIndex {
} }
} }
/** @return array<string, list<array{id:string, offset:int, length:int}>> packId => blobs currently located in that pack */
public function packs(): array {
$packs = [];
foreach ($this->blobs as $id => [$pack, $offset, $length]) {
$packs[$pack][] = ['id' => $id, 'offset' => $offset, 'length' => $length];
}
return $packs;
}
/** Forget a pack: delete its index file and cache row. Blobs that now live elsewhere are kept. */
public function removePack(string $packId): void {
$this->backend->delete('index/' . $packId);
$this->cache?->remove($this->repositoryId, [$packId]);
foreach ($this->blobs as $id => $loc) {
if ($loc[0] === $packId) {
unset($this->blobs[$id]);
}
}
}
/** Bytes of all blobs in the index (encrypted size, i.e. what is stored in packs). */
public function storedBytes(): int {
$total = 0;
foreach ($this->blobs as [, , $length]) {
$total += $length;
}
return $total;
}
public function count(): int { public function count(): int {
return count($this->blobs); return count($this->blobs);
} }
+129
View File
@@ -0,0 +1,129 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Repository;
use OCA\NgBackup\Backend\IBackend;
use OCA\NgBackup\Crypto\StreamCipher;
/**
* Chained catalog of a repository's state, against rollback and hidden snapshots.
*
* Every change to the set of snapshots or trash entries writes catalog/<generation>: the
* encrypted list of snapshot and trash ids plus the hash of the previous generation. The server
* keeps the newest generation it has seen (ICatalogAnchor). On open, the newest catalog on the
* location must be at least the anchored generation and lead back to it through the chain;
* snapshots listed in the catalog must exist. Otherwise the location was rolled back or
* tampered with, and the repository is refused.
*/
final class Catalog {
private const KEEP = 200; // generations kept on the location (enough to verify long gaps)
public int $gen = 0;
/** @var array<string, true> */
public array $snapshots = [];
/** @var array<string, true> */
public array $trash = [];
public string $hash = '';
public function __construct(
private IBackend $backend,
private StreamCipher $cipher,
private string $repositoryId,
private ?ICatalogAnchor $anchor,
) {
}
/** Load and verify the newest catalog; create the first one for repositories that have none. */
public function load(callable $listSnapshots, callable $listTrash): void {
$gens = $this->generations();
if ($gens === []) {
// First use (new or pre-catalog repository): adopt the current state as generation 1.
$anchored = $this->anchor?->get($this->repositoryId);
if ($anchored !== null) {
throw new RepositoryException('The catalog of this repository has disappeared from the location (possible tampering)');
}
$this->snapshots = array_fill_keys($listSnapshots(), true);
$this->trash = array_fill_keys($listTrash(), true);
$this->write();
return;
}
$head = max($gens);
[$data, $hash] = $this->read($head);
$anchored = $this->anchor?->get($this->repositoryId);
if ($anchored !== null) {
if ($head < $anchored['gen']) {
throw new RepositoryException(sprintf('The location shows catalog generation %d, but this server has seen %d: the backups were rolled back', $head, $anchored['gen']));
}
// Walk back to the anchored generation, checking every link.
[$cur, $curHash] = [$data, $hash];
for ($g = $head; $g > $anchored['gen']; $g--) {
[$prev, $prevHash] = $this->read($g - 1);
if (!hash_equals($prevHash, (string)$cur['prev'])) {
throw new RepositoryException("Catalog chain broken at generation $g (possible tampering)");
}
[$cur, $curHash] = [$prev, $prevHash];
}
if (!hash_equals($anchored['hash'], $curHash)) {
throw new RepositoryException('The catalog does not match what this server saw before (possible tampering)');
}
}
$this->gen = $head;
$this->hash = $hash;
$this->snapshots = array_fill_keys($data['snapshots'], true);
$this->trash = array_fill_keys($data['trash'], true);
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
}
/** Write the next generation after the in-memory state was changed. */
public function write(): void {
$next = $this->gen + 1;
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash)], JSON_THROW_ON_ERROR);
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
$stream = fopen('php://memory', 'w+b');
fwrite($stream, $encrypted);
rewind($stream);
$this->backend->put($this->path($next), $stream);
$this->gen = $next;
$this->hash = hash('sha256', $encrypted);
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
$old = $next - self::KEEP;
if ($old > 0) {
$this->backend->delete($this->path($old));
}
}
/** @return array{0: array, 1: string} decoded catalog and the hash of its encrypted bytes */
private function read(int $gen): array {
try {
$fh = $this->backend->get($this->path($gen));
} catch (\Throwable) {
throw new RepositoryException("Catalog generation $gen is missing (possible tampering)");
}
$encrypted = (string)stream_get_contents($fh);
fclose($fh);
$data = json_decode($this->cipher->decryptString($encrypted, $this->ad($gen)), true, 512, JSON_THROW_ON_ERROR);
if (($data['repository'] ?? '') !== $this->repositoryId || ($data['gen'] ?? 0) !== $gen) {
throw new RepositoryException("Catalog generation $gen belongs to another repository or position");
}
return [$data, hash('sha256', $encrypted)];
}
/** @return list<int> */
private function generations(): array {
return array_map(fn ($p) => (int)basename($p), $this->backend->list('catalog'));
}
private function path(int $gen): string {
return sprintf('catalog/%010d', $gen);
}
private function ad(int $gen): string {
return 'catalog:' . $this->repositoryId . ':' . $gen;
}
}
+20
View File
@@ -0,0 +1,20 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Repository;
/**
* Where the server remembers the newest catalog generation it has seen for a repository, outside
* the (untrusted) location. A location that presents an older generation, or a chain that does
* not lead back to this anchor, is rolled back or tampered with.
*/
interface ICatalogAnchor {
/** @return array{gen:int, hash:string}|null */
public function get(string $repositoryId): ?array;
public function set(string $repositoryId, int $gen, string $hash): void;
}
+17
View File
@@ -32,6 +32,7 @@ final class PackWriter {
private StreamCipher $cipher, private StreamCipher $cipher,
private BlobIndex $index, private BlobIndex $index,
private int $targetSize = 32 * 1048576, private int $targetSize = 32 * 1048576,
private ?int $maxBytes = null,
) { ) {
$this->reset(); $this->reset();
} }
@@ -49,11 +50,27 @@ final class PackWriter {
return strlen($encrypted); return strlen($encrypted);
} }
/** Add an already encrypted blob as-is (repacking: no decryption needed, the blob is bound to its id, not to a pack). */
public function addRaw(string $id, string $encrypted, int $rawLength, int $flags): void {
fwrite($this->buffer, $encrypted);
$this->entries[] = ['id' => $id, 'offset' => $this->size, 'length' => strlen($encrypted), 'raw' => $rawLength, 'flags' => $flags];
$this->size += strlen($encrypted);
if ($this->size >= $this->targetSize) {
$this->flush();
}
}
/** Upload the current pack (if any) and record it in the index. */ /** Upload the current pack (if any) and record it in the index. */
public function flush(): void { public function flush(): void {
if ($this->entries === []) { if ($this->entries === []) {
return; return;
} }
if ($this->maxBytes !== null && $this->index->storedBytes() + $this->size > $this->maxBytes) {
$this->index->dropPending(array_column($this->entries, 'id'));
$this->reset();
throw new QuotaExceededException(sprintf('Storage limit of this location reached (%.1f GiB): backup stopped',
$this->maxBytes / 1073741824));
}
$packId = bin2hex(random_bytes(16)); $packId = bin2hex(random_bytes(16));
$header = $this->cipher->encryptString(json_encode($this->entries, JSON_THROW_ON_ERROR), 'pack-header:' . $packId); $header = $this->cipher->encryptString(json_encode($this->entries, JSON_THROW_ON_ERROR), 'pack-header:' . $packId);
fwrite($this->buffer, $header . pack('N', strlen($header))); fwrite($this->buffer, $header . pack('N', strlen($header)));
+11
View File
@@ -0,0 +1,11 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Repository;
final class QuotaExceededException extends RepositoryException {
}
+273 -14
View File
@@ -32,6 +32,8 @@ final class Repository {
private StreamCipher $cipher; private StreamCipher $cipher;
private BlobIndex $index; private BlobIndex $index;
private ?PackWriter $runPacks = null; private ?PackWriter $runPacks = null;
private ?int $maxBytes = null;
private Catalog $catalog;
private function __construct( private function __construct(
private IBackend $backend, private IBackend $backend,
@@ -52,7 +54,7 @@ final class Repository {
* copy (KeyRing::wrap) stored in the repository config, so the repository can be opened * copy (KeyRing::wrap) stored in the repository config, so the repository can be opened
* with the passphrase from the recovery kit even when the server is gone. * with the passphrase from the recovery kit even when the server is gone.
*/ */
public static function initWithKey(IBackend $backend, KeyRing $keys, array $wrappedKey): self { public static function initWithKey(IBackend $backend, KeyRing $keys, array $wrappedKey, ?ICatalogAnchor $anchor = null): self {
if ($backend->exists('config')) { if ($backend->exists('config')) {
throw new RepositoryException('A repository already exists at this location'); throw new RepositoryException('A repository already exists at this location');
} }
@@ -60,11 +62,13 @@ final class Repository {
$config = ['format' => self::FORMAT, 'id' => $id, 'created' => gmdate('c'), 'key' => $wrappedKey, $config = ['format' => self::FORMAT, 'id' => $id, 'created' => gmdate('c'), 'key' => $wrappedKey,
'check' => base64_encode((new StreamCipher($keys))->encryptString($id, 'config-check'))]; 'check' => base64_encode((new StreamCipher($keys))->encryptString($id, 'config-check'))];
self::putString($backend, 'config', json_encode($config, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR)); self::putString($backend, 'config', json_encode($config, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR));
return new self($backend, $keys, $id); $repo = new self($backend, $keys, $id);
$repo->loadCatalog($anchor);
return $repo;
} }
/** Open with the master key the server holds (scheduled backups, no passphrase needed). */ /** Open with the master key the server holds (scheduled backups, no passphrase needed). */
public static function openWithKey(IBackend $backend, KeyRing $keys, ?IIndexCache $cache = null): self { public static function openWithKey(IBackend $backend, KeyRing $keys, ?IIndexCache $cache = null, ?ICatalogAnchor $anchor = null): self {
$config = self::readConfig($backend); $config = self::readConfig($backend);
try { try {
$ok = isset($config['check']) && (new StreamCipher($keys))->decryptString((string)base64_decode($config['check'], true), 'config-check') === $config['id']; $ok = isset($config['check']) && (new StreamCipher($keys))->decryptString((string)base64_decode($config['check'], true), 'config-check') === $config['id'];
@@ -79,9 +83,31 @@ final class Repository {
$repo->index->setCache($cache, $config['id']); $repo->index->setCache($cache, $config['id']);
} }
$repo->index->load(); $repo->index->load();
$repo->loadCatalog($anchor);
return $repo; return $repo;
} }
private function loadCatalog(?ICatalogAnchor $anchor): void {
$this->catalog = new Catalog($this->backend, $this->cipher, $this->repositoryId, $anchor);
$this->catalog->load(
fn () => array_map('basename', $this->backend->list('snapshots')),
fn () => array_map('basename', $this->backend->list('trash/info')));
}
/** Refuse uploads that would make the stored data larger than this (null = no limit). */
public function setMaxBytes(?int $maxBytes): void {
$this->maxBytes = $maxBytes;
}
public function storedBytes(): int {
return $this->index->storedBytes();
}
/** Current catalog generation (for status and tests). */
public function catalogGeneration(): int {
return $this->catalog->gen;
}
/** Index files downloaded when this repository was opened (0 when the local cache was complete). */ /** Index files downloaded when this repository was opened (0 when the local cache was complete). */
public function indexDownloads(): int { public function indexDownloads(): int {
return $this->index->downloaded; return $this->index->downloaded;
@@ -101,14 +127,22 @@ final class Repository {
return $config; return $config;
} }
/** Replace the passphrase-wrapped key slots in this repository's config (after a slot change). */
public function updateWrappedKey(array $wrappedKey): void {
$config = self::readConfig($this->backend);
$config['key'] = $wrappedKey;
self::putString($this->backend, 'config', json_encode($config, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR));
}
public function id(): string { public function id(): string {
return $this->repositoryId; return $this->repositoryId;
} }
public static function open(IBackend $backend, #[\SensitiveParameter] string $passphrase): self { public static function open(IBackend $backend, #[\SensitiveParameter] string $passphrase): self {
$config = self::readConfig($backend); $config = self::readConfig($backend);
$repo = new self($backend, KeyRing::unwrap($config['key'], $passphrase), $config['id']); $repo = new self($backend, KeyRing::unwrapAny($config['key'], $passphrase), $config['id']);
$repo->index->load(); $repo->index->load();
$repo->loadCatalog(null); // disaster recovery on a new server: no anchor yet, trust the newest valid chain
return $repo; return $repo;
} }
@@ -121,7 +155,7 @@ final class Repository {
public function backupDirectory(string $source, ?string $parent = null, string $label = ''): array { public function backupDirectory(string $source, ?string $parent = null, string $label = ''): array {
$source = rtrim($source, '/'); $source = rtrim($source, '/');
$previous = $parent !== null ? $this->loadEntries($parent) : []; $previous = $parent !== null ? $this->loadEntries($parent) : [];
$packs = new PackWriter($this->backend, $this->cipher, $this->index); $packs = new PackWriter($this->backend, $this->cipher, $this->index, 32 * 1048576, $this->maxBytes);
$stats = ['files' => 0, 'reused' => 0, 'read' => 0, 'newBlobs' => 0, 'dupBlobs' => 0, 'uploaded' => 0]; $stats = ['files' => 0, 'reused' => 0, 'read' => 0, 'newBlobs' => 0, 'dupBlobs' => 0, 'uploaded' => 0];
$tree = fopen('php://temp/maxmemory:' . (4 * 1048576), 'w+b'); $tree = fopen('php://temp/maxmemory:' . (4 * 1048576), 'w+b');
@@ -149,12 +183,17 @@ final class Repository {
rewind($encrypted); rewind($encrypted);
$this->backend->put('snapshots/' . $snapshotId, $encrypted); $this->backend->put('snapshots/' . $snapshotId, $encrypted);
fclose($tree); fclose($tree);
$this->catalog->snapshots[$snapshotId] = true;
$this->catalog->write();
return ['snapshot' => $snapshotId] + $stats; return ['snapshot' => $snapshotId] + $stats;
} }
/** Restore a snapshot (or only paths starting with $prefix) into a local directory. */ /** Restore a snapshot (or only paths starting with $prefix) into a local directory. */
public function restore(string $snapshotId, string $target, string $prefix = ''): int { public function restore(string $snapshotId, string $target, string $prefix = ''): int {
if ($prefix !== '' && !self::isSafePath(rtrim($prefix, '/'))) {
throw new RepositoryException('Invalid path prefix');
}
$count = 0; $count = 0;
foreach ($this->streamEntries($snapshotId) as $entry) { foreach ($this->streamEntries($snapshotId) as $entry) {
if ($prefix !== '' && !str_starts_with($entry['p'], $prefix)) { if ($prefix !== '' && !str_starts_with($entry['p'], $prefix)) {
@@ -164,10 +203,19 @@ final class Repository {
if (!is_dir(dirname($dest))) { if (!is_dir(dirname($dest))) {
mkdir(dirname($dest), 0700, true); mkdir(dirname($dest), 0700, true);
} }
// Defence in depth: the resolved directory must stay below the target (no symlink escapes).
$base = realpath($target);
$dir = realpath(dirname($dest));
if ($base === false || $dir === false || ($dir !== $base && !str_starts_with($dir, $base . '/')) || is_link($dest)) {
throw new RepositoryException('Refusing to write outside the restore directory: ' . $entry['p']);
}
// Write next to the destination and rename after all blobs verified: at most one file // Write next to the destination and rename after all blobs verified: at most one file
// is ever present twice. // is ever present twice.
$tmp = $dest . '.ngb-restore'; $tmp = $dest . '.ngb-restore-' . bin2hex(random_bytes(6));
$out = fopen($tmp, 'wb'); $out = @fopen($tmp, 'xb'); // fails if anything (including a symlink) already exists there
if ($out === false) {
throw new RepositoryException('Cannot create a temporary file for ' . $entry['p']);
}
foreach ($entry['b'] as $id) { foreach ($entry['b'] as $id) {
fwrite($out, $this->loadBlob($id)); fwrite($out, $this->loadBlob($id));
} }
@@ -188,7 +236,7 @@ final class Repository {
* run is done, so the last pack is uploaded before anything references it. * run is done, so the last pack is uploaded before anything references it.
*/ */
public function blobWriter(array &$stats): BlobWriter { public function blobWriter(array &$stats): BlobWriter {
$this->runPacks ??= new PackWriter($this->backend, $this->cipher, $this->index); $this->runPacks ??= new PackWriter($this->backend, $this->cipher, $this->index, 32 * 1048576, $this->maxBytes);
$packs = $this->runPacks; $packs = $this->runPacks;
return new BlobWriter(function (string $data) use ($packs, &$stats): string { return new BlobWriter(function (string $data) use ($packs, &$stats): string {
return $this->storeBlob($data, $packs, $stats); return $this->storeBlob($data, $packs, $stats);
@@ -220,7 +268,7 @@ final class Repository {
/** Store one blob of file data in the current run (deduplicated); returns its id. */ /** Store one blob of file data in the current run (deduplicated); returns its id. */
public function storeData(string $data, array &$stats): string { public function storeData(string $data, array &$stats): string {
$this->runPacks ??= new PackWriter($this->backend, $this->cipher, $this->index); $this->runPacks ??= new PackWriter($this->backend, $this->cipher, $this->index, 32 * 1048576, $this->maxBytes);
return $this->storeBlob($data, $this->runPacks, $stats); return $this->storeBlob($data, $this->runPacks, $stats);
} }
@@ -244,6 +292,8 @@ final class Repository {
fclose($tree); fclose($tree);
rewind($encrypted); rewind($encrypted);
$this->backend->put('snapshots/' . $snapshotId, $encrypted); $this->backend->put('snapshots/' . $snapshotId, $encrypted);
$this->catalog->snapshots[$snapshotId] = true;
$this->catalog->write();
} }
/** @return array<string, array{s:int, m:int, b:list<string>}> */ /** @return array<string, array{s:int, m:int, b:list<string>}> */
@@ -261,6 +311,19 @@ final class Repository {
} }
} }
/** Relative path of non-empty segments, without ".", "..", NUL, backslash or control characters. */
public static function isSafePath(string $path): bool {
if ($path === '' || str_starts_with($path, '/') || preg_match('/[\x00-\x1f\\\\]/', $path)) {
return false;
}
foreach (explode('/', $path) as $segment) {
if ($segment === '' || $segment === '.' || $segment === '..') {
return false;
}
}
return true;
}
/** Small encrypted objects (manifests). */ /** Small encrypted objects (manifests). */
public function putObject(string $path, string $data): void { public function putObject(string $path, string $data): void {
self::putString($this->backend, $path, $this->cipher->encryptString($data, 'object:' . $path)); self::putString($this->backend, $path, $this->cipher->encryptString($data, 'object:' . $path));
@@ -273,9 +336,200 @@ final class Repository {
return $data; return $data;
} }
/** @return list<string> */ /**
* Move a snapshot (and its database dump) to the trash on the location. Its data stays in
* use until purgeTrash() removes it after the deletion delay, so a malicious or mistaken
* cleanup can be undone with untrash() within that time.
*/
public function forget(string $snapshotId, string $by = ''): void {
$meta = $this->snapshotMeta($snapshotId);
$dbInTrash = null;
if (isset($meta['db']) && is_string($meta['db']) && $this->backend->exists($meta['db'])) {
// Its encryption is bound to its path (object:db/...), which changes on a move, so it
// must be decrypted and re-encrypted under the new path, not moved as raw bytes.
$dbInTrash = 'trash/' . $meta['db'];
$this->putObject($dbInTrash, $this->getObject($meta['db']));
$this->backend->delete($meta['db']);
}
// The snapshot object is bound to its id (not its path), so it can be moved as raw bytes.
$this->backend->move('snapshots/' . $snapshotId, 'trash/snapshots/' . $snapshotId);
$this->putObject('trash/info/' . $snapshotId, json_encode(['forgottenAt' => time(), 'by' => $by, 'db' => $meta['db'] ?? null,
'dbInTrash' => $dbInTrash, 'time' => $meta['time'] ?? null, 'label' => $meta['label'] ?? ''], JSON_THROW_ON_ERROR));
unset($this->catalog->snapshots[$snapshotId]);
$this->catalog->trash[$snapshotId] = true;
$this->catalog->write();
}
/** Undo forget(): move a snapshot back from the trash. */
public function untrash(string $snapshotId): void {
$info = $this->trashInfo($snapshotId);
if ($info['dbInTrash'] !== null && $this->backend->exists($info['dbInTrash'])) {
$this->putObject($info['db'], $this->getObject($info['dbInTrash']));
$this->backend->delete($info['dbInTrash']);
}
$this->backend->move('trash/snapshots/' . $snapshotId, 'snapshots/' . $snapshotId);
$this->catalog->snapshots[$snapshotId] = true;
unset($this->catalog->trash[$snapshotId]);
$this->catalog->write();
$this->backend->delete('trash/info/' . $snapshotId);
}
/** @return array<string, array{forgottenAt:int, by:string, db:?string, dbInTrash:?string, time:?string, label:string}> */
public function trash(): array {
$out = [];
foreach (array_keys($this->catalog->trash) as $id) {
try {
$out[$id] = $this->trashInfo($id);
} catch (\Throwable) {
throw new RepositoryException("Trash entry $id listed in the catalog is missing on the location (possible tampering)");
}
}
return $out;
}
/** Permanently delete trash entries older than $delaySeconds. @return list<string> purged snapshot ids */
public function purgeTrash(int $delaySeconds): array {
$purged = [];
foreach ($this->trash() as $id => $info) {
if (time() - $info['forgottenAt'] < $delaySeconds) {
continue;
}
if ($info['dbInTrash'] !== null) {
$this->backend->delete($info['dbInTrash']);
}
$this->backend->delete('trash/snapshots/' . $id);
$this->backend->delete('trash/info/' . $id);
unset($this->catalog->trash[$id]);
$purged[] = $id;
}
if ($purged !== []) {
$this->catalog->write();
}
return $purged;
}
private function trashInfo(string $snapshotId): array {
return json_decode($this->getObject('trash/info/' . $snapshotId), true, 512, JSON_THROW_ON_ERROR);
}
/**
* Delete packs no snapshot or user export uses any more, and repack packs where more than
* $repackThreshold of the bytes are unused. Must not run while a backup to this repository
* is in progress (its blobs are not referenced by a snapshot yet); callers hold the lock.
*
* @return array{packs:int, deleted:int, repacked:int, freedBytes:int, keptBlobs:int, unusedBlobs:int}
*/
public function prune(bool $dryRun = false, float $repackThreshold = 0.5, array $ignoreSnapshots = []): array {
// Deleting is decided on the authenticated index files on the location, never on the local cache.
$this->index->load(true);
$used = [];
$ignore = array_flip($ignoreSnapshots); // e.g. snapshots a dry run would forget first
foreach ($this->listSnapshots() as $sid) {
if (isset($ignore[$sid])) {
continue;
}
foreach ($this->streamEntries($sid) as $e) {
foreach ($e['b'] as $b) {
$used[$b] = true;
}
}
$meta = $this->snapshotMeta($sid);
if (isset($meta['db']) && $this->backend->exists($meta['db'])) {
foreach (json_decode($this->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR)['tables'] as $t) {
foreach ($t['blobs'] as $b) {
$used[$b] = true;
}
}
}
}
// Snapshots in the trash keep their data until purgeTrash() removed them.
foreach ($this->trash() as $sid => $info) {
foreach ($this->streamEntries($sid, 'trash/snapshots/') as $e) {
foreach ($e['b'] as $b) {
$used[$b] = true;
}
}
if ($info['dbInTrash'] !== null && $this->backend->exists($info['dbInTrash'])) {
foreach (json_decode($this->getObject($info['dbInTrash']), true, 512, JSON_THROW_ON_ERROR)['tables'] as $t) {
foreach ($t['blobs'] as $b) {
$used[$b] = true;
}
}
}
}
foreach ($this->backend->list('users') as $path) {
foreach (json_decode($this->getObject($path), true, 512, JSON_THROW_ON_ERROR)['entries'] as $e) {
foreach ($e['b'] ?? [] as $b) {
$used[$b] = true;
}
}
}
$stats = ['packs' => 0, 'deleted' => 0, 'repacked' => 0, 'freedBytes' => 0, 'keptBlobs' => count($used), 'unusedBlobs' => 0];
$packs = $this->index->packs();
$stats['packs'] = count($packs);
$writer = null;
$toRemove = [];
foreach ($packs as $packId => $blobs) {
$total = 0;
$unused = 0;
foreach ($blobs as $b) {
$total += $b['length'];
if (!isset($used[$b['id']])) {
$unused += $b['length'];
$stats['unusedBlobs']++;
}
}
if ($unused === 0 || ($unused < $total && $unused / $total <= $repackThreshold)) {
continue;
}
$stats['freedBytes'] += $unused;
if ($unused === $total) {
$stats['deleted']++;
} else {
$stats['repacked']++;
if (!$dryRun) {
// No size limit here: repacking is how space is freed.
$writer ??= new PackWriter($this->backend, $this->cipher, $this->index);
$path = 'packs/' . substr($packId, 0, 2) . '/' . $packId;
foreach ($blobs as $b) {
if (isset($used[$b['id']])) {
[, , , $raw, $flags] = $this->index->get($b['id']);
$writer->addRaw($b['id'], $this->backend->getRange($path, $b['offset'], $b['length']), $raw, $flags);
}
}
}
}
$toRemove[] = $packId;
}
if ($dryRun) {
return $stats;
}
// New packs and their index entries are durable before any old pack is removed.
$writer?->flush();
foreach ($toRemove as $packId) {
$this->index->removePack($packId);
$this->backend->delete('packs/' . substr($packId, 0, 2) . '/' . $packId);
}
return $stats;
}
/**
* Snapshots according to the verified catalog. A snapshot the catalog lists but the location
* no longer has means it was deleted or hidden behind NG Backup's back.
*
* @return list<string>
*/
public function listSnapshots(): array { public function listSnapshots(): array {
return array_map('basename', $this->backend->list('snapshots')); $present = array_flip(array_map('basename', $this->backend->list('snapshots')));
$listed = array_keys($this->catalog->snapshots);
foreach ($listed as $id) {
if (!isset($present[$id])) {
throw new RepositoryException("Snapshot $id is missing on the location (deleted or hidden outside NG Backup)");
}
}
sort($listed);
return $listed;
} }
public function blobCount(): int { public function blobCount(): int {
@@ -368,15 +622,20 @@ final class Repository {
} }
/** @return \Generator<array{p:string, s:int, m:int, b:list<string>}> */ /** @return \Generator<array{p:string, s:int, m:int, b:list<string>}> */
private function streamEntries(string $snapshotId): \Generator { private function streamEntries(string $snapshotId, string $dir = 'snapshots/'): \Generator {
$plain = fopen('php://temp/maxmemory:' . (4 * 1048576), 'w+b'); $plain = fopen('php://temp/maxmemory:' . (4 * 1048576), 'w+b');
$fh = $this->backend->get('snapshots/' . $snapshotId); $fh = $this->backend->get($dir . $snapshotId);
$this->cipher->decrypt($fh, $plain, 'snapshot:' . $snapshotId); $this->cipher->decrypt($fh, $plain, 'snapshot:' . $snapshotId);
fclose($fh); fclose($fh);
rewind($plain); rewind($plain);
fgets($plain); // meta line fgets($plain); // meta line
while (($line = fgets($plain)) !== false) { while (($line = fgets($plain)) !== false) {
yield json_decode($line, true, 512, JSON_THROW_ON_ERROR); $entry = json_decode($line, true, 512, JSON_THROW_ON_ERROR);
if (!is_array($entry) || !is_string($entry['p'] ?? null) || !self::isSafePath($entry['p'])
|| !is_int($entry['s'] ?? null) || !is_array($entry['b'] ?? null)) {
throw new RepositoryException('Invalid entry in snapshot ' . $snapshotId);
}
yield $entry;
} }
fclose($plain); fclose($plain);
} }
+114
View File
@@ -0,0 +1,114 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Service;
use OCA\NgBackup\AppInfo\Application;
use OCA\NgBackup\Db\Run;
use OCA\NgBackup\Db\RunMapper;
use OCA\NgBackup\Db\Target;
use OCP\IGroupManager;
use OCP\Notification\IManager;
/** Notifies administrators about failed runs and locations without a recent successful backup. */
final class AlertService {
public const STALE_AFTER = 2 * 86400;
public function __construct(
private IManager $notifications,
private IGroupManager $groups,
private RunMapper $runs,
private \OCP\IAppConfig $appConfig,
private \OCP\EventDispatcher\IEventDispatcher $events,
private \OCP\IUserSession $session,
) {
}
/**
* Security-relevant change (recovery kit downloaded, location removed, append-only off,
* retention changed, snapshots forgotten): written to the audit log (admin_audit) and sent
* to every administrator, so a single compromised admin account cannot do it unnoticed.
*
* @param array<string, string> $params
*/
public function actor(): string {
return $this->session->getUser()?->getUID() ?? (PHP_SAPI === 'cli' ? 'occ' : 'unknown');
}
public function securityEvent(string $event, array $params = [], bool $notify = true): void {
$actor = $this->actor();
$params = ['actor' => $actor] + $params;
$this->events->dispatchTyped(new \OCP\Log\Audit\CriticalActionPerformedEvent(
'NG Backup: %s by %s ' . json_encode(array_diff_key($params, ['actor' => 1]), JSON_UNESCAPED_SLASHES), ['event' => $event, 'actor' => $actor]));
if ($notify) {
$this->notifyAdmins('security_event', $event . '-' . time(), ['event' => $event] + $params);
}
}
public function runFailed(Run $run, string $targetName): void {
$this->notifyAdmins('run_failed', 'run-' . $run->getId(),
['kind' => $run->getKind(), 'target' => $targetName, 'error' => mb_strimwidth((string)$run->getError(), 0, 300, '…')]);
}
/**
* Remember how much is stored on a location (for the settings page) and warn administrators
* once a day when it is at 80% of its limit or more.
*/
public function storedBytes(Target $target, int $bytes): void {
$this->appConfig->setValueString(Application::APP_ID, 'stored_' . $target->getId(), (string)$bytes);
$limit = $target->getMaxBytes();
if ($limit === null || $limit <= 0 || $bytes < 0.8 * $limit) {
return;
}
$key = 'quota_notified_' . $target->getId();
if ($this->appConfig->getValueString(Application::APP_ID, $key, '') === date('Y-m-d')) {
return;
}
$this->appConfig->setValueString(Application::APP_ID, $key, date('Y-m-d'));
$this->notifyAdmins('quota_warning', 'quota-' . $target->getId() . '-' . date('Y-m-d'), [
'target' => $target->getName(), 'percent' => (string)round($bytes / $limit * 100),
'used' => self::gib($bytes), 'limit' => self::gib($limit)]);
}
private static function gib(int $bytes): string {
$g = $bytes / 1073741824;
return sprintf($g < 10 ? '%.2f GiB' : '%.1f GiB', $g);
}
public function lastStoredBytes(Target $target): ?int {
$v = $this->appConfig->getValueString(Application::APP_ID, 'stored_' . $target->getId(), '');
return $v === '' ? null : (int)$v;
}
/** At most one notification per location per day. */
public function checkStale(Target $target): void {
$last = $this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
$lastTime = $last?->getFinishedAt() ?? $target->getCreatedAt();
if (time() - $lastTime < self::STALE_AFTER || $this->runs->findRunning($target->getId(), BackupService::KIND_FULL) !== []) {
return;
}
$key = 'stale_notified_' . $target->getId();
if ($this->appConfig->getValueString(Application::APP_ID, $key, '') === date('Y-m-d')) {
return;
}
$this->appConfig->setValueString(Application::APP_ID, $key, date('Y-m-d'));
$this->notifyAdmins('backup_stale', 'stale-' . $target->getId() . '-' . date('Y-m-d'),
['target' => $target->getName(), 'since' => $last ? date('Y-m-d H:i', $lastTime) : 'never']);
}
private function notifyAdmins(string $subject, string $objectId, array $params): void {
foreach ($this->groups->get('admin')?->getUsers() ?? [] as $admin) {
$n = $this->notifications->createNotification();
$n->setApp(Application::APP_ID)
->setUser($admin->getUID())
->setDateTime(new \DateTime())
->setObject($subject, $objectId)
->setSubject($subject, $params);
$this->notifications->notify($n);
}
}
}
+72 -14
View File
@@ -28,7 +28,7 @@ use Psr\Log\LoggerInterface;
* Excluded by default: previews, log files, the updater's backups and each user's cache and * Excluded by default: previews, log files, the updater's backups and each user's cache and
* temporary uploads (all regenerable or temporary). * temporary uploads (all regenerable or temporary).
*/ */
class BackupService { final class BackupService {
public const KIND_FULL = 'full'; public const KIND_FULL = 'full';
public function __construct( public function __construct(
@@ -41,16 +41,33 @@ class BackupService {
private IAppManager $appManager, private IAppManager $appManager,
private ILockingProvider $locking, private ILockingProvider $locking,
private LoggerInterface $logger, private LoggerInterface $logger,
private AlertService $alerts,
private LeaseService $leases,
) { ) {
} }
public function start(Target $target, string $label = ''): Run { public function start(Target $target, string $label = ''): Run {
if ($this->keys->recoveryKitConfirmation() === null) { if (!$this->keys->everConfirmed()) {
throw new \RuntimeException('Confirm the recovery kit first (occ backup:key:confirm)'); throw new \RuntimeException('Confirm the recovery kit first (occ backup:key:confirm)');
} }
if ($this->config->getSystemValue('objectstore', null) !== null) { if ($this->config->getSystemValue('objectstore', null) !== null) {
throw new \RuntimeException('Object storage as primary storage is not supported yet'); throw new \RuntimeException('Object storage as primary storage is not supported yet');
} }
// Check and create the run under the location's exclusive lock, so a cleanup cannot start
// in between (it takes the same lock and refuses while a run exists).
// A run that is already in progress is simply continued; no lease needed for that.
$running = $this->runs->findRunning($target->getId(), self::KIND_FULL);
if ($running !== []) {
return $running[0];
}
try {
return $this->leases->with(PruneService::lockKey($target), LeaseService::EXCLUSIVE, 60, fn () => $this->createRun($target, $label));
} catch (LockedException) {
throw new \RuntimeException('A cleanup or another start for ' . $target->getName() . ' is running; try again later');
}
}
private function createRun(Target $target, string $label): Run {
$running = $this->runs->findRunning($target->getId(), self::KIND_FULL); $running = $this->runs->findRunning($target->getId(), self::KIND_FULL);
if ($running !== []) { if ($running !== []) {
return $running[0]; return $running[0];
@@ -86,10 +103,18 @@ class BackupService {
* run locked and returns immediately. * run locked and returns immediately.
*/ */
public function step(Run $run, float $deadline): Run { public function step(Run $run, float $deadline): Run {
$lockKey = 'ng_backup/run/' . $run->getId(); // Leases expire on their own: a step killed by a time limit or OOM blocks nothing for long.
$ttl = (int)min(3600, max(120, $deadline - microtime(true) + 120));
try { try {
$this->locking->acquireLock($lockKey, ILockingProvider::LOCK_EXCLUSIVE); $runLease = $this->leases->acquire('run/' . $run->getId(), LeaseService::EXCLUSIVE, $ttl);
} catch (LockedException) { } catch (LockedException) {
return $run; // another process is doing this step
}
// Shared lease on the location while this step writes: a cleanup (exclusive) cannot run now.
try {
$targetLease = $this->leases->acquire('ng_backup/target/' . $run->getTargetId(), LeaseService::SHARED, $ttl);
} catch (LockedException) {
$this->leases->release($runLease);
return $run; return $run;
} }
try { try {
@@ -102,18 +127,29 @@ class BackupService {
$state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR); $state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
if ($run->getPhase() === 'db') { if ($run->getPhase() === 'db') {
@set_time_limit(0); // the dump is one transaction; it cannot be split across requests yet $dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_'));
$dbStats = []; $dbStats = $state['dbStats'] ?? [];
$manifest = (new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_')))->dump($repo, $dbStats); // With system cron the whole dump usually fits in one step (one transaction, fully
$repo->putObject('db/' . $state['files']['snapshot'], json_encode($manifest, JSON_THROW_ON_ERROR)); // consistent). Under web cron it continues per step; the snapshot then says so.
$state['files']['meta']['db'] = 'db/' . $state['files']['snapshot']; $dump = $dumper->dumpStep($repo, $state['dump'] ?? $dumper->startDump(), $deadline, $dbStats);
$state['dbStats'] = $dbStats + ['tables' => count($manifest['tables']), $state['dbStats'] = $dbStats;
'rows' => array_sum(array_column($manifest['tables'], 'rows'))]; if ($dump['done']) {
$run->setPhase('files'); $manifest = $dumper->manifest($dump);
$repo->putObject('db/' . $state['files']['snapshot'], json_encode($manifest, JSON_THROW_ON_ERROR));
$state['files']['meta']['db'] = 'db/' . $state['files']['snapshot'];
$state['files']['meta']['dbConsistent'] = $manifest['consistent'];
$state['dbStats'] += ['tables' => count($manifest['tables']), 'rows' => array_sum(array_column($manifest['tables'], 'rows')),
'steps' => $dump['steps'], 'consistent' => $manifest['consistent']];
unset($state['dump']);
$run->setPhase('files');
} else {
$state['dump'] = $dump;
}
} else { } else {
$state['files'] = BackupRun::step($repo, $state['files'], $deadline); $state['files'] = BackupRun::step($repo, $state['files'], $deadline);
if ($state['files']['phase'] === 'done') { if ($state['files']['phase'] === 'done') {
$this->recordSnapshot($run, $state['files']); $this->recordSnapshot($run, $state['files']);
$this->alerts->storedBytes($target, $repo->storedBytes());
$run->setStatus(Run::DONE); $run->setStatus(Run::DONE);
$run->setPhase('done'); $run->setPhase('done');
$run->setFinishedAt(time()); $run->setFinishedAt(time());
@@ -129,16 +165,29 @@ class BackupService {
$run->setError($e->getMessage()); $run->setError($e->getMessage());
$run->setFinishedAt(time()); $run->setFinishedAt(time());
$run->setUpdatedAt(time()); $run->setUpdatedAt(time());
return $this->runs->update($run); $run = $this->runs->update($run);
try {
$this->alerts->runFailed($run, isset($target) ? $target->getName() : '#' . $run->getTargetId());
} catch (\Throwable) {
}
return $run;
} finally { } finally {
$this->locking->releaseLock($lockKey, ILockingProvider::LOCK_EXCLUSIVE); $this->leases->release($targetLease);
$this->leases->release($runLease);
} }
} }
/** Run to completion in this process (occ, system cron), reporting progress. */ /** Run to completion in this process (occ, system cron), reporting progress. */
public function runToCompletion(Run $run, ?callable $progress = null, float $stepSeconds = 30): Run { public function runToCompletion(Run $run, ?callable $progress = null, float $stepSeconds = 30): Run {
while ($run->getStatus() === Run::RUNNING) { while ($run->getStatus() === Run::RUNNING) {
$before = $run->getUpdatedAt();
$run = $this->step($run, microtime(true) + $stepSeconds); $run = $this->step($run, microtime(true) + $stepSeconds);
if ($run->getStatus() === Run::RUNNING && $run->getUpdatedAt() === $before) {
// Step was not possible (another process holds the run, or a crashed step's lease has
// not expired yet): wait instead of spinning.
sleep(5);
$run = $this->runs->find($run->getId());
}
if ($progress !== null) { if ($progress !== null) {
$progress($run); $progress($run);
} }
@@ -177,6 +226,15 @@ class BackupService {
'data/*/cache', 'data/*/cache',
'data/*/uploads', 'data/*/uploads',
]; ];
// SQLite: the database file is in the data directory; it is backed up by the logical dump,
// and copying the live file would give an inconsistent copy.
if ($this->config->getSystemValueString('dbtype') === 'sqlite3') {
$dbName = $this->config->getSystemValueString('dbname', 'owncloud');
$exclude[] = "data/$dbName.db";
$exclude[] = "data/$dbName.db-journal";
$exclude[] = "data/$dbName.db-wal";
$exclude[] = "data/$dbName.db-shm";
}
$logFile = $this->config->getSystemValueString('logfile', ''); $logFile = $this->config->getSystemValueString('logfile', '');
if ($logFile !== '' && str_starts_with($logFile, $dataDir . '/')) { if ($logFile !== '' && str_starts_with($logFile, $dataDir . '/')) {
$exclude[] = 'data/' . substr($logFile, strlen($dataDir) + 1); $exclude[] = 'data/' . substr($logFile, strlen($dataDir) + 1);
+159 -10
View File
@@ -22,11 +22,15 @@ use OCP\Security\ICrypto;
* into the recovery kit, so backups can be restored on a fresh server with the passphrase. * into the recovery kit, so backups can be restored on a fresh server with the passphrase.
* - The admin must confirm having stored the recovery kit before the first backup runs. * - The admin must confirm having stored the recovery kit before the first backup runs.
*/ */
class KeyService { final class KeyService {
private const KEY_MASTER = 'master_key'; private const KEY_MASTER = 'master_key';
private const KEY_WRAPPED = 'master_key_wrapped'; private const KEY_WRAPPED = 'master_key_wrapped';
private const KEY_CONFIRMED = 'recovery_kit_confirmed'; private const KEY_CONFIRMED = 'recovery_kit_confirmed';
private const KEY_FINGERPRINT = 'master_key_fingerprint'; private const KEY_FINGERPRINT = 'master_key_fingerprint';
private const KEY_DELETE_DELAY = 'delete_delay_days';
private const KEY_KIT_VERSION = 'recovery_kit_version';
private const KEY_KIT_CODE = 'recovery_kit_code';
public const DEFAULT_DELETE_DELAY = 7;
private ?KeyRing $cached = null; private ?KeyRing $cached = null;
@@ -40,7 +44,15 @@ class KeyService {
return $this->appConfig->getValueString(Application::APP_ID, self::KEY_MASTER, '', true) !== ''; return $this->appConfig->getValueString(Application::APP_ID, self::KEY_MASTER, '', true) !== '';
} }
public function initialize(#[\SensitiveParameter] string $passphrase): void { /**
* @param int $deleteDelayDays how long forgotten snapshots stay in the trash on the location
* before their data is really deleted. Fixed here, at installation:
* there is deliberately no way to change it later from NG Backup.
*/
public function initialize(#[\SensitiveParameter] string $passphrase, int $deleteDelayDays = self::DEFAULT_DELETE_DELAY, string $label = 'Slot 1'): void {
if ($deleteDelayDays < 1 || $deleteDelayDays > 365) {
throw new \InvalidArgumentException('The deletion delay must be between 1 and 365 days');
}
if ($this->isInitialized()) { if ($this->isInitialized()) {
throw new \RuntimeException('A backup key already exists'); throw new \RuntimeException('A backup key already exists');
} }
@@ -50,9 +62,11 @@ class KeyService {
$keys = KeyRing::generate(); $keys = KeyRing::generate();
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER, $this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true); $this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, $this->cached = $keys;
json_encode($keys->wrap($passphrase), JSON_THROW_ON_ERROR), true, true); $this->saveSlots([['slot' => 1, 'label' => self::cleanLabel($label), 'created' => time(), 'key' => $keys->wrap($passphrase)]]);
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true); $this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY, $deleteDelayDays, true);
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, 1, true);
$this->appConfig->deleteKey(Application::APP_ID, self::KEY_CONFIRMED); $this->appConfig->deleteKey(Application::APP_ID, self::KEY_CONFIRMED);
$this->cached = $keys; $this->cached = $keys;
} }
@@ -68,9 +82,99 @@ class KeyService {
return $this->cached; return $this->cached;
} }
/** Deletion delay in days, fixed at installation (installations from before this setting get the default once). */
public function deleteDelayDays(): int {
$days = $this->appConfig->getValueInt(Application::APP_ID, self::KEY_DELETE_DELAY, 0, true);
if ($days === 0 && $this->isInitialized()) {
$days = self::DEFAULT_DELETE_DELAY;
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY, $days, true);
}
return $days;
}
public const MAX_SLOTS = 3;
/** @return list<array{slot:int, label:string, created:int}> passphrase slots, without key material */
public function slots(): array {
return array_map(fn ($s) => ['slot' => $s['slot'], 'label' => $s['label'], 'created' => $s['created']], $this->wrappedKey()['slots']);
}
/** Add a passphrase slot (e.g. one per person who must be able to restore). Returns the slot number. */
public function addSlot(#[\SensitiveParameter] string $passphrase, string $label): int {
self::checkPassphrase($passphrase);
$slots = $this->wrappedKey()['slots'];
if (count($slots) >= self::MAX_SLOTS) {
throw new \InvalidArgumentException('At most ' . self::MAX_SLOTS . ' passphrase slots');
}
$used = array_column($slots, 'slot');
$n = min(array_diff(range(1, self::MAX_SLOTS), $used));
$slots[] = ['slot' => $n, 'label' => self::cleanLabel($label), 'created' => time(), 'key' => $this->keyRing()->wrap($passphrase)];
usort($slots, fn ($a, $b) => $a['slot'] <=> $b['slot']);
$this->saveSlots($slots);
return $n;
}
/** New passphrase for a slot (e.g. its holder left). The master key and all backups stay the same. */
public function replaceSlot(int $slot, #[\SensitiveParameter] string $passphrase, ?string $label = null): void {
self::checkPassphrase($passphrase);
$slots = $this->wrappedKey()['slots'];
$found = false;
foreach ($slots as &$s) {
if ($s['slot'] === $slot) {
$s['key'] = $this->keyRing()->wrap($passphrase);
$s['created'] = time();
if ($label !== null) {
$s['label'] = self::cleanLabel($label);
}
$found = true;
}
}
unset($s);
if (!$found) {
throw new \InvalidArgumentException("There is no slot $slot");
}
$this->saveSlots($slots);
}
public function removeSlot(int $slot): void {
$slots = array_values(array_filter($this->wrappedKey()['slots'], fn ($s) => $s['slot'] !== $slot));
if (count($slots) === count($this->wrappedKey()['slots'])) {
throw new \InvalidArgumentException("There is no slot $slot");
}
if ($slots === []) {
throw new \InvalidArgumentException('The last passphrase slot cannot be removed');
}
$this->saveSlots($slots);
}
/** Every change to the slots asks for a new recovery kit (and its confirmation). */
private function saveSlots(array $slots): void {
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode(['slots' => $slots], JSON_THROW_ON_ERROR), true, true);
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, $this->kitVersion() + 1, true);
}
private static function checkPassphrase(string $passphrase): void {
if (strlen($passphrase) < 12) {
throw new \InvalidArgumentException('The passphrase must be at least 12 characters');
}
}
private static function cleanLabel(string $label): string {
$label = trim(preg_replace('/[\x00-\x1f]/', '', $label) ?? '');
return mb_substr($label === '' ? 'Slot' : $label, 0, 64);
}
public function kitVersion(): int {
return $this->appConfig->getValueInt(Application::APP_ID, self::KEY_KIT_VERSION, 1, true);
}
/** Passphrase-wrapped copy of the master key (for repository configs and the recovery kit). */ /** Passphrase-wrapped copy of the master key (for repository configs and the recovery kit). */
public function wrappedKey(): array { public function wrappedKey(): array {
return json_decode($this->appConfig->getValueString(Application::APP_ID, self::KEY_WRAPPED, '{}', true), true, 512, JSON_THROW_ON_ERROR); $data = json_decode($this->appConfig->getValueString(Application::APP_ID, self::KEY_WRAPPED, '{}', true), true, 512, JSON_THROW_ON_ERROR);
if (!isset($data['slots'])) {
$data = ['slots' => $data === [] ? [] : [['slot' => 1, 'label' => 'Slot 1', 'created' => 0, 'key' => $data]]];
}
return $data;
} }
/** Short identifier of the key, shown in the UI and the kit (not secret). */ /** Short identifier of the key, shown in the UI and the kit (not secret). */
@@ -84,23 +188,68 @@ class KeyService {
'version' => 1, 'version' => 1,
'app' => 'NG Backup', 'app' => 'NG Backup',
'fingerprint' => $this->fingerprint(), 'fingerprint' => $this->fingerprint(),
'kit_version' => $this->kitVersion(),
'confirmation_code' => $this->kitCode(),
'created' => gmdate('c'), 'created' => gmdate('c'),
'wrapped_key' => $this->wrappedKey(), 'wrapped_key' => $this->wrappedKey(),
'instructions' => 'Keep this file and your passphrase safe, outside this server. To restore on a new server: install NG Backup, ' 'slots' => $this->slots(),
. 'add the backup location, and enter this kit with your passphrase. Without both, the backups cannot be decrypted.', 'instructions' => 'Keep this file safe, outside this server. Any one of the passphrases of the slots listed here opens it. '
. 'To restore on a new server: install NG Backup, add the backup location and enter this kit with one passphrase. '
. 'Without the kit (or the location) and a passphrase, the backups cannot be decrypted.',
]; ];
} }
/**
* Short code printed in the current kit file. Typing it back when confirming proves the
* admin actually has the downloaded kit. A new code per kit version; the server only keeps
* its own copy encrypted with the instance secret.
*/
private function kitCode(): string {
$stored = json_decode($this->appConfig->getValueString(Application::APP_ID, self::KEY_KIT_CODE, '{}', true), true) ?: [];
if (($stored['version'] ?? 0) === $this->kitVersion() && isset($stored['code'])) {
return $this->crypto->decrypt($stored['code']);
}
$alphabet = 'ABCDEFGHJKLMNPQRSTUVWXYZ23456789'; // no 0/O, 1/I
$code = '';
for ($i = 0; $i < 8; $i++) {
$code .= $alphabet[random_int(0, strlen($alphabet) - 1)];
}
$code = substr($code, 0, 4) . '-' . substr($code, 4);
$this->appConfig->setValueString(Application::APP_ID, self::KEY_KIT_CODE,
json_encode(['version' => $this->kitVersion(), 'code' => $this->crypto->encrypt($code)], JSON_THROW_ON_ERROR), true, true);
return $code;
}
public function checkKitCode(string $code): bool {
$stored = json_decode($this->appConfig->getValueString(Application::APP_ID, self::KEY_KIT_CODE, '{}', true), true) ?: [];
if (($stored['version'] ?? 0) !== $this->kitVersion() || !isset($stored['code'])) {
return false; // no kit downloaded for this version yet
}
$normalise = static fn (string $c) => strtoupper(preg_replace('/[^A-Za-z0-9]/', '', $c) ?? '');
return hash_equals($normalise($this->crypto->decrypt($stored['code'])), $normalise($code));
}
public function confirmRecoveryKit(string $uid): void { public function confirmRecoveryKit(string $uid): void {
$this->appConfig->setValueString(Application::APP_ID, self::KEY_CONFIRMED, $this->appConfig->setValueString(Application::APP_ID, self::KEY_CONFIRMED,
json_encode(['uid' => $uid, 'time' => time(), 'fingerprint' => $this->fingerprint()], JSON_THROW_ON_ERROR), true); json_encode(['uid' => $uid, 'time' => time(), 'fingerprint' => $this->fingerprint(), 'kit_version' => $this->kitVersion()], JSON_THROW_ON_ERROR), true);
} }
/** Confirmation for the current key, or null. */ /**
* Whether a recovery kit for this key was ever confirmed (any passphrase version). Backups
* keep running after a passphrase change, so changing it cannot be used to stop backups; the
* missing re-confirmation is reported as an error in the admin overview instead.
*/
public function everConfirmed(): bool {
$raw = $this->appConfig->getValueString(Application::APP_ID, self::KEY_CONFIRMED, '', true);
$data = $raw === '' ? null : json_decode($raw, true);
return $data !== null && ($data['fingerprint'] ?? '') === $this->fingerprint();
}
/** Confirmation for the current key and kit version, or null. */
public function recoveryKitConfirmation(): ?array { public function recoveryKitConfirmation(): ?array {
$raw = $this->appConfig->getValueString(Application::APP_ID, self::KEY_CONFIRMED, '', true); $raw = $this->appConfig->getValueString(Application::APP_ID, self::KEY_CONFIRMED, '', true);
$data = $raw === '' ? null : json_decode($raw, true); $data = $raw === '' ? null : json_decode($raw, true);
return ($data !== null && ($data['fingerprint'] ?? '') === $this->fingerprint()) ? $data : null; return ($data !== null && ($data['fingerprint'] ?? '') === $this->fingerprint() && ($data['kit_version'] ?? 1) === $this->kitVersion()) ? $data : null;
} }
private function fingerprintOf(KeyRing $keys): string { private function fingerprintOf(KeyRing $keys): string {
+61
View File
@@ -0,0 +1,61 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Service;
/**
* Passphrase slots (like LUKS): up to three passphrases, each opening the same master key, so
* that one person leaving or dying does not lock the company out of its backups. Every change
* is written to all locations and announced to all administrators; a new recovery kit must be
* downloaded and confirmed afterwards (backups keep running meanwhile).
*/
final class KeySlotService {
public function __construct(
private KeyService $keys,
private TargetService $targets,
private AlertService $alerts,
) {
}
public function add(#[\SensitiveParameter] string $passphrase, string $label): int {
$slot = $this->keys->addSlot($passphrase, $label);
$this->apply('slot_added', $slot);
return $slot;
}
public function replace(int $slot, #[\SensitiveParameter] string $passphrase, ?string $label = null): void {
$this->keys->replaceSlot($slot, $passphrase, $label);
$this->apply('slot_replaced', $slot);
}
public function remove(int $slot): void {
$this->keys->removeSlot($slot);
$this->apply('slot_removed', $slot);
}
private function apply(string $event, int $slot): void {
$wrapped = $this->keys->wrappedKey();
$failed = [];
foreach ($this->targets->list() as $t) {
try {
$this->targets->repository($t)->updateWrappedKey($wrapped);
} catch (\Throwable $e) {
$failed[] = $t->getName();
}
}
$label = '';
foreach ($this->keys->slots() as $s) {
if ($s['slot'] === $slot) {
$label = $s['label'];
}
}
$this->alerts->securityEvent($event, ['slot' => (string)$slot, 'label' => $label]);
if ($failed !== []) {
throw new \RuntimeException('Saved, but these locations could not be updated (they still accept the previous passphrases): ' . implode(', ', $failed));
}
}
}
+88
View File
@@ -0,0 +1,88 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Service;
use OCP\IDBConnection;
use OCP\Lock\ILockingProvider;
use OCP\Lock\LockedException;
/**
* Shared/exclusive leases with an expiry, stored in ngb_leases.
*
* Nextcloud's locks live up to an hour; a PHP process killed while holding one (web cron time
* limit, OOM, kill -9) would block a location that long. A lease expires on its own unless its
* holder keeps refreshing it. Nextcloud's lock is only held for the few milliseconds in which a
* lease is checked and written.
*/
final class LeaseService {
public const SHARED = 's';
public const EXCLUSIVE = 'x';
public function __construct(
private IDBConnection $db,
private ILockingProvider $locking,
) {
}
/** @return string lease id; throws LockedException when it conflicts with a live lease */
public function acquire(string $name, string $mode, int $ttl): string {
$guard = 'ng_backup/lease/' . $name;
$this->locking->acquireLock($guard, ILockingProvider::LOCK_EXCLUSIVE);
try {
$now = time();
$qb = $this->db->getQueryBuilder();
$qb->delete('ngb_leases')
->where($qb->expr()->eq('name', $qb->createNamedParameter($name)))
->andWhere($qb->expr()->lt('expires_at', $qb->createNamedParameter($now)));
$qb->executeStatement();
$qb = $this->db->getQueryBuilder();
$qb->select('mode')->from('ngb_leases')->where($qb->expr()->eq('name', $qb->createNamedParameter($name)));
$modes = $qb->executeQuery()->fetchAll(\PDO::FETCH_COLUMN);
if ($modes !== [] && ($mode === self::EXCLUSIVE || in_array(self::EXCLUSIVE, $modes, true))) {
throw new LockedException($name);
}
$id = bin2hex(random_bytes(16));
$qb = $this->db->getQueryBuilder();
$qb->insert('ngb_leases')->values([
'id' => $qb->createNamedParameter($id),
'name' => $qb->createNamedParameter($name),
'mode' => $qb->createNamedParameter($mode),
'expires_at' => $qb->createNamedParameter($now + $ttl),
]);
$qb->executeStatement();
return $id;
} finally {
$this->locking->releaseLock($guard, ILockingProvider::LOCK_EXCLUSIVE);
}
}
/** Extend a lease while its holder is still working. */
public function refresh(string $id, int $ttl): void {
$qb = $this->db->getQueryBuilder();
$qb->update('ngb_leases')->set('expires_at', $qb->createNamedParameter(time() + $ttl))
->where($qb->expr()->eq('id', $qb->createNamedParameter($id)));
$qb->executeStatement();
}
public function release(string $id): void {
$qb = $this->db->getQueryBuilder();
$qb->delete('ngb_leases')->where($qb->expr()->eq('id', $qb->createNamedParameter($id)));
$qb->executeStatement();
}
/** Run $fn under a lease; $refresh() can be called during long work. */
public function with(string $name, string $mode, int $ttl, callable $fn): mixed {
$id = $this->acquire($name, $mode, $ttl);
try {
return $fn(fn () => $this->refresh($id, $ttl));
} finally {
$this->release($id);
}
}
}
+163
View File
@@ -0,0 +1,163 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Service;
use OCA\NgBackup\AppInfo\Application;
use OCA\NgBackup\Db\RunMapper;
use OCA\NgBackup\Db\SnapshotMapper;
use OCA\NgBackup\Db\Target;
use OCP\IAppConfig;
use OCP\Lock\ILockingProvider;
use OCP\Lock\LockedException;
/**
* Applies the retention policy to a location: forgets snapshots the policy does not keep and
* prunes data no remaining snapshot uses. Never runs on append-only locations, never while a
* backup to the same location is running (the location lock is shared with BackupService).
*/
final class PruneService {
public function __construct(
private TargetService $targets,
private RunMapper $runs,
private SnapshotMapper $snapshots,
private IAppConfig $appConfig,
private ILockingProvider $locking,
private AlertService $alerts,
private KeyService $keys,
private LeaseService $leases,
) {
}
public function untrash(Target $target, string $snapshotId): void {
$lease = $this->lockExclusive($target, 300);
try {
$this->doUntrash($target, $snapshotId);
} finally {
$this->leases->release($lease);
}
}
/**
* Run $fn under the location's exclusive lock (to create a run atomically with respect to a
* cleanup, which takes the same lock and then refuses while a run exists).
*/
public function underStartLock(Target $target, callable $fn): mixed {
try {
return $this->leases->with(self::lockKey($target), LeaseService::EXCLUSIVE, 60, fn () => $fn());
} catch (LockedException) {
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is in progress; try again later');
}
}
/**
* Exclusive lock on the location, then make sure no run of any kind (backup, restore, export)
* is in progress: their data may not be referenced by a snapshot yet, or is being read.
*/
private function lockExclusive(Target $target, int $ttl): string {
try {
$lease = $this->leases->acquire(self::lockKey($target), LeaseService::EXCLUSIVE, $ttl);
} catch (LockedException) {
throw new \RuntimeException('A backup, restore or cleanup of ' . $target->getName() . ' is in progress; try again later');
}
if ($this->runs->findRunning($target->getId()) !== []) {
$this->leases->release($lease);
throw new \RuntimeException('A backup or restore of ' . $target->getName() . ' is running; try again later');
}
return $lease;
}
private function doUntrash(Target $target, string $snapshotId): void {
$repo = $this->targets->repository($target);
if (!array_key_exists($snapshotId, $repo->trash())) {
throw new \InvalidArgumentException('That snapshot is not in the trash (or was already deleted)');
}
$repo->untrash($snapshotId);
// Put it back in the local snapshot list as well.
$meta = $repo->snapshotMeta($snapshotId);
$s = new \OCA\NgBackup\Db\Snapshot();
$s->setTargetId($target->getId());
$s->setSnapshotId($snapshotId);
$s->setKind((string)($meta['kind'] ?? 'full'));
$s->setLabel(($meta['label'] ?? '') !== '' ? $meta['label'] : null);
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
$s->setFiles((int)($meta['stats']['files'] ?? 0));
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
try {
$this->snapshots->insert($s);
} catch (\Throwable) {
}
$this->alerts->securityEvent('snapshot_untrashed', ['target' => $target->getName(), 'snapshot' => $snapshotId]);
}
public static function lockKey(Target $target): string {
return 'ng_backup/target/' . $target->getId();
}
public function policy(): RetentionPolicy {
return RetentionPolicy::fromArray(json_decode($this->appConfig->getValueString(Application::APP_ID, 'retention', '{}'), true) ?: []);
}
public function setPolicy(RetentionPolicy $policy): void {
if ($policy->last < 1) {
throw new \InvalidArgumentException('Keep at least the last snapshot');
}
$old = $this->policy()->toArray();
$this->appConfig->setValueString(Application::APP_ID, 'retention', json_encode($policy->toArray(), JSON_THROW_ON_ERROR));
if ($old !== $policy->toArray()) {
$this->alerts->securityEvent('retention_changed', ['policy' => json_encode($policy->toArray())]);
}
}
/**
* Snapshots the policy drops go to the trash on the location; trash entries older than the
* deletion delay are purged, and only then is their data freed.
*
* @return array{kept: array<string, list<string>>, forgotten: list<string>, purged: list<string>, prune: array}
*/
public function apply(Target $target, bool $dryRun = false, bool $scheduled = false): array {
if ($target->getAppendOnly()) {
throw new \RuntimeException('Location ' . $target->getName() . ' is append-only: NG Backup does not delete anything there');
}
$lease = $this->lockExclusive($target, 1800);
try {
$repo = $this->targets->repository($target);
$times = [];
foreach ($repo->listSnapshots() as $id) {
$meta = $repo->snapshotMeta($id);
$times[$id] = (int)strtotime((string)($meta['time'] ?? '@0'));
}
$keep = $this->policy()->keep($times);
$forget = array_values(array_diff(array_keys($times), array_keys($keep)));
$actor = $scheduled ? 'retention policy' : $this->alerts->actor();
$purged = [];
if (!$dryRun) {
foreach ($forget as $id) {
$repo->forget($id, $actor);
try {
$this->snapshots->delete($this->snapshots->findOne($target->getId(), $id));
} catch (\Throwable) {
}
}
}
// Also removes leftovers of interrupted runs. A dry run counts the snapshots it would forget as gone.
$delay = $this->keys->deleteDelayDays();
if (!$dryRun && $forget !== []) {
// Scheduled cleanup follows the policy: audit log only, no notification every week.
$this->alerts->securityEvent('snapshots_forgotten', ['target' => $target->getName(), 'count' => (string)count($forget),
'until' => date('Y-m-d', time() + $delay * 86400)], !$scheduled);
}
if (!$dryRun) {
$purged = $repo->purgeTrash($delay * 86400);
}
$prune = $repo->prune($dryRun);
return ['kept' => $keep, 'forgotten' => $forget, 'purged' => $purged, 'prune' => $prune];
} finally {
$this->leases->release($lease);
}
}
}
+33 -3
View File
@@ -28,7 +28,7 @@ use OCP\Files\NotFoundException;
* - merge: into the original place, existing files are overwritten (old content kept as a version) * - merge: into the original place, existing files are overwritten (old content kept as a version)
* - replace: like merge, and files that are not in the snapshot are moved to the trash * - replace: like merge, and files that are not in the snapshot are moved to the trash
*/ */
class RestoreService { final class RestoreService {
public const MODE_NEW_FOLDER = 'new-folder'; public const MODE_NEW_FOLDER = 'new-folder';
public const MODE_MERGE = 'merge'; public const MODE_MERGE = 'merge';
public const MODE_REPLACE = 'replace'; public const MODE_REPLACE = 'replace';
@@ -36,9 +36,19 @@ class RestoreService {
public function __construct( public function __construct(
private TargetService $targets, private TargetService $targets,
private IRootFolder $root, private IRootFolder $root,
private LeaseService $leases,
) { ) {
} }
/** Shared lock on the location while restoring: a cleanup (exclusive) must not delete what is being read. */
private function withSharedLock(Target $target, callable $fn, int $ttl = 600): mixed {
try {
return $this->leases->with(PruneService::lockKey($target), LeaseService::SHARED, $ttl, $fn);
} catch (\OCP\Lock\LockedException) {
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is running; try again later');
}
}
/** /**
* Direct children (files and folders) of a path in a snapshot. * Direct children (files and folders) of a path in a snapshot.
* *
@@ -72,7 +82,24 @@ class RestoreService {
* @return array{restored:int, trashed:int, target:string} * @return array{restored:int, trashed:int, target:string}
*/ */
public function restoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode = self::MODE_NEW_FOLDER, ?callable $progress = null): array { public function restoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode = self::MODE_NEW_FOLDER, ?callable $progress = null): array {
if (!preg_match('#^data/([^/]+)/files(?:/(.*))?$#', trim($logicalPath, '/'), $m)) { return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $logicalPath, $mode, $progress) {
$last = time();
// Keep the lease alive while files are being written.
$beat = function (int $done, int $total, string $name) use ($refresh, $progress, &$last): void {
if (time() - $last >= 60) {
$refresh();
$last = time();
}
if ($progress !== null) {
$progress($done, $total, $name);
}
};
return $this->doRestoreUserFiles($target, $snapshotId, $logicalPath, $mode, $beat);
});
}
private function doRestoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode, ?callable $progress): array {
if (!Repository::isSafePath($logicalPath) || !preg_match('#^data/([^/]+)/files(?:/(.*))?$#', $logicalPath, $m)) {
throw new \InvalidArgumentException('Expected a path like data/<user>/files/<folder or file>'); throw new \InvalidArgumentException('Expected a path like data/<user>/files/<folder or file>');
} }
[$uid, $rel] = [$m[1], trim($m[2] ?? '', '/')]; [$uid, $rel] = [$m[1], trim($m[2] ?? '', '/')];
@@ -124,10 +151,13 @@ class RestoreService {
/** Raw restore of any snapshot path to a local directory (administrators, disaster recovery). */ /** Raw restore of any snapshot path to a local directory (administrators, disaster recovery). */
public function restoreToDirectory(Target $target, string $snapshotId, string $prefix, string $directory): int { public function restoreToDirectory(Target $target, string $snapshotId, string $prefix, string $directory): int {
if ($prefix !== '' && !Repository::isSafePath($prefix)) {
throw new \InvalidArgumentException('Invalid path: ' . $prefix);
}
if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) { if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) {
throw new \RuntimeException("Cannot create $directory"); throw new \RuntimeException("Cannot create $directory");
} }
return $this->targets->repository($target)->restore($snapshotId, $directory, trim($prefix, '/') === '' ? '' : trim($prefix, '/') . '/'); return $this->withSharedLock($target, fn () => $this->targets->repository($target)->restore($snapshotId, $directory, trim($prefix, '/') === '' ? '' : trim($prefix, '/') . '/'), 3600);
} }
private function writeFile(Repository $repo, Folder $dest, string $relPath, array $entry): void { private function writeFile(Repository $repo, Folder $dest, string $relPath, array $entry): void {
+60
View File
@@ -0,0 +1,60 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Service;
/**
* Which snapshots to keep: the newest $last, plus the newest snapshot of each of the most recent
* $daily days, $weekly ISO weeks and $monthly months that have a snapshot (like restic/borg).
*/
final class RetentionPolicy {
public function __construct(
public readonly int $last = 3,
public readonly int $daily = 7,
public readonly int $weekly = 4,
public readonly int $monthly = 6,
) {
}
public static function fromArray(array $a): self {
return new self((int)($a['last'] ?? 3), (int)($a['daily'] ?? 7), (int)($a['weekly'] ?? 4), (int)($a['monthly'] ?? 6));
}
public function toArray(): array {
return ['last' => $this->last, 'daily' => $this->daily, 'weekly' => $this->weekly, 'monthly' => $this->monthly];
}
/**
* @param array<string, int> $snapshots id => unix time
* @return array<string, list<string>> id => reasons it is kept (ids not in the result are removed)
*/
public function keep(array $snapshots): array {
arsort($snapshots);
$keep = [];
$i = 0;
foreach ($snapshots as $id => $time) {
if ($i++ < $this->last) {
$keep[$id][] = 'last';
}
}
foreach (['daily' => 'Y-m-d', 'weekly' => 'o-W', 'monthly' => 'Y-m'] as $kind => $format) {
$seen = [];
foreach ($snapshots as $id => $time) {
$bucket = date($format, $time);
if (isset($seen[$bucket])) {
continue;
}
if (count($seen) >= $this->$kind) {
break;
}
$seen[$bucket] = true;
$keep[$id][] = $kind;
}
}
return $keep;
}
}
+43 -8
View File
@@ -22,13 +22,14 @@ use OCP\Server;
* object is built only while it is used; nothing is mounted. files_external is loaded for its * object is built only while it is used; nothing is mounted. files_external is loaded for its
* classes but its enabled/disabled state is left as the admin set it. * classes but its enabled/disabled state is left as the admin set it.
*/ */
class TargetService { final class TargetService {
public function __construct( public function __construct(
private TargetMapper $mapper, private TargetMapper $mapper,
private KeyService $keys, private KeyService $keys,
private ICrypto $crypto, private ICrypto $crypto,
private IAppManager $appManager, private IAppManager $appManager,
private \OCA\NgBackup\Db\DbIndexCache $indexCache, private \OCA\NgBackup\Db\DbIndexCache $indexCache,
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
) { ) {
} }
@@ -56,10 +57,10 @@ class TargetService {
$keys = $this->keys->keyRing(); $keys = $this->keys->keyRing();
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath); $storageBackend = $this->buildBackend($backend, $auth, $options, $basePath);
if ($storageBackend->exists('config')) { if ($storageBackend->exists('config')) {
$repo = Repository::openWithKey($storageBackend, $keys); $repo = Repository::openWithKey($storageBackend, $keys, $this->indexCache, $this->anchor);
$created = false; $created = false;
} else { } else {
$repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey()); $repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey(), $this->anchor);
$created = true; $created = true;
} }
@@ -89,6 +90,17 @@ class TargetService {
return ctype_digit($nameOrId) ? $this->mapper->find((int)$nameOrId) : $this->mapper->findByName($nameOrId); return ctype_digit($nameOrId) ? $this->mapper->find((int)$nameOrId) : $this->mapper->findByName($nameOrId);
} }
/** Size limit for the data stored on this location (null = none). */
public function setMaxBytes(Target $target, ?int $maxBytes): Target {
$target->setMaxBytes($maxBytes);
return $this->mapper->update($target);
}
public function setAppendOnly(Target $target, bool $appendOnly): Target {
$target->setAppendOnly($appendOnly);
return $this->mapper->update($target);
}
/** Remove the target from NG Backup. The backups on the target itself are not touched. */ /** Remove the target from NG Backup. The backups on the target itself are not touched. */
public function remove(Target $target): void { public function remove(Target $target): void {
$this->mapper->delete($target); $this->mapper->delete($target);
@@ -99,10 +111,11 @@ class TargetService {
} }
public function repository(Target $target): Repository { public function repository(Target $target): Repository {
$repo = Repository::openWithKey($this->backend($target), $this->keys->keyRing(), $this->indexCache); $repo = Repository::openWithKey($this->backend($target), $this->keys->keyRing(), $this->indexCache, $this->anchor);
if ($target->getRepositoryId() !== null && $repo->id() !== $target->getRepositoryId()) { if ($target->getRepositoryId() !== null && $repo->id() !== $target->getRepositoryId()) {
throw new \RuntimeException('The target now contains a different repository than when it was added'); throw new \RuntimeException('The target now contains a different repository than when it was added');
} }
$repo->setMaxBytes($target->getMaxBytes());
return $repo; return $repo;
} }
@@ -115,11 +128,33 @@ class TargetService {
return new StorageBackend($this->factory()->create($backend, $auth, $options), $basePath); return new StorageBackend($this->factory()->create($backend, $auth, $options), $basePath);
} }
private ?ExternalStorageFactory $factory = null;
/**
* files_external's connection code, without changing the app's state:
* - enabled: it is booted by Nextcloud; use its own BackendService;
* - disabled: do NOT boot it (booting registers its mount provider and listeners, which use
* its database tables; on installations where it was never enabled those do not exist).
* Only its classes are loaded, and a private BackendService gets its backends and auth
* mechanisms from its Application class.
*/
private function factory(): ExternalStorageFactory { private function factory(): ExternalStorageFactory {
// Load files_external's classes without enabling the app (its state stays as the admin set it). if ($this->factory !== null) {
if (!class_exists(\OCA\Files_External\Service\BackendService::class)) { return $this->factory;
$this->appManager->loadApp('files_external');
} }
return new ExternalStorageFactory(Server::get(\OCA\Files_External\Service\BackendService::class)); if ($this->appManager->isEnabledForAnyone('files_external')) {
$this->appManager->loadApp('files_external');
return $this->factory = new ExternalStorageFactory(Server::get(\OCA\Files_External\Service\BackendService::class));
}
if (!class_exists(\OCA\Files_External\AppInfo\Application::class)) {
$path = $this->appManager->getAppPath('files_external');
require_once $path . '/composer/autoload.php';
}
$service = new \OCA\Files_External\Service\BackendService(
Server::get(\OCP\IAppConfig::class), Server::get(\Psr\Log\LoggerInterface::class));
$app = new \OCA\Files_External\AppInfo\Application();
$service->registerBackendProvider($app);
$service->registerAuthMechanismProvider($app);
return $this->factory = new ExternalStorageFactory($service);
} }
} }
+1 -1
View File
@@ -11,7 +11,7 @@ use OCP\AppFramework\Http\TemplateResponse;
use OCP\Settings\ISettings; use OCP\Settings\ISettings;
use OCP\Util; use OCP\Util;
class Admin implements ISettings { final class Admin implements ISettings {
public function getForm(): TemplateResponse { public function getForm(): TemplateResponse {
Util::addScript('ng_backup', 'ng_backup-admin'); Util::addScript('ng_backup', 'ng_backup-admin');
Util::addStyle('ng_backup', 'ng_backup-admin'); Util::addStyle('ng_backup', 'ng_backup-admin');
+1 -1
View File
@@ -11,7 +11,7 @@ use OCP\IL10N;
use OCP\IURLGenerator; use OCP\IURLGenerator;
use OCP\Settings\IIconSection; use OCP\Settings\IIconSection;
class AdminSection implements IIconSection { final class AdminSection implements IIconSection {
public function __construct( public function __construct(
private IL10N $l, private IL10N $l,
private IURLGenerator $url, private IURLGenerator $url,
+71
View File
@@ -0,0 +1,71 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\SetupCheck;
use OCA\NgBackup\Db\Run;
use OCA\NgBackup\Db\RunMapper;
use OCA\NgBackup\Service\AlertService;
use OCA\NgBackup\Service\BackupService;
use OCA\NgBackup\Service\KeyService;
use OCA\NgBackup\Service\TargetService;
use OCP\IL10N;
use OCP\IURLGenerator;
use OCP\SetupCheck\ISetupCheck;
use OCP\SetupCheck\SetupResult;
/** Shows the backup state in the admin overview (Security & setup warnings). */
final class BackupStatusCheck implements ISetupCheck {
public function __construct(
private IL10N $l,
private IURLGenerator $url,
private KeyService $keys,
private TargetService $targets,
private RunMapper $runs,
) {
}
public function getCategory(): string {
return 'system';
}
public function getName(): string {
return $this->l->t('NG Backup');
}
public function run(): SetupResult {
$link = $this->url->linkToRouteAbsolute('settings.AdminSettings.index', ['section' => 'ng_backup']);
if (!$this->keys->isInitialized() || !$this->keys->everConfirmed()) {
return SetupResult::error($this->l->t('Backups are not set up: create the backup key and confirm the recovery kit.'), $link);
}
if ($this->keys->recoveryKitConfirmation() === null) {
return SetupResult::error($this->l->t('The backup passphrase was changed: download and confirm the new recovery kit.'), $link);
}
$targets = $this->targets->list();
if ($targets === []) {
return SetupResult::warning($this->l->t('No backup location is configured.'), $link);
}
$problems = [];
foreach ($targets as $t) {
$last = $this->runs->lastFinished($t->getId(), BackupService::KIND_FULL);
if ($last === null) {
$problems[] = $this->l->t('%s: no successful backup yet', [$t->getName()]);
} elseif (time() - (int)$last->getFinishedAt() > AlertService::STALE_AFTER) {
$problems[] = $this->l->t('%1$s: last successful backup %2$s', [$t->getName(), date('Y-m-d H:i', (int)$last->getFinishedAt())]);
}
}
foreach ($this->runs->findRecent(10) as $r) {
if ($r->getStatus() === Run::FAILED && $r->getStartedAt() > time() - AlertService::STALE_AFTER) {
$problems[] = $this->l->t('A %1$s run failed: %2$s', [$r->getKind(), mb_strimwidth((string)$r->getError(), 0, 120, '…')]);
break;
}
}
return $problems === []
? SetupResult::success($this->l->t('Recent backups to all locations succeeded.'))
: SetupResult::warning(implode("\n", $problems), $link);
}
}
+18 -6
View File
@@ -8,6 +8,7 @@ declare(strict_types=1);
namespace OCA\NgBackup\UserMigration; namespace OCA\NgBackup\UserMigration;
use OCA\NgBackup\Repository\Repository; use OCA\NgBackup\Repository\Repository;
use OCP\Files\File;
use OCP\Files\Folder; use OCP\Files\Folder;
use OCP\UserMigration\IImportSource; use OCP\UserMigration\IImportSource;
use OCP\UserMigration\UserMigrationException; use OCP\UserMigration\UserMigrationException;
@@ -30,10 +31,14 @@ final class RepositoryImportSource implements IImportSource {
return $data; return $data;
} }
/** @return resource */
public function getFileAsStream(string $path) { public function getFileAsStream(string $path) {
$entry = $this->entry($path, 'f'); $entry = $this->entry($path, 'f');
// Phase 0: buffer in php://temp (memory up to 8 MiB, then a temp file for this one file). // Phase 0: buffer in php://temp (memory up to 8 MiB, then a temp file for this one file).
$stream = fopen('php://temp/maxmemory:' . (8 * 1048576), 'w+b'); $stream = fopen('php://temp/maxmemory:' . (8 * 1048576), 'w+b');
if ($stream === false) {
throw new UserMigrationException('Cannot buffer ' . $path);
}
foreach ($this->repo->readBlobs($entry['b']) as $data) { foreach ($this->repo->readBlobs($entry['b']) as $data) {
for ($done = 0, $len = strlen($data); $done < $len; $done += $n) { for ($done = 0, $len = strlen($data); $done < $len; $done += $n) {
$n = fwrite($stream, $done === 0 ? $data : substr($data, $done)); $n = fwrite($stream, $done === 0 ? $data : substr($data, $done));
@@ -51,6 +56,7 @@ final class RepositoryImportSource implements IImportSource {
$prefix = trim($path, '/'); $prefix = trim($path, '/');
$prefix = $prefix === '' ? '' : $prefix . '/'; $prefix = $prefix === '' ? '' : $prefix . '/';
$names = []; $names = [];
/** @var string $p */
foreach (array_keys($this->manifest['entries']) as $p) { foreach (array_keys($this->manifest['entries']) as $p) {
if ($prefix === '' || str_starts_with($p, $prefix)) { if ($prefix === '' || str_starts_with($p, $prefix)) {
$rest = substr($p, strlen($prefix)); $rest = substr($p, strlen($prefix));
@@ -70,6 +76,7 @@ final class RepositoryImportSource implements IImportSource {
if (isset($this->manifest['entries'][$p])) { if (isset($this->manifest['entries'][$p])) {
return true; return true;
} }
/** @var string $k */
foreach (array_keys($this->manifest['entries']) as $k) { foreach (array_keys($this->manifest['entries']) as $k) {
if (str_starts_with($k, $p . '/')) { if (str_starts_with($k, $p . '/')) {
return true; return true;
@@ -91,12 +98,17 @@ final class RepositoryImportSource implements IImportSource {
$this->copyToFolder($sub, $path); $this->copyToFolder($sub, $path);
} else { } else {
$stream = $this->getFileAsStream($path); $stream = $this->getFileAsStream($path);
if ($destination->nodeExists($name)) { try {
$destination->get($name)->putContent($stream); if ($destination->nodeExists($name)) {
} else { $existing = $destination->get($name);
$destination->newFile($name, $stream); if (!$existing instanceof File) {
} throw new UserMigrationException("$path exists and is not a file");
if (is_resource($stream)) { }
$existing->putContent($stream);
} else {
$destination->newFile($name, $stream);
}
} finally {
fclose($stream); fclose($stream);
} }
} }
+19
View File
@@ -12,6 +12,7 @@
"@nextcloud/axios": "^2.5.1", "@nextcloud/axios": "^2.5.1",
"@nextcloud/dialogs": "^7.4.0", "@nextcloud/dialogs": "^7.4.0",
"@nextcloud/l10n": "^3.4.0", "@nextcloud/l10n": "^3.4.0",
"@nextcloud/password-confirmation": "^6.1.0",
"@nextcloud/router": "^3.0.1", "@nextcloud/router": "^3.0.1",
"@nextcloud/vue": "^9.0.0", "@nextcloud/vue": "^9.0.0",
"vue": "^3.5.0" "vue": "^3.5.0"
@@ -885,6 +886,24 @@
"node": "^20.0.0 || ^22.0.0 || ^24.0.0" "node": "^20.0.0 || ^22.0.0 || ^24.0.0"
} }
}, },
"node_modules/@nextcloud/password-confirmation": {
"version": "6.1.0",
"resolved": "https://registry.npmjs.org/@nextcloud/password-confirmation/-/password-confirmation-6.1.0.tgz",
"integrity": "sha512-N2ChXDbPcUcQCoAjj1yc65zfc61yiKpdM3qm/y3Y/y//NG7XWNNINwBg85lqJ2SISgmVStpsQ1d0blpFCoQXkQ==",
"license": "MIT",
"dependencies": {
"@nextcloud/auth": "^2.5.3",
"@nextcloud/axios": "^2.5.2",
"@nextcloud/l10n": "^3.4.1",
"@nextcloud/logger": "^3.0.3",
"@nextcloud/router": "^3.1.0",
"@nextcloud/vue": "^9.6.0",
"vue": "^3.5.30"
},
"engines": {
"node": "^20.0.0 || ^22.0.0 || ^24.0.0"
}
},
"node_modules/@nextcloud/paths": { "node_modules/@nextcloud/paths": {
"version": "3.1.0", "version": "3.1.0",
"resolved": "https://registry.npmjs.org/@nextcloud/paths/-/paths-3.1.0.tgz", "resolved": "https://registry.npmjs.org/@nextcloud/paths/-/paths-3.1.0.tgz",
+1
View File
@@ -12,6 +12,7 @@
"@nextcloud/axios": "^2.5.1", "@nextcloud/axios": "^2.5.1",
"@nextcloud/dialogs": "^7.4.0", "@nextcloud/dialogs": "^7.4.0",
"@nextcloud/l10n": "^3.4.0", "@nextcloud/l10n": "^3.4.0",
"@nextcloud/password-confirmation": "^6.1.0",
"@nextcloud/router": "^3.0.1", "@nextcloud/router": "^3.0.1",
"@nextcloud/vue": "^9.0.0", "@nextcloud/vue": "^9.0.0",
"vue": "^3.5.0" "vue": "^3.5.0"
+497
View File
@@ -0,0 +1,497 @@
<?xml version="1.0" encoding="UTF-8"?>
<files psalm-version="6.19.1@e2ca44251c1f1aa2e35452d426895746b47de64e">
<file src="lib/Backend/ExternalStorageFactory.php">
<MissingClosureParamType>
<code><![CDATA[$p]]></code>
<code><![CDATA[$p]]></code>
</MissingClosureParamType>
<MissingClosureReturnType>
<code><![CDATA[fn ($p) => $p->jsonSerialize()]]></code>
<code><![CDATA[fn ($p) => $p->jsonSerialize()]]></code>
</MissingClosureReturnType>
</file>
<file src="lib/Backend/IBackend.php">
<MissingParamType>
<code><![CDATA[$stream]]></code>
</MissingParamType>
</file>
<file src="lib/Backend/LocalBackend.php">
<MissingParamType>
<code><![CDATA[$stream]]></code>
</MissingParamType>
<TypeDoesNotContainType>
<code><![CDATA[$out !== false]]></code>
</TypeDoesNotContainType>
</file>
<file src="lib/Backend/StorageBackend.php">
<MissingParamType>
<code><![CDATA[$stream]]></code>
</MissingParamType>
</file>
<file src="lib/BackgroundJob/BackupJob.php">
<MissingParamType>
<code><![CDATA[$argument]]></code>
</MissingParamType>
</file>
<file src="lib/BackgroundJob/RestoreJob.php">
<MissingParamType>
<code><![CDATA[$argument]]></code>
</MissingParamType>
</file>
<file src="lib/Command/Browse.php">
<InvalidOperand>
<code><![CDATA[$b /= 1024]]></code>
</InvalidOperand>
</file>
<file src="lib/Command/KeyConfirm.php">
<UndefinedInterfaceMethod>
<code><![CDATA[ask]]></code>
<code><![CDATA[ask]]></code>
</UndefinedInterfaceMethod>
</file>
<file src="lib/Command/KeyInit.php">
<UndefinedInterfaceMethod>
<code><![CDATA[ask]]></code>
<code><![CDATA[ask]]></code>
</UndefinedInterfaceMethod>
</file>
<file src="lib/Command/KeySlots.php">
<UndefinedInterfaceMethod>
<code><![CDATA[ask]]></code>
<code><![CDATA[ask]]></code>
</UndefinedInterfaceMethod>
</file>
<file src="lib/Command/Prune.php">
<PossiblyFalseOperand>
<code><![CDATA[json_encode($this->prune->policy()->toArray())]]></code>
</PossiblyFalseOperand>
</file>
<file src="lib/Command/RunBackup.php">
<PossiblyNullArgument>
<code><![CDATA[$run->getSnapshotId()]]></code>
</PossiblyNullArgument>
</file>
<file src="lib/Command/Status.php">
<RiskyTruthyFalsyComparison>
<code><![CDATA[$c]]></code>
<code><![CDATA[$r->getFinishedAt()]]></code>
</RiskyTruthyFalsyComparison>
</file>
<file src="lib/Command/TargetAdd.php">
<PossiblyNullArgument>
<code><![CDATA[$r['target']->getRepositoryId()]]></code>
</PossiblyNullArgument>
<PossiblyUndefinedArrayOffset>
<code><![CDATA[$v]]></code>
</PossiblyUndefinedArrayOffset>
</file>
<file src="lib/Command/TargetBackends.php">
<UndefinedClass>
<code><![CDATA[Base]]></code>
</UndefinedClass>
</file>
<file src="lib/Command/TargetLimit.php">
<InvalidOperand>
<code><![CDATA[(float)$m[1] * $factor]]></code>
</InvalidOperand>
</file>
<file src="lib/Command/TargetList.php">
<UndefinedClass>
<code><![CDATA[Base]]></code>
</UndefinedClass>
</file>
<file src="lib/Command/TargetTrustCurrent.php">
<UndefinedInterfaceMethod>
<code><![CDATA[ask]]></code>
</UndefinedInterfaceMethod>
</file>
<file src="lib/Command/Trash.php">
<RedundantCastGivenDocblockType>
<code><![CDATA[(string)($info['time'] ?? '')]]></code>
</RedundantCastGivenDocblockType>
</file>
<file src="lib/Controller/AdminApiController.php">
<InvalidOperand>
<code><![CDATA[$maxGb * 1073741824]]></code>
<code><![CDATA[$maxGb * 1073741824]]></code>
</InvalidOperand>
<PossiblyNullArgument>
<code><![CDATA[$r->getSnapshotId()]]></code>
</PossiblyNullArgument>
</file>
<file src="lib/Crypto/StreamCipher.php">
<MissingParamType>
<code><![CDATA[$in]]></code>
<code><![CDATA[$out]]></code>
</MissingParamType>
<PossiblyFalseArgument>
<code><![CDATA[$in]]></code>
<code><![CDATA[$in]]></code>
<code><![CDATA[$in]]></code>
<code><![CDATA[$in]]></code>
<code><![CDATA[$in]]></code>
<code><![CDATA[$in]]></code>
<code><![CDATA[$out]]></code>
<code><![CDATA[$out]]></code>
<code><![CDATA[$out]]></code>
<code><![CDATA[$out]]></code>
<code><![CDATA[$out]]></code>
<code><![CDATA[$out]]></code>
</PossiblyFalseArgument>
<PossiblyInvalidArrayAccess>
<code><![CDATA[unpack('N', $len)[1]]]></code>
</PossiblyInvalidArrayAccess>
</file>
<file src="lib/Db/AppConfigCatalogAnchor.php">
<PossiblyFalseArgument>
<code><![CDATA[json_encode(['gen' => $gen, 'hash' => $hash])]]></code>
</PossiblyFalseArgument>
</file>
<file src="lib/Db/DbDumper.php">
<InvalidArrayOffset>
<code><![CDATA[$info['sha256']]]></code>
<code><![CDATA[$manifest['sequences']]]></code>
</InvalidArrayOffset>
<MissingClosureReturnType>
<code><![CDATA[fn ($c) => $end[array_search($c, $columns, true)]]]></code>
<code><![CDATA[fn ($c) => $q($c)]]></code>
<code><![CDATA[fn (string $c) => $this->db->quoteIdentifier($c)]]></code>
<code><![CDATA[fn (string $c) => $this->db->quoteIdentifier($c)]]></code>
</MissingClosureReturnType>
<PossiblyFalseArgument>
<code><![CDATA[$value]]></code>
</PossiblyFalseArgument>
<PossiblyNullArgument>
<code><![CDATA[$columns]]></code>
</PossiblyNullArgument>
<UndefinedClass>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[$this->db]]></code>
<code><![CDATA[TransactionIsolationLevel]]></code>
<code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
<code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
<code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
<code><![CDATA[\Doctrine\DBAL\Platforms\PostgreSQLPlatform]]></code>
<code><![CDATA[\Doctrine\DBAL\Platforms\PostgreSQLPlatform]]></code>
<code><![CDATA[\Doctrine\DBAL\Platforms\PostgreSQLPlatform]]></code>
<code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
<code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
<code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
<code><![CDATA[private]]></code>
</UndefinedClass>
</file>
<file src="lib/Db/Run.php">
<PropertyNotSetInConstructor>
<code><![CDATA[Run]]></code>
</PropertyNotSetInConstructor>
</file>
<file src="lib/Db/Snapshot.php">
<PropertyNotSetInConstructor>
<code><![CDATA[Snapshot]]></code>
</PropertyNotSetInConstructor>
</file>
<file src="lib/Db/Target.php">
<PropertyNotSetInConstructor>
<code><![CDATA[Target]]></code>
</PropertyNotSetInConstructor>
</file>
<file src="lib/Job/BackupRun.php">
<ArgumentTypeCoercion>
<code><![CDATA[$blobs]]></code>
</ArgumentTypeCoercion>
<MissingParamType>
<code><![CDATA[$fh]]></code>
</MissingParamType>
<PossiblyNullIterator>
<code><![CDATA[$segments]]></code>
</PossiblyNullIterator>
<PossiblyUndefinedArrayOffset>
<code><![CDATA[$mtime]]></code>
<code><![CDATA[$state['cur']]]></code>
<code><![CDATA[$state['phase']]]></code>
<code><![CDATA[$state['segments']]]></code>
<code><![CDATA[$state['stats']]]></code>
</PossiblyUndefinedArrayOffset>
</file>
<file src="lib/Job/ParentCursor.php">
<PossiblyNullReference>
<code><![CDATA[next]]></code>
</PossiblyNullReference>
</file>
<file src="lib/Job/TreeWalker.php">
<RedundantCastGivenDocblockType>
<code><![CDATA[(string)$name]]></code>
</RedundantCastGivenDocblockType>
</file>
<file src="lib/Migration/Version000001Date20261002000000.php">
<UndefinedDocblockClass>
<code><![CDATA[$schema->createTable('ngb_targets')]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
</UndefinedDocblockClass>
</file>
<file src="lib/Migration/Version000002Date20261002010000.php">
<UndefinedDocblockClass>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
</UndefinedDocblockClass>
</file>
<file src="lib/Migration/Version000003Date20261002020000.php">
<UndefinedDocblockClass>
<code><![CDATA[$schema->getTable('ngb_targets')]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
</UndefinedDocblockClass>
</file>
<file src="lib/Migration/Version000004Date20261002030000.php">
<UndefinedDocblockClass>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
</UndefinedDocblockClass>
</file>
<file src="lib/Migration/Version000005Date20261002040000.php">
<UndefinedDocblockClass>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
</UndefinedDocblockClass>
</file>
<file src="lib/Repository/BlobIndex.php">
<PossiblyFalseArgument>
<code><![CDATA[$stream]]></code>
<code><![CDATA[$stream]]></code>
</PossiblyFalseArgument>
</file>
<file src="lib/Repository/Catalog.php">
<PossiblyFalseArgument>
<code><![CDATA[$stream]]></code>
<code><![CDATA[$stream]]></code>
</PossiblyFalseArgument>
</file>
<file src="lib/Repository/PackWriter.php">
<InvalidPropertyAssignmentValue>
<code><![CDATA[$this->buffer]]></code>
</InvalidPropertyAssignmentValue>
<PossiblyFalsePropertyAssignmentValue>
<code><![CDATA[fopen('php://temp/maxmemory:' . ($this->targetSize + 8 * 1048576), 'w+b')]]></code>
</PossiblyFalsePropertyAssignmentValue>
<RedundantPropertyInitializationCheck>
<code><![CDATA[isset($this->buffer)]]></code>
</RedundantPropertyInitializationCheck>
</file>
<file src="lib/Repository/Repository.php">
<InvalidOperand>
<code><![CDATA[strlen($data) * 0.95]]></code>
</InvalidOperand>
<InvalidReturnStatement>
<code><![CDATA[['snapshot' => $snapshotId] + $stats]]></code>
</InvalidReturnStatement>
<InvalidReturnType>
<code><![CDATA[array{snapshot:string, files:int, reused:int, read:int, newBlobs:int, dupBlobs:int, uploaded:int}]]></code>
</InvalidReturnType>
<MissingParamType>
<code><![CDATA[$fh]]></code>
</MissingParamType>
<PossiblyFalseArgument>
<code><![CDATA[$encrypted]]></code>
<code><![CDATA[$encrypted]]></code>
<code><![CDATA[$encrypted]]></code>
<code><![CDATA[$encrypted]]></code>
<code><![CDATA[$fh]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$plain]]></code>
<code><![CDATA[$s]]></code>
<code><![CDATA[$s]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
<code><![CDATA[$tree]]></code>
</PossiblyFalseArgument>
<PossiblyUndefinedArrayOffset>
<code><![CDATA[$stats['files']]]></code>
<code><![CDATA[$stats['reused']]]></code>
</PossiblyUndefinedArrayOffset>
<PropertyNotSetInConstructor>
<code><![CDATA[$catalog]]></code>
</PropertyNotSetInConstructor>
</file>
<file src="lib/Service/AlertService.php">
<InvalidOperand>
<code><![CDATA[$bytes / $limit * 100]]></code>
<code><![CDATA[0.8 * $limit]]></code>
</InvalidOperand>
<PossiblyFalseOperand>
<code><![CDATA[json_encode(array_diff_key($params, ['actor' => 1]), JSON_UNESCAPED_SLASHES)]]></code>
</PossiblyFalseOperand>
</file>
<file src="lib/Service/BackupService.php">
<InvalidOperand>
<code><![CDATA[$deadline - microtime(true) + 120]]></code>
</InvalidOperand>
<UndefinedClass>
<code><![CDATA[\OC]]></code>
<code><![CDATA[\OC]]></code>
</UndefinedClass>
<UndefinedInterfaceMethod>
<code><![CDATA[getInner]]></code>
</UndefinedInterfaceMethod>
</file>
<file src="lib/Service/KeyService.php">
<ArgumentTypeCoercion>
<code><![CDATA[array_diff(range(1, self::MAX_SLOTS), $used)]]></code>
</ArgumentTypeCoercion>
<InvalidReturnStatement>
<code><![CDATA[[
'version' => 1,
'app' => 'NG Backup',
'fingerprint' => $this->fingerprint(),
'kit_version' => $this->kitVersion(),
'confirmation_code' => $this->kitCode(),
'created' => gmdate('c'),
'wrapped_key' => $this->wrappedKey(),
'slots' => $this->slots(),
'instructions' => 'Keep this file safe, outside this server. Any one of the passphrases of the slots listed here opens it. '
. 'To restore on a new server: install NG Backup, add the backup location and enter this kit with one passphrase. '
. 'Without the kit (or the location) and a passphrase, the backups cannot be decrypted.',
]]]></code>
</InvalidReturnStatement>
<InvalidReturnType>
<code><![CDATA[array{version:int, app:string, fingerprint:string, created:string, wrapped_key:array, instructions:string}]]></code>
</InvalidReturnType>
<LessSpecificReturnStatement>
<code><![CDATA[array_map(fn ($s) => ['slot' => $s['slot'], 'label' => $s['label'], 'created' => $s['created']], $this->wrappedKey()['slots'])]]></code>
</LessSpecificReturnStatement>
<MissingClosureParamType>
<code><![CDATA[$s]]></code>
<code><![CDATA[$s]]></code>
</MissingClosureParamType>
<MissingClosureReturnType>
<code><![CDATA[static fn (string $c) => strtoupper(preg_replace('/[^A-Za-z0-9]/', '', $c) ?? '')]]></code>
</MissingClosureReturnType>
<MoreSpecificReturnType>
<code><![CDATA[list<array{slot:int, label:string, created:int}>]]></code>
</MoreSpecificReturnType>
<PossiblyFalseArgument>
<code><![CDATA[base64_decode($this->crypto->decrypt($enc), true)]]></code>
</PossiblyFalseArgument>
</file>
<file src="lib/Service/PruneService.php">
<LessSpecificReturnStatement>
<code><![CDATA[['kept' => $keep, 'forgotten' => $forget, 'purged' => $purged, 'prune' => $prune]]]></code>
</LessSpecificReturnStatement>
<MissingClosureReturnType>
<code><![CDATA[fn () => $fn()]]></code>
</MissingClosureReturnType>
<MoreSpecificReturnType>
<code><![CDATA[array{kept: array<string, list<string>>, forgotten: list<string>, purged: list<string>, prune: array}]]></code>
</MoreSpecificReturnType>
</file>
<file src="lib/Service/RestoreService.php">
<LessSpecificReturnStatement>
<code><![CDATA[array_values($children)]]></code>
</LessSpecificReturnStatement>
<MissingDependency>
<code><![CDATA[$this->root]]></code>
<code><![CDATA[private]]></code>
</MissingDependency>
<MoreSpecificReturnType>
<code><![CDATA[list<array{name:string, type:string, size:int, mtime:int|null, files:int}>]]></code>
</MoreSpecificReturnType>
<PossiblyFalseArgument>
<code><![CDATA[$in]]></code>
<code><![CDATA[$out]]></code>
<code><![CDATA[$out]]></code>
</PossiblyFalseArgument>
<PossiblyNullArrayOffset>
<code><![CDATA[$children]]></code>
<code><![CDATA[$children]]></code>
</PossiblyNullArrayOffset>
<RedundantCastGivenDocblockType>
<code><![CDATA[(string)$relPath]]></code>
<code><![CDATA[(string)$relPath]]></code>
</RedundantCastGivenDocblockType>
</file>
<file src="lib/Service/TargetService.php">
<InvalidReturnStatement>
<code><![CDATA[$this->factory()->describeBackends()]]></code>
</InvalidReturnStatement>
<InvalidReturnType>
<code><![CDATA[list<array{id:string, name:string, auth:list<string>, parameters:array}>]]></code>
</InvalidReturnType>
<UndefinedDocblockClass>
<code><![CDATA[Server::get(\OCA\Files_External\Service\BackendService::class)]]></code>
</UndefinedDocblockClass>
<UnresolvableInclude>
<code><![CDATA[require_once $path . '/composer/autoload.php']]></code>
</UnresolvableInclude>
</file>
</files>
+45
View File
@@ -0,0 +1,45 @@
<?xml version="1.0"?>
<!-- SPDX-FileCopyrightText: 2026 Dillard Blom -->
<!-- SPDX-License-Identifier: AGPL-3.0-or-later -->
<psalm
errorLevel="2"
resolveFromConfigFile="true"
findUnusedBaselineEntry="true"
findUnusedCode="false"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="https://getpsalm.org/schema/config"
xsi:schemaLocation="https://getpsalm.org/schema/config vendor/vimeo/psalm/config.xsd"
errorBaseline="psalm-baseline.xml"
>
<projectFiles>
<directory name="lib" />
<ignoreFiles>
<directory name="vendor" />
</ignoreFiles>
</projectFiles>
<!-- Interface-only stubs for Nextcloud's public API, not autoloaded at runtime: scanned for
type resolution only, never reported on. lib/ must only use OCP\, never internal OC\
classes, since those differ between the supported Nextcloud versions (PLAN.md section 6). -->
<extraFiles>
<directory name="vendor/nextcloud/ocp/OCP" />
<directory name="vendor/nextcloud/ocp/NCU" />
</extraFiles>
<issueHandlers>
<UndefinedClass>
<errorLevel type="suppress">
<!-- Provided by files_external at runtime; not a composer dependency (PLAN.md 3.5). -->
<referencedClass name="OCA\Files_External\**" />
</errorLevel>
</UndefinedClass>
<!-- The #[\Override] attribute convention (PHP 8.3+) adds no behaviour; this app supports
PHP 8.1+, and applying it to every interface-implementing method project-wide is pure
ceremony with no bug-catching value, so it's suppressed rather than papered over. -->
<MissingOverrideAttribute>
<errorLevel type="suppress">
<directory name="lib" />
</errorLevel>
</MissingOverrideAttribute>
</issueHandlers>
</psalm>
+22 -6
View File
@@ -48,9 +48,11 @@ $seqStates = static function () use ($db, $prefix): array {
return $out; return $out;
}; };
// Let one sequence run ahead of the data (deleted rows, rollbacks): MAX+1 would be wrong. // Let one sequence run ahead of the data (deleted rows, rollbacks): MAX+1 would be wrong.
$fcSeq = $db->fetchOne("SELECT pg_get_serial_sequence(?, 'fileid')", [$prefix . 'filecache']); if ($db->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\PostgreSQLPlatform) {
for ($i = 0; $i < 50; $i++) { $fcSeq = $db->fetchOne("SELECT pg_get_serial_sequence(?, 'fileid')", [$prefix . 'filecache']);
$db->fetchOne('SELECT nextval(CAST(? AS regclass))', [$fcSeq]); for ($i = 0; $i < 50; $i++) {
$db->fetchOne('SELECT nextval(CAST(? AS regclass))', [$fcSeq]);
}
} }
$seqBefore = $seqStates(); $seqBefore = $seqStates();
$before = []; $before = [];
@@ -71,6 +73,17 @@ $dumpBytes = array_sum(array_column($manifest['tables'], 'bytes'));
printf("dump size (plain JSON lines): %.1f MiB\n", $dumpBytes / 1048576); printf("dump size (plain JSON lines): %.1f MiB\n", $dumpBytes / 1048576);
$check('checksums in dump match live tables', array_map(fn ($x) => $x['sha256'], $manifest['tables']) === array_map(fn ($x) => $x['sha256'], $before)); $check('checksums in dump match live tables', array_map(fn ($x) => $x['sha256'], $manifest['tables']) === array_map(fn ($x) => $x['sha256'], $before));
// The same dump in many tiny steps (as under web cron) must give identical table checksums.
$stepped = $dumper->startDump();
$stepStats = [];
while (!$stepped['done']) {
$stepped = $dumper->dumpStep($repo, $stepped, microtime(true) + 0.05, $stepStats);
}
$sm = $dumper->manifest($stepped);
printf("stepped dump: %d steps, consistent=%s\n", $stepped['steps'], json_encode($sm['consistent']));
$check('stepped dump: same checksums as the one-shot dump', array_map(fn ($x) => $x['sha256'], $sm['tables']) === array_map(fn ($x) => $x['sha256'], $manifest['tables']));
$check('stepped dump is marked as not fully consistent', $stepped['steps'] === 1 || $sm['consistent'] === false);
// Second dump without changes: blobs are reused. // Second dump without changes: blobs are reused.
$stats2 = ['newBlobs' => 0, 'dupBlobs' => 0, 'uploaded' => 0]; $stats2 = ['newBlobs' => 0, 'dupBlobs' => 0, 'uploaded' => 0];
$dumper->dump($repo, $stats2); $dumper->dump($repo, $stats2);
@@ -90,8 +103,10 @@ memory_reset_peak_usage();
$t = microtime(true); $t = microtime(true);
$repo2 = Repository::open(new LocalBackend($repoDir), 'spike passphrase'); $repo2 = Repository::open(new LocalBackend($repoDir), 'spike passphrase');
$m = json_decode($repo2->getObject('db/1'), true, 512, JSON_THROW_ON_ERROR); $m = json_decode($repo2->getObject('db/1'), true, 512, JSON_THROW_ON_ERROR);
$rows = $dumper->restore($repo2, $m); $report = [];
printf("restore: %d rows in %.1fs, peak %.1f MiB\n", $rows, microtime(true) - $t, memory_get_peak_usage() / 1048576); $rows = $dumper->restore($repo2, $m, $report);
printf("restore: %d rows in %.1fs, peak %.1f MiB, %d tables restored, %d unchanged skipped\n", $rows, microtime(true) - $t, memory_get_peak_usage() / 1048576, count($report['restored']), count($report['skipped']));
$check('restore only rewrote the damaged tables', count($report['restored']) === count($changed));
$mismatch = []; $mismatch = [];
foreach ($tables as $tb) { foreach ($tables as $tb) {
@@ -109,7 +124,8 @@ if ($mismatch) {
// Sequences must be restored exactly (PostgreSQL), including one that ran ahead of the data. // Sequences must be restored exactly (PostgreSQL), including one that ran ahead of the data.
$seqAfter = $seqStates(); $seqAfter = $seqStates();
$diff = array_filter(array_keys($seqBefore), fn ($k) => ($seqAfter[$k] ?? null) !== $seqBefore[$k]); $diff = array_filter(array_keys($seqBefore), fn ($k) => ($seqAfter[$k] ?? null) !== $seqBefore[$k]);
$check(sprintf('all %d sequences restored exactly (filecache seq 50 ahead of data)', count($seqBefore)), $diff === [] && count($seqBefore) > 0); $isPg = $db->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\PostgreSQLPlatform;
$check(sprintf('all %d sequences restored exactly%s', count($seqBefore), $isPg ? ' (filecache seq 50 ahead of data)' : ' (n/a on this database)'), $diff === [] && (!$isPg || count($seqBefore) > 0));
if ($diff) { if ($diff) {
foreach (array_slice($diff, 0, 5) as $k) { foreach (array_slice($diff, 0, 5) as $k) {
echo "$k before=" . json_encode($seqBefore[$k]) . ' after=' . json_encode($seqAfter[$k] ?? null) . "\n"; echo "$k before=" . json_encode($seqBefore[$k]) . ' after=' . json_encode($seqAfter[$k] ?? null) . "\n";
+30 -5
View File
@@ -11,12 +11,23 @@
</NcSettingsSection> </NcSettingsSection>
<template v-if="status"> <template v-if="status">
<KeySetup :status="status" @changed="refresh" /> <div class="ngb-tabs">
<template v-if="status.key.confirmation"> <NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
<Locations :status="status" @changed="refresh" /> <NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
<Backups v-if="status.targets.length" :status="status" @changed="refresh" /> </div>
<RestoreBrowser v-if="status.targets.length" :status="status" @changed="refresh" />
<template v-if="tab === 'settings'">
<KeySetup :status="status" @changed="refresh" />
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
</template> </template>
<template v-else-if="status.key.everConfirmed">
<template v-if="status.targets.length">
<Backups :status="status" @changed="refresh" />
<RestoreBrowser :status="status" @changed="refresh" />
</template>
<NcNoteCard v-else type="info">{{ t('ng_backup', 'Add a backup location under "Settings" first.') }}</NcNoteCard>
</template>
<NcNoteCard v-else type="warning">{{ t('ng_backup', 'Set up the backup key under "Settings" first.') }}</NcNoteCard>
</template> </template>
</div> </div>
</template> </template>
@@ -24,6 +35,7 @@
<script setup> <script setup>
import { onBeforeUnmount, onMounted, ref } from 'vue' import { onBeforeUnmount, onMounted, ref } from 'vue'
import { t } from '@nextcloud/l10n' import { t } from '@nextcloud/l10n'
import NcButton from '@nextcloud/vue/components/NcButton'
import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon' import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard' import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection' import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
@@ -35,12 +47,20 @@ import RestoreBrowser from './components/RestoreBrowser.vue'
const status = ref(null) const status = ref(null)
const error = ref('') const error = ref('')
const tab = ref('settings')
let tabInitialized = false
let timer = null let timer = null
async function refresh() { async function refresh() {
try { try {
status.value = await api.status() status.value = await api.status()
error.value = '' error.value = ''
// Default to the daily-use tab once the key is set up; stay on Settings otherwise.
// Only decided on first load, so switching tabs ourselves isn't undone by polling.
if (!tabInitialized) {
tab.value = status.value.key.everConfirmed ? 'backup' : 'settings'
tabInitialized = true
}
} catch (e) { } catch (e) {
error.value = e.message error.value = e.message
} }
@@ -59,4 +79,9 @@ onBeforeUnmount(() => clearTimeout(timer))
.ng-backup :deep(.settings-section) { .ng-backup :deep(.settings-section) {
max-width: 900px; max-width: 900px;
} }
.ngb-tabs {
display: flex;
gap: 8px;
margin: 0 0 12px;
}
</style> </style>
+8 -2
View File
@@ -17,13 +17,19 @@ async function call(method, path, data) {
export default { export default {
status: () => call('get', '/status'), status: () => call('get', '/status'),
initKey: (passphrase) => call('post', '/key', { passphrase }), initKey: (passphrase, deleteDelayDays, label) => call('post', '/key', { passphrase, deleteDelayDays, label }),
addSlot: (passphrase, label) => call('post', '/key/slots', { passphrase, label }),
replaceSlot: (slot, passphrase, label) => call('put', `/key/slots/${slot}`, { passphrase, label }),
removeSlot: (slot) => call('delete', `/key/slots/${slot}`),
trash: (targetId) => call('get', `/targets/${targetId}/trash`),
untrash: (targetId, snapshotId) => call('post', `/targets/${targetId}/trash/${snapshotId}`),
kitUrl: () => url('/key/kit'), kitUrl: () => url('/key/kit'),
confirmKit: (accepted, phrase) => call('post', '/key/confirm', { accepted, phrase }), confirmKit: (accepted, phrase, code) => call('post', '/key/confirm', { accepted, phrase, code }),
backends: () => call('get', '/backends'), backends: () => call('get', '/backends'),
addTarget: (target) => call('post', '/targets', target), addTarget: (target) => call('post', '/targets', target),
removeTarget: (id) => call('delete', `/targets/${id}`), removeTarget: (id) => call('delete', `/targets/${id}`),
testTarget: (id) => call('post', `/targets/${id}/test`), testTarget: (id) => call('post', `/targets/${id}/test`),
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }), startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
setSchedule: (time) => call('put', '/schedule', { time }), setSchedule: (time) => call('put', '/schedule', { time }),
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`), snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
+45
View File
@@ -16,12 +16,27 @@
</div> </div>
<NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard> <NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard>
<template v-for="target in status.targets" :key="'trash' + target.id">
<div v-if="trash[target.id]?.length" class="ngb-trash">
<h3>{{ t('ng_backup', 'Trash of {name}', { name: target.name }) }}</h3>
<p class="ngb-muted">{{ t('ng_backup', 'Removed restore points can be brought back until they are deleted permanently.') }}</p>
<ul>
<li v-for="item in trash[target.id]" :key="item.id">
{{ item.time ? new Date(item.time).toLocaleString() : item.id }}{{ item.label ? ' – ' + item.label : '' }}
<span class="ngb-muted">{{ t('ng_backup', 'removed by {by}, deleted after {date}', { by: item.by, date: new Date(item.deleteAfter * 1000).toLocaleDateString() }) }}</span>
<NcButton variant="tertiary" @click="untrash(target, item)">{{ t('ng_backup', 'Restore') }}</NcButton>
</li>
</ul>
</div>
</template>
<table v-if="status.runs.length" class="ngb-table"> <table v-if="status.runs.length" class="ngb-table">
<thead> <thead>
<tr> <tr>
<th>{{ t('ng_backup', 'Started') }}</th> <th>{{ t('ng_backup', 'Started') }}</th>
<th>{{ t('ng_backup', 'Location') }}</th> <th>{{ t('ng_backup', 'Location') }}</th>
<th>{{ t('ng_backup', 'Kind') }}</th> <th>{{ t('ng_backup', 'Kind') }}</th>
<th>{{ t('ng_backup', 'Restore point') }}</th>
<th>{{ t('ng_backup', 'Status') }}</th> <th>{{ t('ng_backup', 'Status') }}</th>
<th>{{ t('ng_backup', 'Files') }}</th> <th>{{ t('ng_backup', 'Files') }}</th>
<th>{{ t('ng_backup', 'Uploaded') }}</th> <th>{{ t('ng_backup', 'Uploaded') }}</th>
@@ -32,6 +47,7 @@
<td>{{ new Date(run.startedAt * 1000).toLocaleString() }}</td> <td>{{ new Date(run.startedAt * 1000).toLocaleString() }}</td>
<td>{{ targetName(run.targetId) }}</td> <td>{{ targetName(run.targetId) }}</td>
<td>{{ run.kind }}</td> <td>{{ run.kind }}</td>
<td>{{ run.snapshot ? new Date(run.snapshot.createdAt * 1000).toLocaleString() + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}</td>
<td :class="'ngb-' + run.status">{{ run.status }}{{ run.status === 'running' ? ' (' + run.phase + ')' : '' }}{{ run.error ? ': ' + run.error : '' }}</td> <td :class="'ngb-' + run.status">{{ run.status }}{{ run.status === 'running' ? ' (' + run.phase + ')' : '' }}{{ run.error ? ': ' + run.error : '' }}</td>
<td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td> <td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
<td>{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}</td> <td>{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}</td>
@@ -50,6 +66,8 @@ import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
import NcTextField from '@nextcloud/vue/components/NcTextField' import NcTextField from '@nextcloud/vue/components/NcTextField'
import api from '../api.js' import api from '../api.js'
import { formatSize } from '../format.js' import { formatSize } from '../format.js'
import { confirmPassword } from '@nextcloud/password-confirmation'
import { reactive } from 'vue'
const props = defineProps({ status: { type: Object, required: true } }) const props = defineProps({ status: { type: Object, required: true } })
const emit = defineEmits(['changed']) const emit = defineEmits(['changed'])
@@ -57,6 +75,30 @@ const schedule = ref(props.status.schedule)
const error = ref('') const error = ref('')
watch(() => props.status.schedule, (s) => { schedule.value = s }) watch(() => props.status.schedule, (s) => { schedule.value = s })
const trash = reactive({})
async function loadTrash() {
for (const target of props.status.targets) {
try {
trash[target.id] = await api.trash(target.id)
} catch (e) {
trash[target.id] = []
}
}
}
watch(() => props.status.targets.map(tg => tg.id).join(','), loadTrash, { immediate: true })
async function untrash(target, item) {
error.value = ''
try {
await confirmPassword()
await api.untrash(target.id, item.id)
await loadTrash()
emit('changed')
} catch (e) {
error.value = e?.message || ''
}
}
const isRunning = (targetId) => props.status.runs.some(r => r.targetId === targetId && r.kind === 'full' && r.status === 'running') const isRunning = (targetId) => props.status.runs.some(r => r.targetId === targetId && r.kind === 'full' && r.status === 'running')
const targetName = (id) => props.status.targets.find(tg => tg.id === id)?.name ?? '#' + id const targetName = (id) => props.status.targets.find(tg => tg.id === id)?.name ?? '#' + id
@@ -87,5 +129,8 @@ async function saveSchedule() {
.ngb-table { width: 100%; border-collapse: collapse; } .ngb-table { width: 100%; border-collapse: collapse; }
.ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); } .ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); }
.ngb-failed { color: var(--color-error-text); } .ngb-failed { color: var(--color-error-text); }
.ngb-muted { color: var(--color-text-maxcontrast); }
.ngb-trash { margin: 12px 0; }
.ngb-trash h3 { font-weight: bold; }
.ngb-done { color: var(--color-success-text); } .ngb-done { color: var(--color-success-text); }
</style> </style>
+102 -37
View File
@@ -4,38 +4,74 @@
--> -->
<template> <template>
<NcSettingsSection :name="t('ng_backup', 'Backup key and recovery kit')" <NcSettingsSection :name="t('ng_backup', 'Backup key and recovery kit')"
:description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups; the recovery kit, protected by your passphrase, is what you need to restore on a new server.')"> :description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups. To restore on a new server you need the recovery kit and one of the passphrases.')">
<!-- 1. Create the key --> <!-- 1. Create the key -->
<form v-if="!status.key.initialized" class="ngb-form" @submit.prevent="createKey"> <form v-if="!status.key.initialized" class="ngb-form" @submit.prevent="createKey">
<NcTextField v-model="label" :label="t('ng_backup', 'Who keeps this passphrase (e.g. Safe, CTO)')" />
<NcPasswordField v-model="pass1" :label="t('ng_backup', 'Passphrase (at least 12 characters)')" autocomplete="new-password" /> <NcPasswordField v-model="pass1" :label="t('ng_backup', 'Passphrase (at least 12 characters)')" autocomplete="new-password" />
<NcPasswordField v-model="pass2" :label="t('ng_backup', 'Repeat passphrase')" autocomplete="new-password" <NcPasswordField v-model="pass2" :label="t('ng_backup', 'Repeat passphrase')" autocomplete="new-password"
:error="pass2 !== '' && pass1 !== pass2" :helper-text="pass2 !== '' && pass1 !== pass2 ? t('ng_backup', 'The passphrases do not match') : ''" /> :error="pass2 !== '' && pass1 !== pass2" :helper-text="pass2 !== '' && pass1 !== pass2 ? t('ng_backup', 'The passphrases do not match') : ''" />
<NcButton type="submit" variant="primary" :disabled="busy || pass1.length < 12 || pass1 !== pass2"> <NcTextField v-model="delay" type="number" min="1" max="365"
:label="t('ng_backup', 'Days a removed backup stays in the trash before it is really deleted')"
:helper-text="t('ng_backup', 'Protects against a malicious or mistaken cleanup. Cannot be changed later.')" />
<NcButton type="submit" variant="primary" :disabled="busy || pass1.length < 12 || pass1 !== pass2 || !(delay >= 1 && delay <= 365)">
{{ t('ng_backup', 'Create backup key') }} {{ t('ng_backup', 'Create backup key') }}
</NcButton> </NcButton>
</form> </form>
<template v-else> <template v-else>
<p>{{ t('ng_backup', 'Key fingerprint: {fingerprint}', { fingerprint: status.key.fingerprint }) }}</p> <p>
{{ t('ng_backup', 'Key fingerprint: {fingerprint}', { fingerprint: status.key.fingerprint }) }} ·
{{ t('ng_backup', 'Removed backups are kept in the trash for {days} days', { days: status.key.deleteDelayDays }) }}
</p>
<!-- 2. Download the kit and confirm --> <!-- Passphrase slots -->
<h3>{{ t('ng_backup', 'Passphrases') }}</h3>
<p class="ngb-muted">
{{ t('ng_backup', 'Up to {max} passphrases can open the key, so one person leaving does not lock you out. For example: one in the safe, one with the CTO, one with the head of IT. Everyone keeps the same recovery kit and only knows their own passphrase.', { max: status.key.maxSlots }) }}
</p>
<table class="ngb-table">
<tbody>
<tr v-for="slot in status.key.slots" :key="slot.slot">
<td>{{ t('ng_backup', 'Slot {n}', { n: slot.slot }) }}</td>
<td><strong>{{ slot.label }}</strong></td>
<td class="ngb-muted">{{ slot.created ? new Date(slot.created * 1000).toLocaleDateString() : '' }}</td>
<td>
<NcButton variant="tertiary" @click="openSlotForm(slot)">{{ t('ng_backup', 'New passphrase') }}</NcButton>
<NcButton v-if="status.key.slots.length > 1" variant="tertiary" @click="removeSlot(slot)">{{ t('ng_backup', 'Remove') }}</NcButton>
</td>
</tr>
</tbody>
</table>
<NcButton v-if="!slotForm && status.key.slots.length < status.key.maxSlots" @click="openSlotForm(null)">{{ t('ng_backup', 'Add passphrase') }}</NcButton>
<form v-if="slotForm" class="ngb-form" @submit.prevent="saveSlot">
<NcTextField v-model="slotForm.label" :label="t('ng_backup', 'Who keeps this passphrase')" />
<NcPasswordField v-model="slotForm.pass1" :label="t('ng_backup', 'Passphrase (at least 12 characters)')" autocomplete="new-password" />
<NcPasswordField v-model="slotForm.pass2" :label="t('ng_backup', 'Repeat passphrase')" autocomplete="new-password" :error="slotForm.pass2 !== '' && slotForm.pass1 !== slotForm.pass2" />
<div class="ngb-row">
<NcButton type="submit" variant="primary" :disabled="busy || slotForm.pass1.length < 12 || slotForm.pass1 !== slotForm.pass2">{{ t('ng_backup', 'Save') }}</NcButton>
<NcButton @click="slotForm = null">{{ t('ng_backup', 'Cancel') }}</NcButton>
</div>
</form>
<!-- Recovery kit -->
<h3>{{ t('ng_backup', 'Recovery kit') }}</h3>
<div v-if="!status.key.confirmation" class="ngb-confirm"> <div v-if="!status.key.confirmation" class="ngb-confirm">
<NcNoteCard type="warning"> <NcNoteCard :type="status.key.everConfirmed ? 'error' : 'warning'">
{{ t('ng_backup', 'Before the first backup: download the recovery kit and store it, together with your passphrase, somewhere safe outside this server.') }} {{ status.key.everConfirmed
? t('ng_backup', 'The passphrases changed: download the new recovery kit, give a copy to every passphrase holder and confirm. Backups keep running meanwhile.')
: t('ng_backup', 'Before the first backup: download the recovery kit and store it, together with the passphrase, somewhere safe outside this server.') }}
</NcNoteCard> </NcNoteCard>
<NcButton :href="kitUrl" download @click="downloaded = true"> <NcButton @click="downloadKit">{{ t('ng_backup', 'Download recovery kit') }}</NcButton>
{{ t('ng_backup', 'Download recovery kit') }} <NcTextField v-model="code" :disabled="!downloaded" :label="t('ng_backup', 'Confirmation code from the kit file')"
</NcButton> :helper-text="t('ng_backup', 'Open the downloaded file and type the value of confirmation_code, e.g. ABCD-2345.')" />
<NcCheckboxRadioSwitch v-model="accepted" :disabled="!downloaded"> <NcCheckboxRadioSwitch v-model="accepted" :disabled="!downloaded">{{ status.statement }}</NcCheckboxRadioSwitch>
{{ status.statement }} <NcButton variant="primary" :disabled="!downloaded || !accepted || code.length < 8 || busy" @click="confirm">{{ status.confirmationPhrase }}</NcButton>
</NcCheckboxRadioSwitch>
<NcButton variant="primary" :disabled="!downloaded || !accepted || busy" @click="confirm">
{{ status.confirmationPhrase }}
</NcButton>
</div> </div>
<p v-else> <p v-else>
{{ t('ng_backup', 'Recovery kit confirmed by {user} on {date}.', { user: status.key.confirmation.uid, date: new Date(status.key.confirmation.time * 1000).toLocaleString() }) }} {{ t('ng_backup', 'Recovery kit confirmed by {user} on {date}.', { user: status.key.confirmation.uid, date: new Date(status.key.confirmation.time * 1000).toLocaleString() }) }}
<a :href="kitUrl" download>{{ t('ng_backup', 'Download the kit again') }}</a> <NcButton variant="tertiary" @click="downloadKit">{{ t('ng_backup', 'Download the kit again') }}</NcButton>
<span class="ngb-muted">{{ t('ng_backup', 'Requires your password; all administrators are notified.') }}</span>
</p> </p>
</template> </template>
<NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard> <NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard>
@@ -45,57 +81,86 @@
<script setup> <script setup>
import { ref } from 'vue' import { ref } from 'vue'
import { t } from '@nextcloud/l10n' import { t } from '@nextcloud/l10n'
import { confirmPassword } from '@nextcloud/password-confirmation'
import '@nextcloud/password-confirmation/style.css'
import NcButton from '@nextcloud/vue/components/NcButton' import NcButton from '@nextcloud/vue/components/NcButton'
import NcCheckboxRadioSwitch from '@nextcloud/vue/components/NcCheckboxRadioSwitch' import NcCheckboxRadioSwitch from '@nextcloud/vue/components/NcCheckboxRadioSwitch'
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard' import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
import NcPasswordField from '@nextcloud/vue/components/NcPasswordField' import NcPasswordField from '@nextcloud/vue/components/NcPasswordField'
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection' import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
import NcTextField from '@nextcloud/vue/components/NcTextField'
import api from '../api.js' import api from '../api.js'
const props = defineProps({ status: { type: Object, required: true } }) const props = defineProps({ status: { type: Object, required: true } })
const emit = defineEmits(['changed']) const emit = defineEmits(['changed'])
const label = ref('')
const pass1 = ref('') const pass1 = ref('')
const pass2 = ref('') const pass2 = ref('')
const delay = ref(props.status.key.deleteDelayDays)
const slotForm = ref(null)
const downloaded = ref(false) const downloaded = ref(false)
const accepted = ref(false) const accepted = ref(false)
const code = ref('')
const busy = ref(false) const busy = ref(false)
const error = ref('') const error = ref('')
const kitUrl = api.kitUrl() const kitUrl = api.kitUrl()
async function createKey() { /** Run fn after a password confirmation; report errors in the card. */
async function guarded(fn) {
busy.value = true busy.value = true
error.value = '' error.value = ''
try { try {
await api.initKey(pass1.value) await confirmPassword()
pass1.value = pass2.value = '' await fn()
emit('changed') emit('changed')
} catch (e) { } catch (e) {
error.value = e.message error.value = e?.message || ''
} finally { } finally {
busy.value = false busy.value = false
} }
} }
async function confirm() { const createKey = () => guarded(async () => {
busy.value = true await api.initKey(pass1.value, Number(delay.value), label.value || 'Slot 1')
error.value = '' pass1.value = pass2.value = ''
try { })
await api.confirmKit(accepted.value, props.status.confirmationPhrase)
emit('changed') const downloadKit = () => guarded(async () => {
} catch (e) { window.location.href = kitUrl
error.value = e.message downloaded.value = true
} finally { })
busy.value = false
} function openSlotForm(slot) {
slotForm.value = { slot: slot?.slot ?? null, label: slot?.label ?? '', pass1: '', pass2: '' }
} }
const saveSlot = () => guarded(async () => {
const f = slotForm.value
if (f.slot === null) {
await api.addSlot(f.pass1, f.label || 'Slot')
} else {
await api.replaceSlot(f.slot, f.pass1, f.label || null)
}
slotForm.value = null
downloaded.value = accepted.value = false
})
const removeSlot = (slot) => guarded(async () => {
await api.removeSlot(slot.slot)
downloaded.value = accepted.value = false
})
const confirm = () => guarded(async () => {
await api.confirmKit(accepted.value, props.status.confirmationPhrase, code.value)
code.value = ''
})
</script> </script>
<style scoped> <style scoped>
.ngb-form, .ngb-confirm { .ngb-muted { color: var(--color-text-maxcontrast); }
display: flex; .ngb-form, .ngb-confirm { display: flex; flex-direction: column; gap: 12px; max-width: 600px; margin: 8px 0; }
flex-direction: column; .ngb-row { display: flex; gap: 8px; }
gap: 12px; .ngb-table td { padding: 2px 12px 2px 0; }
max-width: 600px; h3 { margin-top: 16px; font-weight: bold; }
}
</style> </style>
+28 -5
View File
@@ -8,7 +8,8 @@
<ul v-if="status.targets.length" class="ngb-list"> <ul v-if="status.targets.length" class="ngb-list">
<li v-for="target in status.targets" :key="target.id"> <li v-for="target in status.targets" :key="target.id">
<strong>{{ target.name }}</strong> <strong>{{ target.name }}</strong>
<span class="ngb-muted">{{ target.backend }} · {{ target.path }}</span> <span class="ngb-muted">{{ target.backend }} · {{ target.path }}{{ target.appendOnly ? ' · ' + t('ng_backup', 'append-only') : '' }}</span>
<span :class="usageClass(target)">{{ usageText(target) }}</span>
<span v-if="tests[target.id]" :class="tests[target.id].ok ? 'ngb-ok' : 'ngb-err'"> <span v-if="tests[target.id]" :class="tests[target.id].ok ? 'ngb-ok' : 'ngb-err'">
{{ tests[target.id].ok ? t('ng_backup', 'OK, {n} snapshots', { n: tests[target.id].snapshots }) : tests[target.id].error }} {{ tests[target.id].ok ? t('ng_backup', 'OK, {n} snapshots', { n: tests[target.id].snapshots }) : tests[target.id].error }}
</span> </span>
@@ -28,6 +29,8 @@
<NcTextField v-else v-model="form.options[field.key]" :label="field.label + (field.optional ? '' : ' *')" /> <NcTextField v-else v-model="form.options[field.key]" :label="field.label + (field.optional ? '' : ' *')" />
</template> </template>
<NcTextField v-model="form.path" :label="t('ng_backup', 'Folder on the location for the backups')" /> <NcTextField v-model="form.path" :label="t('ng_backup', 'Folder on the location for the backups')" />
<NcTextField v-model="form.maxGb" type="number" min="0" :label="t('ng_backup', 'Maximum size in GiB')"
:helper-text="form.maxGb ? t('ng_backup', 'Backups stop at this size; administrators are warned at 80%.') : t('ng_backup', 'Empty = unlimited: everything stored is billed by your provider, and nothing stops a backup that keeps growing.')" />
<div class="ngb-row"> <div class="ngb-row">
<NcButton type="submit" variant="primary" :disabled="busy || !form.name || !backend || !auth">{{ t('ng_backup', 'Connect and add') }}</NcButton> <NcButton type="submit" variant="primary" :disabled="busy || !form.name || !backend || !auth">{{ t('ng_backup', 'Connect and add') }}</NcButton>
<NcButton @click="adding = false">{{ t('ng_backup', 'Cancel') }}</NcButton> <NcButton @click="adding = false">{{ t('ng_backup', 'Cancel') }}</NcButton>
@@ -47,6 +50,8 @@ import NcPasswordField from '@nextcloud/vue/components/NcPasswordField'
import NcSelect from '@nextcloud/vue/components/NcSelect' import NcSelect from '@nextcloud/vue/components/NcSelect'
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection' import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
import NcTextField from '@nextcloud/vue/components/NcTextField' import NcTextField from '@nextcloud/vue/components/NcTextField'
import { confirmPassword } from '@nextcloud/password-confirmation'
import '@nextcloud/password-confirmation/style.css'
import api from '../api.js' import api from '../api.js'
defineProps({ status: { type: Object, required: true } }) defineProps({ status: { type: Object, required: true } })
@@ -59,7 +64,18 @@ const backends = ref([])
const backend = ref(null) const backend = ref(null)
const auth = ref(null) const auth = ref(null)
const tests = reactive({}) const tests = reactive({})
const form = reactive({ name: '', path: 'ng_backup', options: {} }) const form = reactive({ name: '', path: 'ng_backup', options: {}, maxGb: '' })
const gib = (b) => (b / 1073741824).toFixed(1) + ' GiB'
function usageText(target) {
if (target.storedBytes === null || target.storedBytes === undefined) {
return target.maxBytes ? t('ng_backup', 'limit {limit}', { limit: gib(target.maxBytes) }) : t('ng_backup', 'no limit')
}
return target.maxBytes
? t('ng_backup', '{used} of {limit}', { used: gib(target.storedBytes), limit: gib(target.maxBytes) })
: t('ng_backup', '{used} stored, no limit', { used: gib(target.storedBytes) })
}
const usageClass = (target) => (target.maxBytes && target.storedBytes >= 0.8 * target.maxBytes) ? 'ngb-err' : 'ngb-muted'
// files_external DefinitionParameter: type 0 text, 1 boolean, 2 password; flag 1 = optional, 4 = hidden // files_external DefinitionParameter: type 0 text, 1 boolean, 2 password; flag 1 = optional, 4 = hidden
const toFields = (params) => Object.entries(params || {}) const toFields = (params) => Object.entries(params || {})
@@ -88,9 +104,10 @@ async function add() {
busy.value = true busy.value = true
error.value = '' error.value = ''
try { try {
await api.addTarget({ name: form.name, backend: backend.value.id, auth: auth.value.id, options: form.options, path: form.path }) await api.addTarget({ name: form.name, backend: backend.value.id, auth: auth.value.id, options: form.options, path: form.path,
maxGb: form.maxGb === '' ? null : Number(form.maxGb) })
adding.value = false adding.value = false
Object.assign(form, { name: '', path: 'ng_backup', options: {} }) Object.assign(form, { name: '', path: 'ng_backup', options: {}, maxGb: '' })
emit('changed') emit('changed')
} catch (e) { } catch (e) {
error.value = e.message error.value = e.message
@@ -107,7 +124,13 @@ async function remove(target) {
if (!window.confirm(t('ng_backup', 'Remove "{name}" from NG Backup? The backups on the location itself are kept.', { name: target.name }))) { if (!window.confirm(t('ng_backup', 'Remove "{name}" from NG Backup? The backups on the location itself are kept.', { name: target.name }))) {
return return
} }
await api.removeTarget(target.id) try {
await confirmPassword()
await api.removeTarget(target.id)
} catch (e) {
error.value = e?.message || ''
return
}
emit('changed') emit('changed')
} }
</script> </script>
+7 -3
View File
@@ -7,7 +7,7 @@
:description="t('ng_backup', 'Pick a restore point and a user folder or file. Restores run in the background and go through Nextcloud, so overwritten files keep their old content as a version and removed files go to the trash.')"> :description="t('ng_backup', 'Pick a restore point and a user folder or file. Restores run in the background and go through Nextcloud, so overwritten files keep their old content as a version and removed files go to the trash.')">
<div class="ngb-row"> <div class="ngb-row">
<NcSelect v-model="target" :options="status.targets" label="name" :input-label="t('ng_backup', 'Location')" :clearable="false" /> <NcSelect v-model="target" :options="status.targets" label="name" :input-label="t('ng_backup', 'Location')" :clearable="false" />
<NcSelect v-if="target" v-model="snapshot" :options="snapshots" :get-option-label="snapshotLabel" :input-label="t('ng_backup', 'Restore point')" :clearable="false" /> <NcSelect v-if="target" v-model="snapshot" :options="snapshots" label="displayLabel" :input-label="t('ng_backup', 'Restore point')" :clearable="false" />
</div> </div>
<template v-if="snapshot"> <template v-if="snapshot">
@@ -30,7 +30,7 @@
<NcButton v-if="restorable(path) && path !== selected" variant="tertiary" @click="selected = path">{{ t('ng_backup', 'Select this folder') }}</NcButton> <NcButton v-if="restorable(path) && path !== selected" variant="tertiary" @click="selected = path">{{ t('ng_backup', 'Select this folder') }}</NcButton>
<div v-if="selected" class="ngb-restore"> <div v-if="selected" class="ngb-restore">
<p>{{ t('ng_backup', 'Selected: {path}', { path: selected }) }}</p> <p>{{ t('ng_backup', 'Selected: {path}, from the restore point of {date}', { path: selected, date: snapshotLabel(snapshot) }) }}</p>
<NcCheckboxRadioSwitch v-for="m in modes" :key="m.id" v-model="mode" :value="m.id" type="radio" name="ngb-mode">{{ m.label }}</NcCheckboxRadioSwitch> <NcCheckboxRadioSwitch v-for="m in modes" :key="m.id" v-model="mode" :value="m.id" type="radio" name="ngb-mode">{{ m.label }}</NcCheckboxRadioSwitch>
<NcButton variant="primary" :disabled="busy" @click="restore">{{ t('ng_backup', 'Restore') }}</NcButton> <NcButton variant="primary" :disabled="busy" @click="restore">{{ t('ng_backup', 'Restore') }}</NcButton>
</div> </div>
@@ -49,6 +49,8 @@ import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard' import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
import NcSelect from '@nextcloud/vue/components/NcSelect' import NcSelect from '@nextcloud/vue/components/NcSelect'
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection' import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
import { confirmPassword } from '@nextcloud/password-confirmation'
import '@nextcloud/password-confirmation/style.css'
import api from '../api.js' import api from '../api.js'
import { formatSize } from '../format.js' import { formatSize } from '../format.js'
@@ -82,7 +84,8 @@ const crumbs = computed(() => {
watch(target, async (tg) => { watch(target, async (tg) => {
snapshot.value = null snapshot.value = null
snapshots.value = tg ? await api.snapshots(tg.id) : [] const list = tg ? await api.snapshots(tg.id) : []
snapshots.value = list.map(s => ({ ...s, displayLabel: snapshotLabel(s) }))
snapshot.value = snapshots.value[0] ?? null snapshot.value = snapshots.value[0] ?? null
}, { immediate: true }) }, { immediate: true })
watch(snapshot, () => { selected.value = ''; open('data') }) watch(snapshot, () => { selected.value = ''; open('data') })
@@ -107,6 +110,7 @@ async function restore() {
busy.value = true busy.value = true
error.value = message.value = '' error.value = message.value = ''
try { try {
await confirmPassword()
await api.startRestore(target.value.id, snapshot.value.id, selected.value, mode.value) await api.startRestore(target.value.id, snapshot.value.id, selected.value, mode.value)
message.value = t('ng_backup', 'Restore queued. It runs with the next cron job; progress is shown under Backups.') message.value = t('ng_backup', 'Restore queued. It runs with the next cron job; progress is shown under Backups.')
emit('changed') emit('changed')
+15
View File
@@ -33,4 +33,19 @@ class KeyRingTest extends TestCase {
$this->expectException(\InvalidArgumentException::class); $this->expectException(\InvalidArgumentException::class);
new KeyRing('short'); new KeyRing('short');
} }
public function testAnySlotOpensTheSameKey(): void {
$keys = KeyRing::generate();
$field = ['slots' => [
['slot' => 1, 'label' => 'Safe', 'key' => $keys->wrap('passphrase in the safe')],
['slot' => 2, 'label' => 'CTO', 'key' => $keys->wrap('passphrase of the cto')],
['slot' => 3, 'label' => 'Head of IT', 'key' => $keys->wrap('passphrase head of it')],
]];
foreach (['passphrase in the safe', 'passphrase of the cto', 'passphrase head of it'] as $p) {
$this->assertSame($keys->blobId('x'), KeyRing::unwrapAny($field, $p)->blobId('x'));
}
$this->assertSame($keys->blobId('x'), KeyRing::unwrapAny($keys->wrap('legacy single'), 'legacy single')->blobId('x'));
$this->expectException(CryptoException::class);
KeyRing::unwrapAny($field, 'nobody knows this');
}
} }
+90
View File
@@ -0,0 +1,90 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Repository;
use OCA\NgBackup\Backend\LocalBackend;
use OCA\NgBackup\Crypto\KeyRing;
use OCA\NgBackup\Repository\ICatalogAnchor;
use OCA\NgBackup\Repository\Repository;
use OCA\NgBackup\Repository\RepositoryException;
use OCA\NgBackup\Tests\Unit\TempDirTrait;
use PHPUnit\Framework\TestCase;
class CatalogTest extends TestCase {
use TempDirTrait;
private LocalBackend $backend;
private KeyRing $keys;
private ICatalogAnchor $anchor;
private string $root;
private string $src;
protected function setUp(): void {
$this->root = $this->tempDir();
$this->backend = new LocalBackend($this->root);
$this->keys = KeyRing::generate();
$this->anchor = new class implements ICatalogAnchor {
public array $data = [];
public function get(string $r): ?array { return $this->data[$r] ?? null; }
public function set(string $r, int $g, string $h): void { $this->data[$r] = ['gen' => $g, 'hash' => $h]; }
};
$this->src = $this->tempDir();
file_put_contents("$this->src/a", random_bytes(100000));
}
private function open(): Repository {
return Repository::openWithKey($this->backend, $this->keys, null, $this->anchor);
}
public function testNormalUseAdvancesTheChain(): void {
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
$s1 = $repo->backupDirectory($this->src);
file_put_contents("$this->src/b", 'x');
$s2 = $repo->backupDirectory($this->src, $s1['snapshot']);
$repo->forget($s1['snapshot']);
$again = $this->open();
$this->assertSame([$s2['snapshot']], $again->listSnapshots());
$this->assertArrayHasKey($s1['snapshot'], $again->trash());
$this->assertSame(4, $again->catalogGeneration());
}
public function testRollbackIsDetected(): void {
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
$repo->backupDirectory($this->src);
$repo->backupDirectory($this->src);
// The location "loses" the newest generation and presents an older, validly encrypted state.
$catalogs = $this->backend->list('catalog');
unlink($this->root . '/' . end($catalogs));
$this->expectException(RepositoryException::class);
$this->expectExceptionMessage('rolled back');
$this->open();
}
public function testHiddenSnapshotIsDetected(): void {
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
$s = $repo->backupDirectory($this->src);
unlink($this->root . '/snapshots/' . $s['snapshot']);
$this->expectException(RepositoryException::class);
$this->open()->listSnapshots();
}
public function testSwappedGenerationIsRejected(): void {
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
$repo->backupDirectory($this->src);
$repo->backupDirectory($this->src);
$c = $this->backend->list('catalog');
// Put generation 2's bytes where generation 3 should be.
copy($this->root . '/' . $c[1], $this->root . '/' . $c[2]);
$this->expectException(\RuntimeException::class);
$this->open();
}
public function testWithoutAnchorTheNewestChainIsAccepted(): void {
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
$s = $repo->backupDirectory($this->src);
// Disaster recovery on a new server: opened with the passphrase, no anchor.
$this->assertSame([$s['snapshot']], Repository::open($this->backend, 'pw')->listSnapshots());
}
}
+115
View File
@@ -0,0 +1,115 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Repository;
use OCA\NgBackup\Backend\LocalBackend;
use OCA\NgBackup\Crypto\KeyRing;
use OCA\NgBackup\Repository\Repository;
use OCA\NgBackup\Tests\Unit\TempDirTrait;
use PHPUnit\Framework\TestCase;
class PruneTest extends TestCase {
use TempDirTrait;
private function du(LocalBackend $b): int {
$root = (new \ReflectionProperty(LocalBackend::class, 'root'))->getValue($b);
$total = 0;
foreach ($b->list('packs') as $p) {
$total += filesize("$root/$p");
}
return $total;
}
public function testForgetAndPruneFreesSpaceAndKeepsTheRest(): void {
$src = $this->tempDir();
$backend = new LocalBackend($this->tempDir());
$keys = KeyRing::generate();
$repo = Repository::initWithKey($backend, $keys, $keys->wrap('pw'));
file_put_contents("$src/keep.bin", random_bytes(6 * 1048576));
file_put_contents("$src/old.bin", random_bytes(40 * 1048576));
$old = $repo->backupDirectory($src);
unlink("$src/old.bin");
file_put_contents("$src/new.bin", random_bytes(3 * 1048576));
$new = $repo->backupDirectory($src, $old['snapshot']);
$expected = self::hashTree($src);
$before = $this->du($backend);
$dry = $repo->prune(true);
$this->assertSame(0, $dry['freedBytes'], 'nothing is unused while both snapshots exist');
$preview = $repo->prune(true, 0.5, [$old['snapshot']]);
$this->assertGreaterThan(35 * 1048576, $preview['freedBytes'], 'dry run can preview forgetting a snapshot');
$repo->forget($old['snapshot'], 'tester');
$this->assertSame(0, $repo->prune(true)['freedBytes'], 'a snapshot in the trash keeps its data');
$this->assertArrayHasKey($old['snapshot'], $repo->trash());
$this->assertSame([], $repo->purgeTrash(3600), 'not purged before the delay');
$this->assertSame([$old['snapshot']], $repo->purgeTrash(0));
$plan = $repo->prune(true);
$this->assertSame($plan, $repo->prune(true, 0.5, []));
$this->assertGreaterThan(35 * 1048576, $plan['freedBytes']);
$this->assertSame($before, $this->du($backend), 'dry run changes nothing');
$result = $repo->prune();
$this->assertGreaterThan(0, $result['deleted'] + $result['repacked']);
$this->assertLessThan($before - 35 * 1048576, $this->du($backend));
$reopened = Repository::openWithKey($backend, $keys);
$target = $this->tempDir();
$reopened->restore($new['snapshot'], $target);
$this->assertSame($expected, self::hashTree($target));
$this->assertSame([$new['snapshot']], $reopened->listSnapshots());
}
public function testRepackKeepsUsedBlobsOfMixedPacks(): void {
$src = $this->tempDir();
$backend = new LocalBackend($this->tempDir());
$keys = KeyRing::generate();
$repo = Repository::initWithKey($backend, $keys, $keys->wrap('pw'));
// Small files share packs: deleting most of them leaves mixed packs to repack.
for ($i = 0; $i < 40; $i++) {
file_put_contents("$src/f$i", random_bytes(500000));
}
$a = $repo->backupDirectory($src);
for ($i = 0; $i < 40; $i++) {
if ($i % 5 !== 0) {
unlink("$src/f$i");
}
}
$b = $repo->backupDirectory($src, $a['snapshot']);
$repo->forget($a['snapshot']);
$repo->purgeTrash(0);
$r = $repo->prune(false, 0.5);
$this->assertGreaterThan(0, $r['repacked']);
$target = $this->tempDir();
Repository::openWithKey($backend, $keys)->restore($b['snapshot'], $target);
$this->assertSame(self::hashTree($src), self::hashTree($target));
}
public function testUntrashRestoresAWorkingSnapshot(): void {
$src = $this->tempDir();
$backend = new LocalBackend($this->tempDir());
$keys = KeyRing::generate();
$repo = Repository::initWithKey($backend, $keys, $keys->wrap('pw'));
file_put_contents("$src/a", random_bytes(2 * 1048576));
$repo->putObject('db/x', '{"tables":{}}');
$s = $repo->backupDirectory($src);
$expected = self::hashTree($src);
unlink("$src/a");
file_put_contents("$src/b", 'other');
$s2 = $repo->backupDirectory($src, $s['snapshot']);
$repo->forget($s['snapshot'], 'attacker');
$this->assertNotContains($s['snapshot'], $repo->listSnapshots());
$repo->prune(); // must not free the trashed snapshot's data
$repo->untrash($s['snapshot']);
$this->assertContains($s['snapshot'], $repo->listSnapshots());
$this->assertSame([], $repo->trash());
$target = $this->tempDir();
Repository::openWithKey($backend, $keys)->restore($s['snapshot'], $target);
$this->assertSame($expected, self::hashTree($target));
}
}
+38
View File
@@ -120,4 +120,42 @@ class RepositoryTest extends TestCase {
$this->expectException(CryptoException::class); $this->expectException(CryptoException::class);
$repo->getObject('db/2'); $repo->getObject('db/2');
} }
public function testSafePaths(): void {
foreach (['data/alice/files/a.txt', 'config/config.php', 'a b/c-d_e.f'] as $ok) {
$this->assertTrue(Repository::isSafePath($ok), $ok);
}
foreach (['', '/etc/passwd', '../x', 'a/../b', 'a/./b', 'a//b', 'a/', "a\0b", 'a\\b', "a\nb"] as $bad) {
$this->assertFalse(Repository::isSafePath($bad), json_encode($bad));
}
}
public function testMaliciousSnapshotPathIsRejectedOnRestore(): void {
$repo = Repository::init($this->backend, 'pw');
$stats = [];
$blob = $repo->storeData('evil', $stats);
$repo->flushPacks();
$repo->writeSnapshot('bad', ['time' => gmdate('c')], [json_encode(['p' => '../../escaped.txt', 's' => 4, 'm' => 0, 'b' => [$blob]])]);
$target = $this->tempDir() . '/inner';
mkdir($target);
try {
$repo->restore('bad', $target);
$this->fail('restore of a ../ path must fail');
} catch (RepositoryException) {
}
$this->assertFileDoesNotExist(dirname($target, 2) . '/escaped.txt');
$this->assertFileDoesNotExist(dirname($target) . '/escaped.txt');
}
public function testStorageLimitStopsTheBackup(): void {
$repo = Repository::init($this->backend, 'pw');
$repo->setMaxBytes(6 * 1048576); // less than the test tree
try {
$repo->backupDirectory($this->src);
$this->fail('backup should hit the limit');
} catch (\OCA\NgBackup\Repository\QuotaExceededException) {
}
$this->assertLessThanOrEqual(6 * 1048576, $repo->storedBytes());
$this->assertSame([], $repo->listSnapshots(), 'no snapshot from an incomplete backup');
}
} }
@@ -0,0 +1,32 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Service;
use OCA\NgBackup\Service\RetentionPolicy;
use PHPUnit\Framework\TestCase;
class RetentionPolicyTest extends TestCase {
public function testDailyWeeklyMonthly(): void {
$snaps = [];
$start = strtotime('2026-01-01 03:00:00');
for ($d = 0; $d < 120; $d++) {
$snaps["d$d"] = $start + $d * 86400; // one per day for 120 days
$snaps["d$d-b"] = $start + $d * 86400 + 3600; // and a second one an hour later
}
$keep = (new RetentionPolicy(2, 7, 4, 3))->keep($snaps);
$this->assertArrayHasKey('d119-b', $keep, 'newest is always kept');
$this->assertArrayHasKey('d119', $keep, 'last=2');
$this->assertContains('daily', $keep['d113-b']);
$this->assertArrayNotHasKey('d113', $keep, 'only the newest of a day');
$this->assertLessThanOrEqual(2 + 7 + 4 + 3, count($keep));
$months = array_unique(array_map(fn ($id) => date('Y-m', $snaps[$id]), array_keys(array_filter($keep, fn ($r) => in_array('monthly', $r, true)))));
$this->assertCount(3, $months);
}
public function testKeepsEverythingWhenFewSnapshots(): void {
$keep = (new RetentionPolicy())->keep(['a' => 100, 'b' => 200]);
$this->assertSame(['a', 'b'], array_values(array_intersect(['a', 'b'], array_keys($keep))));
}
}
+34
View File
@@ -0,0 +1,34 @@
#!/bin/bash
# Local test matrix for NG Backup: fresh Nextcloud per (version, database), run the round trip, tear down.
# Usage: tests/integration/matrix.sh "32 35" "sqlite mariadb pgsql"
VERSIONS=${1:-"32 35"}; DBS=${2:-"sqlite mariadb pgsql"}
APP=$(cd "$(dirname "$0")/../.." && pwd)
RESULTS=()
for v in $VERSIONS; do for db in $DBS; do
n=ngbm-$v-$db; net=$n-net
docker network create $net >/dev/null
env=(-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin-matrix-pw -e NEXTCLOUD_TRUSTED_DOMAINS=localhost)
case $db in
sqlite) env+=(-e SQLITE_DATABASE=nextcloud) ;;
mariadb) docker run -d --name $n-db --network $net -e MARIADB_ROOT_PASSWORD=r -e MARIADB_DATABASE=nextcloud -e MARIADB_USER=nc -e MARIADB_PASSWORD=nc mariadb:11 --transaction-isolation=READ-COMMITTED --binlog-format=ROW >/dev/null
env+=(-e MYSQL_HOST=$n-db -e MYSQL_DATABASE=nextcloud -e MYSQL_USER=nc -e MYSQL_PASSWORD=nc) ;;
pgsql) docker run -d --name $n-db --network $net -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nc -e POSTGRES_PASSWORD=nc postgres:16-alpine >/dev/null
env+=(-e POSTGRES_HOST=$n-db -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nc -e POSTGRES_PASSWORD=nc) ;;
esac
[ "$db" != sqlite ] && sleep 8
docker run -d --name $n --network $net "${env[@]}" -v "$APP:/var/www/html/custom_apps/ng_backup:ro" nextcloud:$v-apache >/dev/null
for i in $(seq 1 90); do docker exec -u www-data $n php occ status 2>/dev/null | grep -q "installed: true" && break; sleep 4; done
docker exec $n chown www-data:www-data /var/www/html/custom_apps
echo "=== NC $v / $db: $(docker exec -u www-data $n php occ -V 2>/dev/null)"
if bash ${NGB_TRACE:+-x} "$APP/tests/integration/roundtrip.sh" "docker exec -u www-data $n" /var/www/html/data /var/www/html/custom_apps; then
RESULTS+=("NC $v / $db: PASS")
else
RESULTS+=("NC $v / $db: FAIL"); docker exec $n tail -c 2000 /var/www/html/data/nextcloud.log | tr ',' '\n' | grep -E '"message"' | tail -3
fi
# -v removes only these containers' own anonymous volumes (never a global prune)
docker rm -fv $n $n-db >/dev/null 2>&1; docker network rm $net >/dev/null
done; done
printf '%s\n' "${RESULTS[@]}"
# Non-zero exit when any combination failed (CI).
printf '%s\n' "${RESULTS[@]}" | grep -q FAIL && exit 1
exit 0
+71
View File
@@ -0,0 +1,71 @@
#!/bin/bash
# End-to-end round trip of NG Backup inside a Nextcloud installation (CI and local matrix).
# Usage: roundtrip.sh "<command prefix to run occ/php as the web user in the instance>" <datadir> <apps-path> [target-args]
# e.g. roundtrip.sh "docker exec -u www-data ngb-nc35-pgsql" /var/www/html/data /var/www/html/custom_apps
# target-args defaults to a local files_external location; pass backend/auth/options to exercise
# a different backend (e.g. amazons3 or sftp), see tests/integration/targets-matrix.sh.
set -euo pipefail
X="$1"; DATA="$2"; APPS="$3"
TARGET_ARGS="${4:-local -a null::null -o datadir=/tmp/ngb-rt-target}"
occ() { $X php occ "$@"; }
fail() { echo "FAIL: $*"; exit 1; }
ok() { echo "ok $*"; }
U=ngbtest-$RANDOM
occ app:enable ng_backup >/dev/null
sleep 8 # APCu app config cache
$X sh -c "printf 'roundtrip passphrase 2026\n' > /tmp/ngb-rt-pass && mkdir -p /tmp/ngb-rt-target"
occ backup:key:init --passphrase-file=/tmp/ngb-rt-pass --label=CI --delete-delay-days=7 >/dev/null || fail "key init"
CODE=$(occ backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
occ backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
ok "key, kit code and confirmation"
occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add"
ok "location added"
# Test data: a user with files.
$X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin"
occ files:scan "$U" >/dev/null
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
occ backup:run ci -l first >/dev/null || fail "first backup"
SECOND=$(occ backup:run ci -l second)
echo "$SECOND" | tail -2 | head -1
echo "$SECOND" | grep -qE "uploaded 0\.[0-9] MiB" || fail "unchanged second backup uploaded more than 1 MiB"
ok "full backup and unchanged incremental backup"
SNAP=$(occ backup:list ci | awk -F'|' '!f && / second /{gsub(/ /,"",$3); print $3; f=1}')
[ -n "$SNAP" ] || fail "snapshot not listed"
# Damage, then restore in each mode.
$X sh -c "cd $DATA/$U/files && rm Docs/Sub/note.txt && echo broken > Docs/f1.bin && echo extra > Docs/extra.txt"
occ files:scan "$U" >/dev/null
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m new-folder >/dev/null || fail "restore new-folder"
$X sh -c "ls $DATA/$U/files | grep -q '^Restored '" || fail "no Restored folder"
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m replace >/dev/null || fail "restore replace"
$X sh -c "rm -rf '$DATA/$U/files/Restored '*" ; occ files:scan "$U" >/dev/null
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore"
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
ok "restore new-folder and replace: files identical, extra file in the trash"
# Database dump/restore round trip (all tables, checksums, sequences on PostgreSQL).
$X php -d memory_limit=512M "$APPS/ng_backup/spikes/03-db-dump.php" /tmp/ngb-rt-db | grep "ALL PASS" >/dev/null || fail "database round trip"
ok "database dump and restore"
# Retention, trash and catalog.
occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null
occ backup:prune ci | grep "forgot 1" >/dev/null || fail "prune"
occ backup:trash ci | grep " first " >/dev/null || fail "trash"
TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
occ backup:trash:restore ci "$TID" >/dev/null || fail "trash restore"
[ "$(occ backup:list ci | grep -c ' ci ')" = "2" ] || fail "snapshot not back from the trash"
ok "retention, trash and restore from trash"
# No errors caused by loading files_external while it is disabled (its tables may not exist).
if $X sh -c "grep -q 'external_mounts' $DATA/nextcloud.log 2>/dev/null"; then fail "files_external table errors in the log"; fi
ok "no files_external errors in the log"
occ user:delete "$U" >/dev/null
$X sh -c "rm -rf /tmp/ngb-rt-target /tmp/ngb-rt-pass /tmp/ngb-rt-db"
echo "ROUNDTRIP PASS"
+65
View File
@@ -0,0 +1,65 @@
#!/bin/bash
# Round trip of NG Backup against real S3 (s3mock) and SFTP backends, through files_external's own
# backend/auth classes, same as a production location would use. Backend correctness does not
# depend on the Nextcloud version or database, so this runs once (not matrixed across NC/DB),
# on a recent Nextcloud with sqlite. Each backend gets its own fresh Nextcloud container, since
# NG Backup's key can only be initialised once per instance.
# Usage: tests/integration/targets-matrix.sh [nextcloud-version]
set -euo pipefail
V=${1:-35}
APP=$(cd "$(dirname "$0")/../.." && pwd)
NET=ngb-targets-net
RESULTS=()
cleanup() { docker rm -fv ngb-targets-nc-s3 ngb-targets-nc-sftp ngb-targets-s3 ngb-targets-sftp >/dev/null 2>&1; docker network rm "$NET" >/dev/null 2>&1; }
trap cleanup EXIT
wait_tcp() { # host port
docker run --rm --network "$NET" alpine:3 sh -c "for i in \$(seq 1 30); do nc -z $1 $2 2>/dev/null && exit 0; sleep 1; done; exit 1" \
|| { echo "FAIL: $1:$2 never came up"; exit 1; }
}
start_nc() { # container-name
docker run -d --name "$1" --network "$NET" \
-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin-targets-pw -e NEXTCLOUD_TRUSTED_DOMAINS=localhost \
-e SQLITE_DATABASE=nextcloud \
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" nextcloud:$V-apache >/dev/null
for i in $(seq 1 90); do docker exec -u www-data "$1" php occ status 2>/dev/null | grep -q "installed: true" && break; sleep 4; done
docker exec "$1" chown www-data:www-data /var/www/html/custom_apps
echo "=== NC $V / sqlite ($1): $(docker exec -u www-data "$1" php occ -V 2>/dev/null)"
}
run() { # label, nc-container, target-args
echo "=== $1"
if bash ${NGB_TRACE:+-x} "$APP/tests/integration/roundtrip.sh" "docker exec -u www-data $2" /var/www/html/data /var/www/html/custom_apps "$3"; then
RESULTS+=("$1: PASS")
else
RESULTS+=("$1: FAIL")
docker exec "$2" tail -c 2000 /var/www/html/data/nextcloud.log | tr ',' '\n' | grep -E '"message"' | tail -3
fi
}
docker network create "$NET" >/dev/null
# adobe/s3mock: an S3-compatible test server (MinIO's Docker Hub image now needs a paid login;
# this one is freely pullable and pre-creates a bucket via INITIAL_BUCKETS, no auth to configure).
docker run -d --name ngb-targets-s3 --network "$NET" \
-e INITIAL_BUCKETS=ngbackups \
adobe/s3mock >/dev/null
# atmoz/sftp: "user:pass:::dir" creates /home/user/dir as the upload root.
docker run -d --name ngb-targets-sftp --network "$NET" \
atmoz/sftp ngbackup:ngbackuppw:::upload >/dev/null
wait_tcp ngb-targets-s3 9090
wait_tcp ngb-targets-sftp 22
start_nc ngb-targets-nc-s3
run "S3 (s3mock)" ngb-targets-nc-s3 "amazons3 -a amazons3::accesskey -o bucket=ngbackups -o hostname=ngb-targets-s3 -o port=9090 -o region=us-east-1 -o use_ssl=false -o use_path_style=true -o key=test -o secret=test"
start_nc ngb-targets-nc-sftp
run "SFTP" ngb-targets-nc-sftp "sftp -a password::password -o host=ngb-targets-sftp -o root=/upload -o user=ngbackup -o password=ngbackuppw"
printf '%s\n' "${RESULTS[@]}"
printf '%s\n' "${RESULTS[@]}" | grep -q FAIL && exit 1
exit 0