A key made by mistake on a server that is being restored left no way back: key:import and the
passphrase route refused because a key existed, and nothing removed it. As long as the key never
protected anything (no location created or adopted with it, no location or run in the database),
occ backup:key:import --replace-unused-key and occ backup:target:add --replace-unused-key replace
it, after the new key was opened. A key that was used is never replaced, also after its location
was removed. The refusal names the option, and all administrators are notified of a replacement.
On a server without a backup key, backup:target:add finds the backups at
the location and takes their key from its config with one of the old
server's passphrases. Backups then wait until a new recovery kit is
confirmed. The deletion delay and the repository id are now stored on the
location, sealed with the key, so the delay survives a restore without a
kit and --repair-config can write a lost config again from the kit.
The key page warns against creating a new key when restoring a server.
New test: tests/integration/recovery-scenarios.sh, and a no-kit mode in
full-restore-roundtrip.sh.
Full restore, raw directory restore, per-user export and the safety export
held a fixed-TTL lease on the location that nothing refreshed. A restore or
export running longer than that TTL let a cleanup take over and delete data
still in use. They now refresh the lease while they work (per blob, per row
batch, per written chunk), throttled to once a minute.
Also:
- backup:prune still applies snapshot retention when user-export retention
fails for a location.
- The recovery kit carries the deletion delay, so a disaster recovery keeps
the original protection instead of falling back to 7 days. Older kits
still import with the default.
- CI checkouts no longer persist the token in the git config.
- recoveryKit() has its real return type; the stale baseline entry is gone.
New commands, per PLAN.md 3.7:
- occ backup:key:import: adopt the master key from a recovery kit on a fresh installation
(instead of occ backup:key:init generating a new one), so occ backup:target:add opens the
existing repository on the location rather than creating an empty one.
- occ backup:restore:full: replace the database (consistent, one transaction), restore the data
directory in place, and merge config.php -- refuses up front and lists what's missing when the
Nextcloud major version or any app enabled at backup time doesn't match here (their tables must
exist before the database restore can use them), rather than guessing or partially applying.
Repository::restore() gained an optional $stripPrefix (write "data/<uid>/..." straight into the
real data directory, not a "data/" subdirectory of it) -- restoring "data/" and "config/" this way
needs no temporary copy of what can be the largest part of an instance.
occ backup:target:add now backfills the local snapshot cache (ngb_snapshots, what occ backup:list
actually reads) when it adopts an existing repository: without this, a freshly connected location
showed no restore points at all, even though the repository itself had them. One unreadable
snapshot is skipped rather than aborting the backfill or the target add itself.
Three self-referential problems found and fixed by testing this against two real, separate
Nextcloud instances sharing only the backup location (tests/integration/full-restore-roundtrip.sh
covers this; the per-process nature of these bugs meant they were invisible from inside a single
occ invocation, including the one used to write the fixes, which is exactly why a second, fresh
process re-running occ backup:status etc. was necessary to catch them):
- The full database dump includes ng_backup's own tables and appconfig rows; restoring them
verbatim would overwrite the key/target that is actively performing the restore, on THIS
installation, with the backed-up instance's. Both are now preserved: ng_backup's own tables
(and, on PostgreSQL, their sequences -- excluding a table does not exclude its sequence) are
excluded from what gets restored, and its appconfig rows are captured before the database
restore and written back after, in their own transaction.
- config.php's 'secret' is merged in from the backup (needed so already-encrypted content in the
restored database, e.g. share tokens, stays decryptable), but Target::options, the master key,
the wrapped key and the recovery kit code are all encrypted with that same instance secret
(OCP's ICrypto, or Nextcloud AppConfig's own "sensitive" flag underneath), not with ng_backup's
own key. Changing it without re-encrypting them first left all four permanently undecryptable
the moment any later occ command ran. Fixed with two helpers
(TargetService::reencryptOptionsAround(), KeyService::reencryptForSecretRotation()) that decrypt
before and re-encrypt after whatever callback actually changes the secret; config.php is applied
dead last in restoreFull(), after every other repository access. Not crash-safe (a process kill
mid-rotation would leave a key missing, not just stale) -- disaster recovery is run once by
hand, right after occ maintenance:install, a window too narrow yet to justify a two-phase write.
- A subtler variant of the same bug: Nextcloud's own AppConfig additionally encrypts a "sensitive"
value itself, tied to the same secret, underneath our own encryption. Overwriting an existing
sensitive key makes it read the old value first to compare, which then also fails against the
new secret. Deleting the key before setting the new value avoids that read entirely.
Roadmap (PLAN.md 9): phase 3 (in-place full restore, user self-service) moved out to v1.3.0; phase
2 continues toward v1.2.0 with FTP/SMB, a verification/checksum-audit command, user-export
retention, and deciding whether per-user export joins the automatic schedule.
RepositoryImportSource::copyToFolder() called putContent() on the generic
return type of Node::get(), which doesn't declare that method -- only
File does. Existing-file overwrite during a per-user restore would have
fatally errored at runtime; added an instanceof check and a resource leak
fix (finally-close) while there. Also guarded getFileAsStream() against
fopen() returning false.
Everything else Psalm (strict level) currently flags is mechanical
(missing `final` on leaf classes -- applied everywhere safe) or pre-existing
array-shape/type gaps, captured in psalm-baseline.xml so CI fails only on
new issues from here on, not the whole backlog at once.
- one locking protocol per location: backup steps and restores hold a
shared lock, prune and untrash an exclusive one and refuse while any
run (backup, restore, export) exists; starting a backup checks and
creates the run under the exclusive lock, closing the race with prune
- snapshot entries are validated when read (relative path without '.',
'..', NUL, backslash or control characters); raw restore also checks
that the resolved directory stays below the target; the restore API
and service accept canonical paths only
- recovery kit confirmation requires a recent password confirmation and
the confirmation code printed in the current kit file (new code per
kit version), proving the kit was actually downloaded
- prune decides on the authenticated index files on the location and
refreshes the local index cache from them
- restore job marks the run failed on any error; restore runs without
progress for 6 hours are failed so they cannot block cleanup forever
- tests for safe paths and a malicious snapshot path
- removed snapshots go to a trash on the location and their data is only
freed after the deletion delay, which is chosen when the key is created
(default 7 days, 1-365) and cannot be changed afterwards; any admin can
bring them back (occ backup:trash, backup:trash:restore, web UI)
- up to three passphrase slots open the same master key (like LUKS), so
one person leaving or dying does not lock the company out; slot changes
are written to every location, announced to all admins and require a
new recovery kit confirmation, while backups keep running
- occ backup:key:slots, backup:key:slot:add|replace|remove; key init takes
--label and --delete-delay-days
- web UI: deletion delay and first slot at key creation, slot management,
trash per location with restore; all behind password confirmation
- README: key, slots (safe / CTO / head of IT), and protection against
ransomware: object lock in compliance mode (e.g. 90 days), credentials
without delete rights, provider snapshots, off-site copy
- migration: ngb_targets, ngb_runs, ngb_snapshots
- KeyService: master key stored encrypted with the instance secret
(unattended backups), passphrase-wrapped copy in every repository and
in the recovery kit, recorded admin confirmation per key fingerprint
- TargetService: locations built from files_external backends without a
mount; files_external is loaded through IAppManager::loadApp() and its
enabled state is left alone; credentials encrypted with ICrypto;
existing repositories are adopted only if made with this key, and a
repository cannot be registered twice
- Repository: initWithKey/openWithKey with an encrypted key check
- occ: backup:key:init|kit|confirm, backup:target:backends|add|list|remove
dillard