Files
nextcloud-ng_backup/lib/Repository/Catalog.php
T
dillard 6cbc5d79d4
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (push) Has been cancelled
Phase 2: per-user backup and restore via user_migration
Wires the existing RepositoryExportDestination/RepositoryImportSource (see
lib/UserMigration/) into user_migration's own UserMigrationService, resolved on demand
through a new UserMigrationServiceLocator since the app is an optional dependency
(PLAN.md 3.8) -- without it, only this feature is unavailable, the full backup still
works.

New: UserBackupService::backupUser() (occ backup:backup:user) streams a user_migration
export straight into the repository, under the same shared location lock as a regular
backup. UserRestoreService::restoreUser() (occ backup:restore:user) never imports over a
live account: --mode=replace safety-exports the current state first, then deletes and
recreates the account from the backup under the same uid; --mode=as-backup restores into
a new "<uid>-bak" account instead, since Nextcloud cannot rename a uid.

The catalog now also tracks user export manifests (Repository::recordUserExport/
listUserExports/forgetUserExport/isRecordedUserExport), rollback-protected like a
snapshot: listUserExports() verifies each one is still present on the location, and
restore only ever reads a manifest the verified catalog actually recorded, never an
arbitrary caller-supplied path. prune()'s existing raw users/ scan for blob retention is
unchanged. A new short exclusive "catalog" lock (PruneService::catalogLockKey())
serializes the catalog write itself, so two concurrent per-user backups (or one
alongside a full backup) can no longer race to write the same next generation and
silently drop one of them -- the broader per-target shared lock only ever protected
against a concurrent cleanup, not against other shared holders.

Two real bugs found by testing this against a live Nextcloud instance, not by unit
tests (RepositoryExportDestination/RepositoryImportSource implement OCP interfaces that
plain PHPUnit can't load, see tests/Unit/Repository/UserExportTest.php's docblock; the
new tests/integration/user-roundtrip.sh covers the real round trip instead):
- RepositoryImportSource::copyToFolder() double-closed the stream it handed to
  putContent()/newFile() (both already close it themselves), which throws a TypeError on
  PHP 8 and broke every restored file with content.
- A replace-mode restore failed on NotPermittedException: IRootFolder::getUserFolder()
  caches a Folder per uid for the process lifetime, so after delete+recreate of the same
  uid the import still got the deleted account's (now storage-less) folder. Fixed by
  dispatching FilesystemTornDownEvent, which Root already listens for to clear exactly
  that cache -- no internal OC\ classes needed.

Local review (CodeRabbit + Codex) against this diff found three more real issues, all
fixed: the catalog write race above, a failed as-backup import leaving a broken stub
account behind (now deleted on failure), and an export stream leak on a read/backend
failure in RepositoryExportDestination::copyFolder() (now closed in a finally).

Deliberately left out: wiring per-user export into the automatic backup schedule
(on-demand via occ only, for now) and retention/pruning of old user exports -- both open
design questions, not yet decided. The lease covering a single export/import still has a
fixed TTL (3600s) with no mid-call refresh, since UserMigrationService exposes no
progress; a resumable, step-based per-user backup (like the main BackupService run)
would remove that cap but is a bigger change than this pass.
2026-10-05 17:36:46 +02:00

137 lines
5.5 KiB
PHP

<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Repository;
use OCA\NgBackup\Backend\IBackend;
use OCA\NgBackup\Crypto\StreamCipher;
/**
* Chained catalog of a repository's state, against rollback and hidden snapshots.
*
* Every change to the set of snapshots or trash entries writes catalog/<generation>: the
* encrypted list of snapshot and trash ids plus the hash of the previous generation. The server
* keeps the newest generation it has seen (ICatalogAnchor). On open, the newest catalog on the
* location must be at least the anchored generation and lead back to it through the chain;
* snapshots listed in the catalog must exist. Otherwise the location was rolled back or
* tampered with, and the repository is refused.
*/
final class Catalog {
private const KEEP = 200; // generations kept on the location (enough to verify long gaps)
public int $gen = 0;
/** @var array<string, true> */
public array $snapshots = [];
/** @var array<string, true> */
public array $trash = [];
/** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */
public array $userExports = [];
public string $hash = '';
public function __construct(
private IBackend $backend,
private StreamCipher $cipher,
private string $repositoryId,
private ?ICatalogAnchor $anchor,
) {
}
/** Load and verify the newest catalog; create the first one for repositories that have none. */
public function load(callable $listSnapshots, callable $listTrash, callable $listUserExports): void {
$gens = $this->generations();
if ($gens === []) {
// First use (new or pre-catalog repository): adopt the current state as generation 1.
$anchored = $this->anchor?->get($this->repositoryId);
if ($anchored !== null) {
throw new RepositoryException('The catalog of this repository has disappeared from the location (possible tampering)');
}
$this->snapshots = array_fill_keys($listSnapshots(), true);
$this->trash = array_fill_keys($listTrash(), true);
$this->userExports = array_fill_keys($listUserExports(), true);
$this->write();
return;
}
$head = max($gens);
[$data, $hash] = $this->read($head);
$anchored = $this->anchor?->get($this->repositoryId);
if ($anchored !== null) {
if ($head < $anchored['gen']) {
throw new RepositoryException(sprintf('The location shows catalog generation %d, but this server has seen %d: the backups were rolled back', $head, $anchored['gen']));
}
// Walk back to the anchored generation, checking every link.
[$cur, $curHash] = [$data, $hash];
for ($g = $head; $g > $anchored['gen']; $g--) {
[$prev, $prevHash] = $this->read($g - 1);
if (!hash_equals($prevHash, (string)$cur['prev'])) {
throw new RepositoryException("Catalog chain broken at generation $g (possible tampering)");
}
[$cur, $curHash] = [$prev, $prevHash];
}
if (!hash_equals($anchored['hash'], $curHash)) {
throw new RepositoryException('The catalog does not match what this server saw before (possible tampering)');
}
}
$this->gen = $head;
$this->hash = $hash;
$this->snapshots = array_fill_keys($data['snapshots'], true);
$this->trash = array_fill_keys($data['trash'], true);
// Catalogs written before user exports existed have no such key: default to empty rather
// than adopting the location's current users/ listing (that would skip verification).
$this->userExports = array_fill_keys($data['userExports'] ?? [], true);
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
}
/** Write the next generation after the in-memory state was changed. */
public function write(): void {
$next = $this->gen + 1;
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash),
'userExports' => array_keys($this->userExports)], JSON_THROW_ON_ERROR);
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
$stream = fopen('php://memory', 'w+b');
fwrite($stream, $encrypted);
rewind($stream);
$this->backend->put($this->path($next), $stream);
$this->gen = $next;
$this->hash = hash('sha256', $encrypted);
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
$old = $next - self::KEEP;
if ($old > 0) {
$this->backend->delete($this->path($old));
}
}
/** @return array{0: array, 1: string} decoded catalog and the hash of its encrypted bytes */
private function read(int $gen): array {
try {
$fh = $this->backend->get($this->path($gen));
} catch (\Throwable) {
throw new RepositoryException("Catalog generation $gen is missing (possible tampering)");
}
$encrypted = (string)stream_get_contents($fh);
fclose($fh);
$data = json_decode($this->cipher->decryptString($encrypted, $this->ad($gen)), true, 512, JSON_THROW_ON_ERROR);
if (($data['repository'] ?? '') !== $this->repositoryId || ($data['gen'] ?? 0) !== $gen) {
throw new RepositoryException("Catalog generation $gen belongs to another repository or position");
}
return [$data, hash('sha256', $encrypted)];
}
/** @return list<int> */
private function generations(): array {
return array_map(fn ($p) => (int)basename($p), $this->backend->list('catalog'));
}
private function path(int $gen): string {
return sprintf('catalog/%010d', $gen);
}
private function ad(int $gen): string {
return 'catalog:' . $this->repositoryId . ':' . $gen;
}
}