6cbc5d79d4
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (push) Has been cancelled
Wires the existing RepositoryExportDestination/RepositoryImportSource (see lib/UserMigration/) into user_migration's own UserMigrationService, resolved on demand through a new UserMigrationServiceLocator since the app is an optional dependency (PLAN.md 3.8) -- without it, only this feature is unavailable, the full backup still works. New: UserBackupService::backupUser() (occ backup:backup:user) streams a user_migration export straight into the repository, under the same shared location lock as a regular backup. UserRestoreService::restoreUser() (occ backup:restore:user) never imports over a live account: --mode=replace safety-exports the current state first, then deletes and recreates the account from the backup under the same uid; --mode=as-backup restores into a new "<uid>-bak" account instead, since Nextcloud cannot rename a uid. The catalog now also tracks user export manifests (Repository::recordUserExport/ listUserExports/forgetUserExport/isRecordedUserExport), rollback-protected like a snapshot: listUserExports() verifies each one is still present on the location, and restore only ever reads a manifest the verified catalog actually recorded, never an arbitrary caller-supplied path. prune()'s existing raw users/ scan for blob retention is unchanged. A new short exclusive "catalog" lock (PruneService::catalogLockKey()) serializes the catalog write itself, so two concurrent per-user backups (or one alongside a full backup) can no longer race to write the same next generation and silently drop one of them -- the broader per-target shared lock only ever protected against a concurrent cleanup, not against other shared holders. Two real bugs found by testing this against a live Nextcloud instance, not by unit tests (RepositoryExportDestination/RepositoryImportSource implement OCP interfaces that plain PHPUnit can't load, see tests/Unit/Repository/UserExportTest.php's docblock; the new tests/integration/user-roundtrip.sh covers the real round trip instead): - RepositoryImportSource::copyToFolder() double-closed the stream it handed to putContent()/newFile() (both already close it themselves), which throws a TypeError on PHP 8 and broke every restored file with content. - A replace-mode restore failed on NotPermittedException: IRootFolder::getUserFolder() caches a Folder per uid for the process lifetime, so after delete+recreate of the same uid the import still got the deleted account's (now storage-less) folder. Fixed by dispatching FilesystemTornDownEvent, which Root already listens for to clear exactly that cache -- no internal OC\ classes needed. Local review (CodeRabbit + Codex) against this diff found three more real issues, all fixed: the catalog write race above, a failed as-backup import leaving a broken stub account behind (now deleted on failure), and an export stream leak on a read/backend failure in RepositoryExportDestination::copyFolder() (now closed in a finally). Deliberately left out: wiring per-user export into the automatic backup schedule (on-demand via occ only, for now) and retention/pruning of old user exports -- both open design questions, not yet decided. The lease covering a single export/import still has a fixed TTL (3600s) with no mid-call refresh, since UserMigrationService exposes no progress; a resumable, step-based per-user backup (like the main BackupService run) would remove that cap but is a bigger change than this pass.
177 lines
6.8 KiB
PHP
177 lines
6.8 KiB
PHP
<?php
|
|
|
|
declare(strict_types=1);
|
|
|
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
|
|
|
namespace OCA\NgBackup\Service;
|
|
|
|
use OCA\NgBackup\AppInfo\Application;
|
|
use OCA\NgBackup\Db\RunMapper;
|
|
use OCA\NgBackup\Db\SnapshotMapper;
|
|
use OCA\NgBackup\Db\Target;
|
|
use OCP\IAppConfig;
|
|
use OCP\Lock\ILockingProvider;
|
|
use OCP\Lock\LockedException;
|
|
|
|
/**
|
|
* Applies the retention policy to a location: forgets snapshots the policy does not keep and
|
|
* prunes data no remaining snapshot uses. Never runs on append-only locations, never while a
|
|
* backup to the same location is running (the location lock is shared with BackupService).
|
|
*/
|
|
final class PruneService {
|
|
public function __construct(
|
|
private TargetService $targets,
|
|
private RunMapper $runs,
|
|
private SnapshotMapper $snapshots,
|
|
private IAppConfig $appConfig,
|
|
private ILockingProvider $locking,
|
|
private AlertService $alerts,
|
|
private KeyService $keys,
|
|
private LeaseService $leases,
|
|
) {
|
|
}
|
|
|
|
public function untrash(Target $target, string $snapshotId): void {
|
|
$lease = $this->lockExclusive($target, 300);
|
|
try {
|
|
$this->doUntrash($target, $snapshotId);
|
|
} finally {
|
|
$this->leases->release($lease);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Run $fn under the location's exclusive lock (to create a run atomically with respect to a
|
|
* cleanup, which takes the same lock and then refuses while a run exists).
|
|
*/
|
|
public function underStartLock(Target $target, callable $fn): mixed {
|
|
try {
|
|
return $this->leases->with(self::lockKey($target), LeaseService::EXCLUSIVE, 60, fn () => $fn());
|
|
} catch (LockedException) {
|
|
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is in progress; try again later');
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Exclusive lock on the location, then make sure no run of any kind (backup, restore, export)
|
|
* is in progress: their data may not be referenced by a snapshot yet, or is being read.
|
|
*/
|
|
private function lockExclusive(Target $target, int $ttl): string {
|
|
try {
|
|
$lease = $this->leases->acquire(self::lockKey($target), LeaseService::EXCLUSIVE, $ttl);
|
|
} catch (LockedException) {
|
|
throw new \RuntimeException('A backup, restore or cleanup of ' . $target->getName() . ' is in progress; try again later');
|
|
}
|
|
if ($this->runs->findRunning($target->getId()) !== []) {
|
|
$this->leases->release($lease);
|
|
throw new \RuntimeException('A backup or restore of ' . $target->getName() . ' is running; try again later');
|
|
}
|
|
return $lease;
|
|
}
|
|
|
|
private function doUntrash(Target $target, string $snapshotId): void {
|
|
$repo = $this->targets->repository($target);
|
|
if (!array_key_exists($snapshotId, $repo->trash())) {
|
|
throw new \InvalidArgumentException('That snapshot is not in the trash (or was already deleted)');
|
|
}
|
|
$repo->untrash($snapshotId);
|
|
// Put it back in the local snapshot list as well.
|
|
$meta = $repo->snapshotMeta($snapshotId);
|
|
$s = new \OCA\NgBackup\Db\Snapshot();
|
|
$s->setTargetId($target->getId());
|
|
$s->setSnapshotId($snapshotId);
|
|
$s->setKind((string)($meta['kind'] ?? 'full'));
|
|
$s->setLabel(($meta['label'] ?? '') !== '' ? $meta['label'] : null);
|
|
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
|
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
|
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
|
try {
|
|
$this->snapshots->insert($s);
|
|
} catch (\Throwable) {
|
|
}
|
|
$this->alerts->securityEvent('snapshot_untrashed', ['target' => $target->getName(), 'snapshot' => $snapshotId]);
|
|
}
|
|
|
|
public static function lockKey(Target $target): string {
|
|
return 'ng_backup/target/' . $target->getId();
|
|
}
|
|
|
|
/**
|
|
* A separate, short-lived exclusive lock serializing writes to the catalog itself. The
|
|
* per-target lock above only keeps readers/writers out while a cleanup runs; it does not
|
|
* stop two SHARED holders (e.g. two concurrent per-user backups, or one alongside a full
|
|
* backup) from both reading the same catalog generation and racing to write the next one,
|
|
* silently dropping whichever's change loses. The full-backup path avoids this today only
|
|
* because a Run row makes concurrent runs for the same target mutually exclusive; per-user
|
|
* backup/restore has no such row, so it takes this lock around the catalog write itself.
|
|
*/
|
|
public static function catalogLockKey(Target $target): string {
|
|
return self::lockKey($target) . '/catalog';
|
|
}
|
|
|
|
public function policy(): RetentionPolicy {
|
|
return RetentionPolicy::fromArray(json_decode($this->appConfig->getValueString(Application::APP_ID, 'retention', '{}'), true) ?: []);
|
|
}
|
|
|
|
public function setPolicy(RetentionPolicy $policy): void {
|
|
if ($policy->last < 1) {
|
|
throw new \InvalidArgumentException('Keep at least the last snapshot');
|
|
}
|
|
$old = $this->policy()->toArray();
|
|
$this->appConfig->setValueString(Application::APP_ID, 'retention', json_encode($policy->toArray(), JSON_THROW_ON_ERROR));
|
|
if ($old !== $policy->toArray()) {
|
|
$this->alerts->securityEvent('retention_changed', ['policy' => json_encode($policy->toArray())]);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Snapshots the policy drops go to the trash on the location; trash entries older than the
|
|
* deletion delay are purged, and only then is their data freed.
|
|
*
|
|
* @return array{kept: array<string, list<string>>, forgotten: list<string>, purged: list<string>, prune: array}
|
|
*/
|
|
public function apply(Target $target, bool $dryRun = false, bool $scheduled = false): array {
|
|
if ($target->getAppendOnly()) {
|
|
throw new \RuntimeException('Location ' . $target->getName() . ' is append-only: NG Backup does not delete anything there');
|
|
}
|
|
$lease = $this->lockExclusive($target, 1800);
|
|
try {
|
|
$repo = $this->targets->repository($target);
|
|
$times = [];
|
|
foreach ($repo->listSnapshots() as $id) {
|
|
$meta = $repo->snapshotMeta($id);
|
|
$times[$id] = (int)strtotime((string)($meta['time'] ?? '@0'));
|
|
}
|
|
$keep = $this->policy()->keep($times);
|
|
$forget = array_values(array_diff(array_keys($times), array_keys($keep)));
|
|
$actor = $scheduled ? 'retention policy' : $this->alerts->actor();
|
|
$purged = [];
|
|
if (!$dryRun) {
|
|
foreach ($forget as $id) {
|
|
$repo->forget($id, $actor);
|
|
try {
|
|
$this->snapshots->delete($this->snapshots->findOne($target->getId(), $id));
|
|
} catch (\Throwable) {
|
|
}
|
|
}
|
|
}
|
|
// Also removes leftovers of interrupted runs. A dry run counts the snapshots it would forget as gone.
|
|
$delay = $this->keys->deleteDelayDays();
|
|
if (!$dryRun && $forget !== []) {
|
|
// Scheduled cleanup follows the policy: audit log only, no notification every week.
|
|
$this->alerts->securityEvent('snapshots_forgotten', ['target' => $target->getName(), 'count' => (string)count($forget),
|
|
'until' => date('Y-m-d', time() + $delay * 86400)], !$scheduled);
|
|
}
|
|
if (!$dryRun) {
|
|
$purged = $repo->purgeTrash($delay * 86400);
|
|
}
|
|
$prune = $repo->prune($dryRun);
|
|
return ['kept' => $keep, 'forgotten' => $forget, 'purged' => $purged, 'prune' => $prune];
|
|
} finally {
|
|
$this->leases->release($lease);
|
|
}
|
|
}
|
|
}
|