f18b4f0862
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
New commands, per PLAN.md 3.7: - occ backup:key:import: adopt the master key from a recovery kit on a fresh installation (instead of occ backup:key:init generating a new one), so occ backup:target:add opens the existing repository on the location rather than creating an empty one. - occ backup:restore:full: replace the database (consistent, one transaction), restore the data directory in place, and merge config.php -- refuses up front and lists what's missing when the Nextcloud major version or any app enabled at backup time doesn't match here (their tables must exist before the database restore can use them), rather than guessing or partially applying. Repository::restore() gained an optional $stripPrefix (write "data/<uid>/..." straight into the real data directory, not a "data/" subdirectory of it) -- restoring "data/" and "config/" this way needs no temporary copy of what can be the largest part of an instance. occ backup:target:add now backfills the local snapshot cache (ngb_snapshots, what occ backup:list actually reads) when it adopts an existing repository: without this, a freshly connected location showed no restore points at all, even though the repository itself had them. One unreadable snapshot is skipped rather than aborting the backfill or the target add itself. Three self-referential problems found and fixed by testing this against two real, separate Nextcloud instances sharing only the backup location (tests/integration/full-restore-roundtrip.sh covers this; the per-process nature of these bugs meant they were invisible from inside a single occ invocation, including the one used to write the fixes, which is exactly why a second, fresh process re-running occ backup:status etc. was necessary to catch them): - The full database dump includes ng_backup's own tables and appconfig rows; restoring them verbatim would overwrite the key/target that is actively performing the restore, on THIS installation, with the backed-up instance's. Both are now preserved: ng_backup's own tables (and, on PostgreSQL, their sequences -- excluding a table does not exclude its sequence) are excluded from what gets restored, and its appconfig rows are captured before the database restore and written back after, in their own transaction. - config.php's 'secret' is merged in from the backup (needed so already-encrypted content in the restored database, e.g. share tokens, stays decryptable), but Target::options, the master key, the wrapped key and the recovery kit code are all encrypted with that same instance secret (OCP's ICrypto, or Nextcloud AppConfig's own "sensitive" flag underneath), not with ng_backup's own key. Changing it without re-encrypting them first left all four permanently undecryptable the moment any later occ command ran. Fixed with two helpers (TargetService::reencryptOptionsAround(), KeyService::reencryptForSecretRotation()) that decrypt before and re-encrypt after whatever callback actually changes the secret; config.php is applied dead last in restoreFull(), after every other repository access. Not crash-safe (a process kill mid-rotation would leave a key missing, not just stale) -- disaster recovery is run once by hand, right after occ maintenance:install, a window too narrow yet to justify a two-phase write. - A subtler variant of the same bug: Nextcloud's own AppConfig additionally encrypts a "sensitive" value itself, tied to the same secret, underneath our own encryption. Overwriting an existing sensitive key makes it read the old value first to compare, which then also fails against the new secret. Deleting the key before setting the new value avoids that read entirely. Roadmap (PLAN.md 9): phase 3 (in-place full restore, user self-service) moved out to v1.3.0; phase 2 continues toward v1.2.0 with FTP/SMB, a verification/checksum-audit command, user-export retention, and deciding whether per-user export joins the automatic schedule.
258 lines
9.1 KiB
PHP
258 lines
9.1 KiB
PHP
<?php
|
|
|
|
declare(strict_types=1);
|
|
|
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
|
|
|
namespace OCA\NgBackup\Service;
|
|
|
|
use OCA\NgBackup\Db\Target;
|
|
use OCA\NgBackup\Repository\Repository;
|
|
use OCP\Files\File;
|
|
use OCP\Files\Folder;
|
|
use OCP\Files\IRootFolder;
|
|
use OCP\Files\NotFoundException;
|
|
|
|
/**
|
|
* Restoring from a snapshot.
|
|
*
|
|
* User files (data/<uid>/files/...) are written through the Nextcloud Files API, so the file
|
|
* cache, versions and trash stay consistent: an overwritten file becomes a version and a
|
|
* removed file goes to the trash, which makes a restore itself reversible. Every file is
|
|
* written to a temporary name first and only renamed into place after all its blobs were
|
|
* decrypted and verified.
|
|
*
|
|
* Modes for user files:
|
|
* - new-folder (default): into "Restored <date>/<same path>", nothing existing is touched
|
|
* - merge: into the original place, existing files are overwritten (old content kept as a version)
|
|
* - replace: like merge, and files that are not in the snapshot are moved to the trash
|
|
*/
|
|
final class RestoreService {
|
|
public const MODE_NEW_FOLDER = 'new-folder';
|
|
public const MODE_MERGE = 'merge';
|
|
public const MODE_REPLACE = 'replace';
|
|
|
|
public function __construct(
|
|
private TargetService $targets,
|
|
private IRootFolder $root,
|
|
private LeaseService $leases,
|
|
) {
|
|
}
|
|
|
|
/** Shared lock on the location while restoring: a cleanup (exclusive) must not delete what is being read. */
|
|
private function withSharedLock(Target $target, callable $fn, int $ttl = 600): mixed {
|
|
try {
|
|
return $this->leases->with(PruneService::lockKey($target), LeaseService::SHARED, $ttl, $fn);
|
|
} catch (\OCP\Lock\LockedException) {
|
|
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is running; try again later');
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Direct children (files and folders) of a path in a snapshot.
|
|
*
|
|
* @return list<array{name:string, type:string, size:int, mtime:int|null, files:int}>
|
|
*/
|
|
public function browse(Target $target, string $snapshotId, string $path = ''): array {
|
|
$prefix = trim($path, '/');
|
|
$prefix = $prefix === '' ? '' : $prefix . '/';
|
|
$children = [];
|
|
foreach ($this->targets->repository($target)->entries($snapshotId) as $e) {
|
|
if ($prefix !== '' && !str_starts_with($e['p'], $prefix)) {
|
|
continue;
|
|
}
|
|
$rest = substr($e['p'], strlen($prefix));
|
|
[$name, $more] = array_pad(explode('/', $rest, 2), 2, null);
|
|
$c = $children[$name] ?? ['name' => $name, 'type' => $more === null ? 'file' : 'dir', 'size' => 0, 'mtime' => null, 'files' => 0];
|
|
$c['size'] += $e['s'];
|
|
$c['files']++;
|
|
if ($more === null) {
|
|
$c['mtime'] = $e['m'];
|
|
}
|
|
$children[$name] = $c;
|
|
}
|
|
ksort($children, SORT_STRING);
|
|
return array_values($children);
|
|
}
|
|
|
|
/**
|
|
* Restore user files from "data/<uid>/files[/<path>]" into that user's Nextcloud files.
|
|
*
|
|
* @return array{restored:int, trashed:int, target:string}
|
|
*/
|
|
public function restoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode = self::MODE_NEW_FOLDER, ?callable $progress = null): array {
|
|
return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $logicalPath, $mode, $progress) {
|
|
$last = time();
|
|
// Keep the lease alive while files are being written.
|
|
$beat = function (int $done, int $total, string $name) use ($refresh, $progress, &$last): void {
|
|
if (time() - $last >= 60) {
|
|
$refresh();
|
|
$last = time();
|
|
}
|
|
if ($progress !== null) {
|
|
$progress($done, $total, $name);
|
|
}
|
|
};
|
|
return $this->doRestoreUserFiles($target, $snapshotId, $logicalPath, $mode, $beat);
|
|
});
|
|
}
|
|
|
|
private function doRestoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode, ?callable $progress): array {
|
|
if (!Repository::isSafePath($logicalPath) || !preg_match('#^data/([^/]+)/files(?:/(.*))?$#', $logicalPath, $m)) {
|
|
throw new \InvalidArgumentException('Expected a path like data/<user>/files/<folder or file>');
|
|
}
|
|
[$uid, $rel] = [$m[1], trim($m[2] ?? '', '/')];
|
|
$userFolder = $this->root->getUserFolder($uid);
|
|
$repo = $this->targets->repository($target);
|
|
$source = 'data/' . $uid . '/files' . ($rel === '' ? '' : '/' . $rel);
|
|
|
|
$entries = [];
|
|
$single = false;
|
|
foreach ($repo->entries($snapshotId) as $e) {
|
|
if ($e['p'] === $source) {
|
|
$single = true;
|
|
$entries[''] = $e;
|
|
break;
|
|
}
|
|
if (str_starts_with($e['p'], $source . '/')) {
|
|
$entries[substr($e['p'], strlen($source) + 1)] = $e;
|
|
}
|
|
}
|
|
if ($entries === []) {
|
|
throw new NotFoundException("Nothing at $source in snapshot $snapshotId");
|
|
}
|
|
|
|
// Where to write. new-folder mirrors the path under "Restored <date>" in the user's root.
|
|
$parentRel = $single ? (dirname($rel) === '.' ? '' : dirname($rel)) : $rel;
|
|
if ($mode === self::MODE_NEW_FOLDER) {
|
|
$parentRel = trim('Restored ' . date('Y-m-d H.i') . '/' . $parentRel, '/');
|
|
} elseif ($mode !== self::MODE_MERGE && $mode !== self::MODE_REPLACE) {
|
|
throw new \InvalidArgumentException("Unknown mode $mode");
|
|
}
|
|
$dest = $this->ensureFolder($userFolder, $parentRel);
|
|
$names = $single ? [basename($rel) => $entries['']] : $entries;
|
|
|
|
$restored = 0;
|
|
foreach ($names as $relPath => $entry) {
|
|
$this->writeFile($repo, $dest, (string)$relPath, $entry);
|
|
$restored++;
|
|
if ($progress !== null) {
|
|
$progress($restored, count($names), (string)$relPath);
|
|
}
|
|
}
|
|
|
|
$trashed = 0;
|
|
if ($mode === self::MODE_REPLACE && !$single) {
|
|
$trashed = $this->trashExtras($dest, '', array_flip(array_keys($names)));
|
|
}
|
|
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
|
|
}
|
|
|
|
/**
|
|
* Raw restore of any snapshot path to a local directory (administrators, disaster recovery).
|
|
*
|
|
* @param bool $stripPrefix write into $directory itself instead of $directory/$prefix (see
|
|
* Repository::restore()); used to restore "data/" or "config/" straight into the
|
|
* real data/config directory for occ backup:restore:full.
|
|
*/
|
|
public function restoreToDirectory(Target $target, string $snapshotId, string $prefix, string $directory, bool $stripPrefix = false): int {
|
|
if ($prefix !== '' && !Repository::isSafePath($prefix)) {
|
|
throw new \InvalidArgumentException('Invalid path: ' . $prefix);
|
|
}
|
|
if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) {
|
|
throw new \RuntimeException("Cannot create $directory");
|
|
}
|
|
return $this->withSharedLock($target, fn () => $this->targets->repository($target)->restore($snapshotId, $directory, trim($prefix, '/') === '' ? '' : trim($prefix, '/') . '/', $stripPrefix), 3600);
|
|
}
|
|
|
|
private function writeFile(Repository $repo, Folder $dest, string $relPath, array $entry): void {
|
|
$dir = dirname($relPath);
|
|
$folder = $dir === '.' ? $dest : $this->ensureFolder($dest, $dir);
|
|
$name = basename($relPath);
|
|
// Stage under a temporary name; only rename into place after every blob was verified.
|
|
$tmpName = '.' . $name . '.ngb-restore-' . bin2hex(random_bytes(3));
|
|
$tmp = $folder->newFile($tmpName);
|
|
try {
|
|
$out = $tmp->fopen('wb');
|
|
$written = 0;
|
|
foreach ($repo->readBlobs($entry['b']) as $data) { // each blob is verified on read
|
|
if (fwrite($out, $data) !== strlen($data)) {
|
|
throw new \RuntimeException('Write failed for ' . $relPath);
|
|
}
|
|
$written += strlen($data);
|
|
}
|
|
if (!fclose($out)) {
|
|
throw new \RuntimeException('Write failed for ' . $relPath);
|
|
}
|
|
if ($written !== $entry['s']) {
|
|
throw new \RuntimeException("Size mismatch restoring $relPath");
|
|
}
|
|
if ($folder->nodeExists($name)) {
|
|
$existing = $folder->get($name);
|
|
if (!$existing instanceof File) {
|
|
throw new \RuntimeException("$relPath exists and is not a file");
|
|
}
|
|
// Overwrite through the Files API: the current content becomes a version.
|
|
$in = $tmp->fopen('rb');
|
|
$existing->putContent($in);
|
|
if (is_resource($in)) {
|
|
fclose($in);
|
|
}
|
|
$tmp->delete();
|
|
$existing->touch($entry['m']);
|
|
} else {
|
|
$tmp->move($folder->getPath() . '/' . $name);
|
|
$folder->get($name)->touch($entry['m']);
|
|
}
|
|
} catch (\Throwable $e) {
|
|
try {
|
|
$tmp->delete();
|
|
} catch (\Throwable) {
|
|
}
|
|
throw $e;
|
|
}
|
|
}
|
|
|
|
private function trashExtras(Folder $folder, string $prefix, array $keep): int {
|
|
$count = 0;
|
|
foreach ($folder->getDirectoryListing() as $node) {
|
|
$rel = $prefix === '' ? $node->getName() : $prefix . '/' . $node->getName();
|
|
if ($node instanceof Folder) {
|
|
$hasKept = false;
|
|
foreach (array_keys($keep) as $k) {
|
|
if (str_starts_with((string)$k, $rel . '/')) {
|
|
$hasKept = true;
|
|
break;
|
|
}
|
|
}
|
|
if ($hasKept) {
|
|
$count += $this->trashExtras($node, $rel, $keep);
|
|
} else {
|
|
$node->delete(); // to the trash
|
|
$count++;
|
|
}
|
|
} elseif (!isset($keep[$rel])) {
|
|
$node->delete();
|
|
$count++;
|
|
}
|
|
}
|
|
return $count;
|
|
}
|
|
|
|
private function ensureFolder(Folder $base, string $path): Folder {
|
|
$path = trim($path, '/');
|
|
if ($path === '' || $path === '.') {
|
|
return $base;
|
|
}
|
|
if ($base->nodeExists($path)) {
|
|
$node = $base->get($path);
|
|
if (!$node instanceof Folder) {
|
|
throw new \RuntimeException("$path exists and is not a folder");
|
|
}
|
|
return $node;
|
|
}
|
|
return $base->newFolder($path);
|
|
}
|
|
}
|