Files
nextcloud-ng_backup/lib/Service/TargetService.php
T
dillard f18b4f0862
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
Phase 2: disaster recovery (occ backup:restore:full)
New commands, per PLAN.md 3.7:
- occ backup:key:import: adopt the master key from a recovery kit on a fresh installation
  (instead of occ backup:key:init generating a new one), so occ backup:target:add opens the
  existing repository on the location rather than creating an empty one.
- occ backup:restore:full: replace the database (consistent, one transaction), restore the data
  directory in place, and merge config.php -- refuses up front and lists what's missing when the
  Nextcloud major version or any app enabled at backup time doesn't match here (their tables must
  exist before the database restore can use them), rather than guessing or partially applying.

Repository::restore() gained an optional $stripPrefix (write "data/<uid>/..." straight into the
real data directory, not a "data/" subdirectory of it) -- restoring "data/" and "config/" this way
needs no temporary copy of what can be the largest part of an instance.

occ backup:target:add now backfills the local snapshot cache (ngb_snapshots, what occ backup:list
actually reads) when it adopts an existing repository: without this, a freshly connected location
showed no restore points at all, even though the repository itself had them. One unreadable
snapshot is skipped rather than aborting the backfill or the target add itself.

Three self-referential problems found and fixed by testing this against two real, separate
Nextcloud instances sharing only the backup location (tests/integration/full-restore-roundtrip.sh
covers this; the per-process nature of these bugs meant they were invisible from inside a single
occ invocation, including the one used to write the fixes, which is exactly why a second, fresh
process re-running occ backup:status etc. was necessary to catch them):
- The full database dump includes ng_backup's own tables and appconfig rows; restoring them
  verbatim would overwrite the key/target that is actively performing the restore, on THIS
  installation, with the backed-up instance's. Both are now preserved: ng_backup's own tables
  (and, on PostgreSQL, their sequences -- excluding a table does not exclude its sequence) are
  excluded from what gets restored, and its appconfig rows are captured before the database
  restore and written back after, in their own transaction.
- config.php's 'secret' is merged in from the backup (needed so already-encrypted content in the
  restored database, e.g. share tokens, stays decryptable), but Target::options, the master key,
  the wrapped key and the recovery kit code are all encrypted with that same instance secret
  (OCP's ICrypto, or Nextcloud AppConfig's own "sensitive" flag underneath), not with ng_backup's
  own key. Changing it without re-encrypting them first left all four permanently undecryptable
  the moment any later occ command ran. Fixed with two helpers
  (TargetService::reencryptOptionsAround(), KeyService::reencryptForSecretRotation()) that decrypt
  before and re-encrypt after whatever callback actually changes the secret; config.php is applied
  dead last in restoreFull(), after every other repository access. Not crash-safe (a process kill
  mid-rotation would leave a key missing, not just stale) -- disaster recovery is run once by
  hand, right after occ maintenance:install, a window too narrow yet to justify a two-phase write.
- A subtler variant of the same bug: Nextcloud's own AppConfig additionally encrypts a "sensitive"
  value itself, tied to the same secret, underneath our own encryption. Overwriting an existing
  sensitive key makes it read the old value first to compare, which then also fails against the
  new secret. Deleting the key before setting the new value avoids that read entirely.

Roadmap (PLAN.md 9): phase 3 (in-place full restore, user self-service) moved out to v1.3.0; phase
2 continues toward v1.2.0 with FTP/SMB, a verification/checksum-audit command, user-export
retention, and deciding whether per-user export joins the automatic schedule.
2026-10-05 23:31:08 +02:00

220 lines
8.5 KiB
PHP

<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Service;
use OCA\NgBackup\Backend\ExternalStorageFactory;
use OCA\NgBackup\Backend\IBackend;
use OCA\NgBackup\Backend\StorageBackend;
use OCA\NgBackup\Db\Snapshot;
use OCA\NgBackup\Db\SnapshotMapper;
use OCA\NgBackup\Db\Target;
use OCA\NgBackup\Db\TargetMapper;
use OCA\NgBackup\Repository\Repository;
use OCP\App\IAppManager;
use OCP\AppFramework\Db\DoesNotExistException;
use OCP\Security\ICrypto;
use OCP\Server;
/**
* Backup targets: a files_external backend + auth mechanism + options + base path. The storage
* object is built only while it is used; nothing is mounted. files_external is loaded for its
* classes but its enabled/disabled state is left as the admin set it.
*/
final class TargetService {
public function __construct(
private TargetMapper $mapper,
private KeyService $keys,
private ICrypto $crypto,
private IAppManager $appManager,
private \OCA\NgBackup\Db\DbIndexCache $indexCache,
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
private SnapshotMapper $snapshots,
) {
}
/** @return list<array{id:string, name:string, auth:list<string>, parameters:array}> */
public function availableBackends(): array {
return $this->factory()->describeBackends();
}
/**
* Add a target: connect, create the repository there (or adopt an existing one made with this
* installation's key), then save it.
*
* @param array<string, mixed> $options
* @return array{target: Target, created: bool}
*/
public function add(string $name, string $backend, string $auth, array $options, string $basePath): array {
if (!preg_match('/^[A-Za-z0-9][A-Za-z0-9 _.-]{0,63}$/', $name)) {
throw new \InvalidArgumentException('Name: letters, digits, space, _ . - (max 64)');
}
try {
$this->mapper->findByName($name);
throw new \InvalidArgumentException("A target named '$name' already exists");
} catch (\OCP\AppFramework\Db\DoesNotExistException) {
}
$keys = $this->keys->keyRing();
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath);
if ($storageBackend->exists('config')) {
$repo = Repository::openWithKey($storageBackend, $keys, $this->indexCache, $this->anchor);
$created = false;
} else {
$repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey(), $this->anchor);
$created = true;
}
foreach ($this->mapper->findAll() as $existing) {
if ($existing->getRepositoryId() === $repo->id()) {
throw new \InvalidArgumentException("This repository is already registered as '{$existing->getName()}'");
}
}
$target = new Target();
$target->setName($name);
$target->setBackend($backend);
$target->setAuth($auth);
$target->setOptions($this->crypto->encrypt(json_encode($options, JSON_THROW_ON_ERROR)));
$target->setBasePath(trim($basePath, '/'));
$target->setRepositoryId($repo->id());
$target->setCreatedAt(time());
$target = $this->mapper->insert($target);
if (!$created) {
// Disaster recovery: occ backup:list reads this local cache, not the repository
// directly, so a snapshot made by another installation must be backfilled here too.
$this->adoptSnapshots($target, $repo);
}
return ['target' => $target, 'created' => $created];
}
/**
* A target's options are encrypted with Nextcloud's own instance secret (ICrypto), not
* ng_backup's key, so changing that secret (occ backup:restore:full merging config.php)
* would otherwise leave every target permanently undecryptable. Call $applySecretChange
* (which must be the only thing that changes the secret) through here instead: every
* target's options are decrypted first, then re-encrypted under whatever secret is active
* once $applySecretChange returns.
*/
public function reencryptOptionsAround(callable $applySecretChange): mixed {
$plain = [];
foreach ($this->mapper->findAll() as $target) {
$plain[$target->getId()] = $this->options($target);
}
$result = $applySecretChange();
foreach ($this->mapper->findAll() as $target) {
if (!isset($plain[$target->getId()])) {
continue; // added by $applySecretChange itself; nothing to re-encrypt
}
$target->setOptions($this->crypto->encrypt(json_encode($plain[$target->getId()], JSON_THROW_ON_ERROR)));
$this->mapper->update($target);
}
return $result;
}
private function adoptSnapshots(Target $target, Repository $repo): void {
foreach ($repo->listSnapshots() as $snapshotId) {
try {
$this->snapshots->findOne($target->getId(), $snapshotId);
continue; // already known locally
} catch (DoesNotExistException) {
}
// One unreadable/corrupt snapshot must not stop the rest of the location's
// snapshots from being backfilled, nor abort adding the target itself.
try {
$meta = $repo->snapshotMeta($snapshotId);
$s = new Snapshot();
$s->setTargetId($target->getId());
$s->setSnapshotId($snapshotId);
$s->setKind((string)($meta['kind'] ?? 'full'));
$s->setLabel(($meta['label'] ?? '') !== '' ? $meta['label'] : null);
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
$s->setFiles((int)($meta['stats']['files'] ?? 0));
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
$this->snapshots->insert($s);
} catch (\Throwable) {
}
}
}
/** @return list<Target> */
public function list(): array {
return $this->mapper->findAll();
}
public function get(string $nameOrId): Target {
return ctype_digit($nameOrId) ? $this->mapper->find((int)$nameOrId) : $this->mapper->findByName($nameOrId);
}
/** Size limit for the data stored on this location (null = none). */
public function setMaxBytes(Target $target, ?int $maxBytes): Target {
$target->setMaxBytes($maxBytes);
return $this->mapper->update($target);
}
public function setAppendOnly(Target $target, bool $appendOnly): Target {
$target->setAppendOnly($appendOnly);
return $this->mapper->update($target);
}
/** Remove the target from NG Backup. The backups on the target itself are not touched. */
public function remove(Target $target): void {
$this->mapper->delete($target);
}
public function backend(Target $target): IBackend {
return $this->buildBackend($target->getBackend(), $target->getAuth(), $this->options($target), $target->getBasePath());
}
public function repository(Target $target): Repository {
$repo = Repository::openWithKey($this->backend($target), $this->keys->keyRing(), $this->indexCache, $this->anchor);
if ($target->getRepositoryId() !== null && $repo->id() !== $target->getRepositoryId()) {
throw new \RuntimeException('The target now contains a different repository than when it was added');
}
$repo->setMaxBytes($target->getMaxBytes());
return $repo;
}
/** @return array<string, mixed> */
public function options(Target $target): array {
return json_decode($this->crypto->decrypt($target->getOptions()), true, 512, JSON_THROW_ON_ERROR);
}
private function buildBackend(string $backend, string $auth, array $options, string $basePath): StorageBackend {
return new StorageBackend($this->factory()->create($backend, $auth, $options), $basePath);
}
private ?ExternalStorageFactory $factory = null;
/**
* files_external's connection code, without changing the app's state:
* - enabled: it is booted by Nextcloud; use its own BackendService;
* - disabled: do NOT boot it (booting registers its mount provider and listeners, which use
* its database tables; on installations where it was never enabled those do not exist).
* Only its classes are loaded, and a private BackendService gets its backends and auth
* mechanisms from its Application class.
*/
private function factory(): ExternalStorageFactory {
if ($this->factory !== null) {
return $this->factory;
}
if ($this->appManager->isEnabledForAnyone('files_external')) {
$this->appManager->loadApp('files_external');
return $this->factory = new ExternalStorageFactory(Server::get(\OCA\Files_External\Service\BackendService::class));
}
if (!class_exists(\OCA\Files_External\AppInfo\Application::class)) {
$path = $this->appManager->getAppPath('files_external');
require_once $path . '/composer/autoload.php';
}
$service = new \OCA\Files_External\Service\BackendService(
Server::get(\OCP\IAppConfig::class), Server::get(\Psr\Log\LoggerInterface::class));
$app = new \OCA\Files_External\AppInfo\Application();
$service->registerBackendProvider($app);
$service->registerAuthMechanismProvider($app);
return $this->factory = new ExternalStorageFactory($service);
}
}