Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ba7f70e0f8 | |||
| d70a869070 | |||
| c9f82a79e4 | |||
| 932f60bc5e | |||
| 935c7b6a08 | |||
| f18b4f0862 | |||
| 6cbc5d79d4 | |||
| 58711b67ef |
@@ -60,3 +60,35 @@ jobs:
|
|||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
- name: Round trip against S3 and SFTP
|
- name: Round trip against S3 and SFTP
|
||||||
run: tests/integration/targets-matrix.sh
|
run: tests/integration/targets-matrix.sh
|
||||||
|
|
||||||
|
integration-user:
|
||||||
|
# Per-user backup/restore via user_migration (occ backup:user:backup / backup:user:restore),
|
||||||
|
# an optional dependency installed from the app store. It calls user_migration's internal
|
||||||
|
# UserMigrationService, whose API can change between Nextcloud versions, so this runs on every
|
||||||
|
# supported version (not the database, which the main integration job already covers).
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
strategy:
|
||||||
|
fail-fast: false
|
||||||
|
matrix:
|
||||||
|
nextcloud: ['32', '33', '34', '35']
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- name: Set up a fresh Nextcloud instance
|
||||||
|
run: |
|
||||||
|
docker run -d --name ngbu -e SQLITE_DATABASE=nextcloud -e NEXTCLOUD_ADMIN_USER=admin \
|
||||||
|
-e NEXTCLOUD_ADMIN_PASSWORD=admin-user-roundtrip -e NEXTCLOUD_TRUSTED_DOMAINS=localhost \
|
||||||
|
-v "$PWD:/var/www/html/custom_apps/ng_backup:ro" nextcloud:${{ matrix.nextcloud }}-apache
|
||||||
|
for i in $(seq 1 90); do docker exec -u www-data ngbu php occ status 2>/dev/null | grep -q "installed: true" && break; sleep 4; done
|
||||||
|
docker exec ngbu chown www-data:www-data /var/www/html/custom_apps
|
||||||
|
- name: Per-user backup/restore round trip on Nextcloud ${{ matrix.nextcloud }}
|
||||||
|
run: tests/integration/user-roundtrip.sh "docker exec -u www-data ngbu" /var/www/html/data
|
||||||
|
|
||||||
|
integration-full-restore:
|
||||||
|
# Disaster recovery (occ backup:restore:full) on a SECOND, fresh Nextcloud instance that
|
||||||
|
# never saw the first one, sharing only the backup location. Does not depend on the database
|
||||||
|
# beyond what the main integration job already covers, so this runs once, on one NC version.
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- name: Full restore round trip
|
||||||
|
run: tests/integration/full-restore-roundtrip.sh 35-apache
|
||||||
|
|||||||
@@ -42,3 +42,5 @@ and not implemented yet.
|
|||||||
- No FTP/SMB coverage in CI yet, and no verification/checksum-audit command yet (phase 2).
|
- No FTP/SMB coverage in CI yet, and no verification/checksum-audit command yet (phase 2).
|
||||||
- See the "Storage backend experiences" section of the README for provider-specific quirks found
|
- See the "Storage backend experiences" section of the README for provider-specific quirks found
|
||||||
so far (e.g. Hetzner Storage Box sub-accounts).
|
so far (e.g. Hetzner Storage Box sub-accounts).
|
||||||
|
|
||||||
|
[0.1.0-alpha.1]: https://github.com/dillardblom/ng_backup/releases/tag/v0.1.0-alpha.1
|
||||||
|
|||||||
@@ -276,7 +276,7 @@ onderdelen wel en niet terugkomen, op basis van de geregistreerde migrators.
|
|||||||
| `backup:list [--target=]` | herstelpunten |
|
| `backup:list [--target=]` | herstelpunten |
|
||||||
| `backup:verify <snapshot> [--deep]` | checksums controleren, optioneel alles ontsleutelen |
|
| `backup:verify <snapshot> [--deep]` | checksums controleren, optioneel alles ontsleutelen |
|
||||||
| `backup:restore:file <snapshot> <user> <pad> [--to=]` | bestand/map terugzetten |
|
| `backup:restore:file <snapshot> <user> <pad> [--to=]` | bestand/map terugzetten |
|
||||||
| `backup:restore:user <snapshot> <user> [--to-user=]` | gebruiker terugzetten via user_migration (fase 2) |
|
| `backup:user:restore <snapshot> <user> [--to-user=]` | gebruiker terugzetten via user_migration (fase 2) |
|
||||||
| `backup:restore:full <snapshot>` | volledige restore (fase 2, verse installatie) |
|
| `backup:restore:full <snapshot>` | volledige restore (fase 2, verse installatie) |
|
||||||
| `backup:target:add/list/remove/test` | opslagdoelen beheren en testen |
|
| `backup:target:add/list/remove/test` | opslagdoelen beheren en testen |
|
||||||
| `backup:key:export / import` | herstelkit |
|
| `backup:key:export / import` | herstelkit |
|
||||||
@@ -338,6 +338,25 @@ een backup-app die niet betrouwbaar is, is erger dan geen backup-app.
|
|||||||
Volgorde: fase 1 grondig testen, reviewen en debuggen; dan nadenken over 0.1.0-alpha.1. Een
|
Volgorde: fase 1 grondig testen, reviewen en debuggen; dan nadenken over 0.1.0-alpha.1. Een
|
||||||
minimale volledige restore (`occ backup:restore:full` op een verse installatie) komt vóór de beta.
|
minimale volledige restore (`occ backup:restore:full` op een verse installatie) komt vóór de beta.
|
||||||
|
|
||||||
|
**Roadmap-indeling fase 2 (besluit 2026-10-05):** naar v1.2.0 toewerken met in fase 2:
|
||||||
|
1. `backup:restore:full` (disaster recovery, verse installatie) — **klaar** (2026-10-05)
|
||||||
|
2. FTP/SMB-ondersteuning — **klaar** (2026-10-06): beide liepen al via files_external's generieke
|
||||||
|
adapter zonder enige ng_backup-code; alleen testdekking toegevoegd
|
||||||
|
(`tests/integration/targets-matrix.sh`) + bevindingen in README ("Storage backend experiences").
|
||||||
|
SMB vereist de `smbclient` PECL-extensie (niet standaard in de Nextcloud-image, in CI nu
|
||||||
|
on-the-fly gebouwd). NFS is geen eigen backend: lokaal mounten + `local`-backend erop richten.
|
||||||
|
3. Verificatie/checksum-audit-commando — **klaar** (2026-10-05)
|
||||||
|
4. Retentie/prune voor user-exports — **klaar** (2026-10-06): `occ backup:retention --user-last=N`
|
||||||
|
(standaard 3, per installatie, net als de bestaande snapshot-policy), toegepast door
|
||||||
|
`occ backup:prune` (geen prullenbak voor user-exports, in tegenstelling tot snapshots — on-demand,
|
||||||
|
niet het primaire vangnet).
|
||||||
|
5. Automatische user-export inpluggen in het reguliere schema (of expliciet besluit dat het on-demand blijft)
|
||||||
|
9. SMB/NFS intern testen op de eigen OMV-instantie — nog open (intern, niet in CI)
|
||||||
|
|
||||||
|
Doorgeschoven naar **v1.3.0** (fase 3, niet meer fase 2): in-place volledige restore
|
||||||
|
(eigen restore-modus i.p.v. onderhoudsmodus) en zelfbediening voor gebruikers (eigen bestanden
|
||||||
|
terugzetten vanuit de Files-app).
|
||||||
|
|
||||||
**Let op (ontdekt 2026-10-05):** Nextcloud's eigen `info.xsd` staat voor `<version>` alleen
|
**Let op (ontdekt 2026-10-05):** Nextcloud's eigen `info.xsd` staat voor `<version>` alleen
|
||||||
`[0-9]+(\.[0-9]+){0,2}` toe — geen `-alpha`/`-beta`/`-rc`-achtervoegsel. De stabiliteitsaanduiding
|
`[0-9]+(\.[0-9]+){0,2}` toe — geen `-alpha`/`-beta`/`-rc`-achtervoegsel. De stabiliteitsaanduiding
|
||||||
uit deze tabel gaat dus niet in `info.xml` (dat blijft gewoon oplopend, bijv. `0.1.0`), maar in de
|
uit deze tabel gaat dus niet in `info.xml` (dat blijft gewoon oplopend, bijv. `0.1.0`), maar in de
|
||||||
|
|||||||
@@ -2,8 +2,10 @@
|
|||||||
|
|
||||||
Encrypted, incremental backups of Nextcloud (database, configuration and data) with restore
|
Encrypted, incremental backups of Nextcloud (database, configuration and data) with restore
|
||||||
from the web interface. Works without shell access: no external programs, no extra PHP
|
from the web interface. Works without shell access: no external programs, no extra PHP
|
||||||
extensions beyond sodium. Runs on Nextcloud All-in-One, regular installations and hosted
|
extensions beyond sodium for ng_backup itself. Runs on Nextcloud All-in-One, regular
|
||||||
Nextcloud.
|
installations and hosted Nextcloud. (Choosing SMB as a storage backend is the one exception:
|
||||||
|
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
||||||
|
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
||||||
|
|
||||||
**Status: in development (phase 1). Not ready for production use.** See `PLAN.md` for the design.
|
**Status: in development (phase 1). Not ready for production use.** See `PLAN.md` for the design.
|
||||||
|
|
||||||
@@ -119,6 +121,29 @@ Contributions welcome via PR — add what you ran into with your own provider.
|
|||||||
verified whether this also applies to a Hetzner Storage Box **main** account (only tested with a
|
verified whether this also applies to a Hetzner Storage Box **main** account (only tested with a
|
||||||
sub-account so far) — if you've tried a main account, please say so in a PR.
|
sub-account so far) — if you've tried a main account, please say so in a PR.
|
||||||
|
|
||||||
|
- **FTP, server-dependent absolute-path handling:** files_external's FTP backend always builds
|
||||||
|
absolute paths (e.g. `MKD /repo`) for the configured root, never relative ones. `atmoz/sftp`-style
|
||||||
|
`vsftpd` test servers (chrooted) can reject that with a generic "create directory operation
|
||||||
|
failed", even though a plain `ftp_mkdir()` with a *relative* path on the same server succeeds;
|
||||||
|
`pure-ftpd` accepts the absolute form fine. If a working FTP server rejects ng_backup's writes,
|
||||||
|
try a different FTP server implementation before assuming the backend itself is broken.
|
||||||
|
|
||||||
|
- **SMB/CIFS *lists* as available with just the `smbclient` CLI binary, but does not actually
|
||||||
|
work for ng_backup without the `smbclient` PECL extension too.** files_external falls back to a
|
||||||
|
shell-wrapped `smbclient` process when the native extension is missing, which registers the
|
||||||
|
backend in `occ backup:target:backends` fine, but that fallback's stream cannot seek. ng_backup
|
||||||
|
reads blobs back by byte range while backing up, so even the very first (small) backup fails
|
||||||
|
with `Seek failed in packs/...`. Install the extension (`libsmbclient-dev` + `pecl install
|
||||||
|
smbclient` + `docker-php-ext-enable smbclient` on Debian-based images), not just the CLI tool —
|
||||||
|
the backend being listed is not proof it will work here. Once the extension is present it works
|
||||||
|
the same as any other backend; no ng_backup-specific config beyond host/share/user/password.
|
||||||
|
|
||||||
|
- **NFS is not a files_external backend at all.** Mount the NFS export on the server yourself
|
||||||
|
(e.g. via `/etc/fstab` or a systemd `.mount` unit) and point ng_backup's `local` backend at the
|
||||||
|
mount point; ng_backup never mounts or unmounts anything on its own, so an NFS mount dropping
|
||||||
|
mid-run behaves like any other local-path failure (the run resumes). See PLAN.md 3.5 for why NFS
|
||||||
|
can't be driven from PHP directly.
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
| Command | Purpose |
|
| Command | Purpose |
|
||||||
|
|||||||
@@ -47,6 +47,11 @@ relying on this for anything.]]></description>
|
|||||||
<command>OCA\NgBackup\Command\KeySlotAdd</command>
|
<command>OCA\NgBackup\Command\KeySlotAdd</command>
|
||||||
<command>OCA\NgBackup\Command\KeySlotReplace</command>
|
<command>OCA\NgBackup\Command\KeySlotReplace</command>
|
||||||
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
||||||
|
<command>OCA\NgBackup\Command\BackupUser</command>
|
||||||
|
<command>OCA\NgBackup\Command\RestoreUser</command>
|
||||||
|
<command>OCA\NgBackup\Command\KeyImport</command>
|
||||||
|
<command>OCA\NgBackup\Command\RestoreFull</command>
|
||||||
|
<command>OCA\NgBackup\Command\Verify</command>
|
||||||
</commands>
|
</commands>
|
||||||
<settings>
|
<settings>
|
||||||
<admin>OCA\NgBackup\Settings\Admin</admin>
|
<admin>OCA\NgBackup\Settings\Admin</admin>
|
||||||
|
|||||||
@@ -0,0 +1,59 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use OCA\NgBackup\Service\UserBackupService;
|
||||||
|
use OCP\IUserManager;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class BackupUser extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private UserBackupService $userBackup,
|
||||||
|
private IUserManager $userManager,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:user:backup')
|
||||||
|
->setDescription('Back up one user via user_migration (account, settings, files and everything else a migrator covers)')
|
||||||
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
|
->addArgument('user', InputArgument::REQUIRED, 'User id')
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
Streams a user_migration export straight into the encrypted repository: no zip, no copy in the
|
||||||
|
user's own storage. Shares data with the regular backup; this needs the user_migration app
|
||||||
|
enabled. The manifest path this prints is what occ backup:user:restore expects.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
occ backup:user:backup offsite alice
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$uid = (string)$input->getArgument('user');
|
||||||
|
$user = $this->userManager->get($uid);
|
||||||
|
if ($user === null) {
|
||||||
|
$output->writeln("<error>No such user: $uid</error>");
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
$target = $this->targets->get($input->getArgument('target'));
|
||||||
|
$manifest = $this->userBackup->backupUser($target, $user);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln("Backed up <info>$uid</info> to $manifest.");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\KeyService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Input\InputOption;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
use Symfony\Component\Console\Question\Question;
|
||||||
|
|
||||||
|
final class KeyImport extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private KeyService $keys,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:key:import')
|
||||||
|
->setDescription('Disaster recovery: adopt the backup key from a recovery kit on a fresh installation')
|
||||||
|
->addArgument('kit-file', InputArgument::REQUIRED, 'The recovery kit file (occ backup:key:kit on the original installation)')
|
||||||
|
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Read the passphrase from this file instead of asking')
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
For a fresh installation only (occ backup:key:init must not have been run here yet). Reads one
|
||||||
|
of the kit's passphrase slots to recover the original master key, so occ backup:target:add opens
|
||||||
|
the existing repository on the location instead of creating a new, empty one.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
occ backup:key:import /root/ng_backup-recovery-kit.json
|
||||||
|
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=...
|
||||||
|
occ backup:restore:full offsite <snapshot>
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
if ($this->keys->isInitialized()) {
|
||||||
|
$output->writeln('<error>A backup key already exists here (fingerprint ' . $this->keys->fingerprint() . '). occ backup:key:import is only for a fresh installation.</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$kitFile = (string)$input->getArgument('kit-file');
|
||||||
|
$raw = @file_get_contents($kitFile);
|
||||||
|
if ($raw === false) {
|
||||||
|
$output->writeln("<error>Cannot read $kitFile</error>");
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
$kit = json_decode($raw, true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
} catch (\JsonException $e) {
|
||||||
|
$output->writeln('<error>Not a valid recovery kit file: ' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$file = $input->getOption('passphrase-file');
|
||||||
|
if ($file !== null) {
|
||||||
|
$pass = rtrim((string)@file_get_contents($file), "\r\n");
|
||||||
|
} else {
|
||||||
|
$helper = $this->getHelper('question');
|
||||||
|
$q = (new Question('Passphrase of one of the kit\'s slots: '))->setHidden(true)->setHiddenFallback(false);
|
||||||
|
$pass = (string)$helper->ask($input, $output, $q);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
$this->keys->importRecoveryKit($kit, $pass);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln('Backup key imported, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
||||||
|
$output->writeln('Next: connect the backup location with the same details as before (<info>occ backup:target:add</info>), then <info>occ backup:list</info> and <info>occ backup:restore:full</info>.');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -37,6 +37,20 @@ final class Prune extends Command {
|
|||||||
$output->writeln('Policy: ' . json_encode($this->prune->policy()->toArray()) . ($dry ? ' (dry run)' : ''));
|
$output->writeln('Policy: ' . json_encode($this->prune->policy()->toArray()) . ($dry ? ' (dry run)' : ''));
|
||||||
$rc = 0;
|
$rc = 0;
|
||||||
foreach ($targets as $t) {
|
foreach ($targets as $t) {
|
||||||
|
// User exports first: Repository::prune()'s blob garbage collection (run as part of
|
||||||
|
// apply() below) scans the users/ manifests still on the location at that point, so
|
||||||
|
// forgetting one here lets its now-unused blobs be freed in this same pass.
|
||||||
|
try {
|
||||||
|
$ru = $this->prune->applyUserExports($t, $dry);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $t->getName() . ' (user exports): ' . $e->getMessage() . '</error>');
|
||||||
|
$rc = 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if ($ru['forgotten'] !== []) {
|
||||||
|
$output->writeln(sprintf('<info>%s</info>: %s %d user export(s) for %d user(s)',
|
||||||
|
$t->getName(), $dry ? 'would forget' : 'forgot', count($ru['forgotten']), count($ru['kept'])));
|
||||||
|
}
|
||||||
try {
|
try {
|
||||||
$r = $this->prune->apply($t, $dry);
|
$r = $this->prune->apply($t, $dry);
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\RestoreFullService;
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class RestoreFull extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private RestoreFullService $restoreFull,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:restore:full')
|
||||||
|
->setDescription('Disaster recovery: restore the database, data directory and config on a fresh installation')
|
||||||
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
|
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
For a FRESH Nextcloud installation only, same major version as the backup, right after
|
||||||
|
occ maintenance:install, occ backup:key:import and occ backup:target:add -- before anyone else
|
||||||
|
uses this instance.
|
||||||
|
|
||||||
|
Every app that was enabled when the snapshot was taken must already be installed here (its
|
||||||
|
database tables must exist before the restore can write into them); the command lists what is
|
||||||
|
missing and refuses to continue rather than guess.
|
||||||
|
|
||||||
|
Steps: the database is replaced (consistent, in one transaction); the data directory is restored
|
||||||
|
directly in place; config.php is merged in -- everything from the backup except the database
|
||||||
|
connection and data directory settings, which stay as occ maintenance:install configured them
|
||||||
|
here. Other files under config/ (if any) are not restored.
|
||||||
|
|
||||||
|
Afterwards: check trusted_domains and overwritehost/overwriteprotocol match this server, and
|
||||||
|
occ maintenance:repair.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
occ backup:restore:full offsite 537bf7e9c1a4...
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
try {
|
||||||
|
$target = $this->targets->get($input->getArgument('target'));
|
||||||
|
$snapshot = (string)$input->getArgument('snapshot');
|
||||||
|
$result = $this->restoreFull->restoreFull($target, $snapshot);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln(sprintf(
|
||||||
|
'Database restored (<info>%d</info> tables, <info>%d</info> unchanged), config merged (<info>%d</info> keys), <info>%d</info> data files restored.',
|
||||||
|
count($result['dbReport']['restored']),
|
||||||
|
count($result['dbReport']['skipped']),
|
||||||
|
$result['configKeys'],
|
||||||
|
$result['dataFiles'],
|
||||||
|
));
|
||||||
|
$output->writeln('Next: check trusted_domains and overwritehost/overwriteprotocol match this server, then occ maintenance:repair.');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use OCA\NgBackup\Service\UserRestoreService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Input\InputOption;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class RestoreUser extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private UserRestoreService $userRestore,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:user:restore')
|
||||||
|
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
||||||
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
|
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>)')
|
||||||
|
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
Never imports over a live account. Modes:
|
||||||
|
replace (default) safety-exports the current account first, then deletes and recreates it
|
||||||
|
fresh from the backup, under the same user id
|
||||||
|
as-backup restores into a new account "<uid>-bak" (or "-bak-2", ...) instead, so an admin can
|
||||||
|
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
||||||
|
the new name, never the existing account
|
||||||
|
|
||||||
|
Example:
|
||||||
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
||||||
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
try {
|
||||||
|
$target = $this->targets->get($input->getArgument('target'));
|
||||||
|
$user = $this->userRestore->restoreUser($target, (string)$input->getArgument('manifest'), (string)$input->getOption('mode'));
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln('Restored <info>' . $user->getUID() . '</info>.');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -28,7 +28,8 @@ final class Retention extends Command {
|
|||||||
->addOption('daily', null, InputOption::VALUE_REQUIRED, 'Keep one per day for the last N days with a backup')
|
->addOption('daily', null, InputOption::VALUE_REQUIRED, 'Keep one per day for the last N days with a backup')
|
||||||
->addOption('weekly', null, InputOption::VALUE_REQUIRED, 'Keep one per week for N weeks')
|
->addOption('weekly', null, InputOption::VALUE_REQUIRED, 'Keep one per week for N weeks')
|
||||||
->addOption('monthly', null, InputOption::VALUE_REQUIRED, 'Keep one per month for N months')
|
->addOption('monthly', null, InputOption::VALUE_REQUIRED, 'Keep one per month for N months')
|
||||||
->setHelp("Example: occ backup:retention --last=3 --daily=7 --weekly=4 --monthly=12");
|
->addOption('user-last', null, InputOption::VALUE_REQUIRED, 'Keep the newest N of each user\'s own exports (occ backup:user:backup)')
|
||||||
|
->setHelp("Example: occ backup:retention --last=3 --daily=7 --weekly=4 --monthly=12 --user-last=3");
|
||||||
}
|
}
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
@@ -48,7 +49,16 @@ final class Retention extends Command {
|
|||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if ($input->getOption('user-last') !== null) {
|
||||||
|
try {
|
||||||
|
$this->prune->setUserExportRetention((int)$input->getOption('user-last'));
|
||||||
|
} catch (\InvalidArgumentException $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
$output->writeln(sprintf('Keep the last %d, plus one per day for %d days, per week for %d weeks, per month for %d months.', $p['last'], $p['daily'], $p['weekly'], $p['monthly']));
|
$output->writeln(sprintf('Keep the last %d, plus one per day for %d days, per week for %d weeks, per month for %d months.', $p['last'], $p['daily'], $p['weekly'], $p['monthly']));
|
||||||
|
$output->writeln(sprintf('Keep the newest %d of each user\'s own exports.', $this->prune->userExportRetention()));
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,75 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use OCA\NgBackup\Service\VerifyService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Input\InputOption;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class Verify extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private VerifyService $verify,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:verify')
|
||||||
|
->setDescription('Checksum audit of a snapshot: every blob it needs must be in the (freshly re-checked) index and its pack present')
|
||||||
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
|
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||||
|
->addOption('deep', null, InputOption::VALUE_NONE, 'Also download and decrypt every blob (verifies its authentication tag and content hash, not just that it is listed); much slower')
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
Without --deep this re-downloads and re-authenticates the index (never trusting the local cache)
|
||||||
|
and confirms every blob the snapshot needs is listed and its pack is present -- fast, no blob
|
||||||
|
content is read, so it cannot by itself catch a truncated or bit-rotted pack. With --deep every
|
||||||
|
blob is also downloaded and decrypted, which fails loudly on exactly that even if the index still
|
||||||
|
lists it -- the same verification an actual restore would do, run here deliberately instead of
|
||||||
|
waiting to find out during one.
|
||||||
|
|
||||||
|
Run this periodically (cron, picking a sample of recent snapshots) rather than only right before
|
||||||
|
you'd need to rely on a restore.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
occ backup:verify offsite 537bf7e9c1a4...
|
||||||
|
occ backup:verify offsite 537bf7e9c1a4... --deep
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
try {
|
||||||
|
$target = $this->targets->get($input->getArgument('target'));
|
||||||
|
$ref = (string)$input->getArgument('snapshot');
|
||||||
|
$deep = (bool)$input->getOption('deep');
|
||||||
|
$result = str_starts_with($ref, 'users/')
|
||||||
|
? $this->verify->verifyUserExport($target, $ref, $deep)
|
||||||
|
: $this->verify->verify($target, $ref, $deep);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln(sprintf('%d files, %d blobs checked%s.', $result['filesChecked'], $result['blobsChecked'],
|
||||||
|
$deep ? sprintf(' (%.1f MiB decrypted)', $result['bytesChecked'] / 1048576) : ''));
|
||||||
|
if ($result['missing'] !== []) {
|
||||||
|
$output->writeln('<error>' . count($result['missing']) . ' blob(s) missing from the index:</error> ' . implode(', ', $result['missing']));
|
||||||
|
}
|
||||||
|
if ($result['failed'] !== []) {
|
||||||
|
$output->writeln('<error>' . count($result['failed']) . ' blob(s) failed verification (corrupted or tampered):</error> ' . implode(', ', $result['failed']));
|
||||||
|
}
|
||||||
|
if ($result['missing'] === [] && $result['failed'] === []) {
|
||||||
|
$output->writeln('<info>OK</info>');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -28,6 +28,8 @@ final class Catalog {
|
|||||||
public array $snapshots = [];
|
public array $snapshots = [];
|
||||||
/** @var array<string, true> */
|
/** @var array<string, true> */
|
||||||
public array $trash = [];
|
public array $trash = [];
|
||||||
|
/** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */
|
||||||
|
public array $userExports = [];
|
||||||
public string $hash = '';
|
public string $hash = '';
|
||||||
|
|
||||||
public function __construct(
|
public function __construct(
|
||||||
@@ -39,7 +41,7 @@ final class Catalog {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/** Load and verify the newest catalog; create the first one for repositories that have none. */
|
/** Load and verify the newest catalog; create the first one for repositories that have none. */
|
||||||
public function load(callable $listSnapshots, callable $listTrash): void {
|
public function load(callable $listSnapshots, callable $listTrash, callable $listUserExports): void {
|
||||||
$gens = $this->generations();
|
$gens = $this->generations();
|
||||||
if ($gens === []) {
|
if ($gens === []) {
|
||||||
// First use (new or pre-catalog repository): adopt the current state as generation 1.
|
// First use (new or pre-catalog repository): adopt the current state as generation 1.
|
||||||
@@ -49,6 +51,7 @@ final class Catalog {
|
|||||||
}
|
}
|
||||||
$this->snapshots = array_fill_keys($listSnapshots(), true);
|
$this->snapshots = array_fill_keys($listSnapshots(), true);
|
||||||
$this->trash = array_fill_keys($listTrash(), true);
|
$this->trash = array_fill_keys($listTrash(), true);
|
||||||
|
$this->userExports = array_fill_keys($listUserExports(), true);
|
||||||
$this->write();
|
$this->write();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -76,6 +79,9 @@ final class Catalog {
|
|||||||
$this->hash = $hash;
|
$this->hash = $hash;
|
||||||
$this->snapshots = array_fill_keys($data['snapshots'], true);
|
$this->snapshots = array_fill_keys($data['snapshots'], true);
|
||||||
$this->trash = array_fill_keys($data['trash'], true);
|
$this->trash = array_fill_keys($data['trash'], true);
|
||||||
|
// Catalogs written before user exports existed have no such key: default to empty rather
|
||||||
|
// than adopting the location's current users/ listing (that would skip verification).
|
||||||
|
$this->userExports = array_fill_keys($data['userExports'] ?? [], true);
|
||||||
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
|
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -83,7 +89,8 @@ final class Catalog {
|
|||||||
public function write(): void {
|
public function write(): void {
|
||||||
$next = $this->gen + 1;
|
$next = $this->gen + 1;
|
||||||
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
|
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
|
||||||
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash)], JSON_THROW_ON_ERROR);
|
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash),
|
||||||
|
'userExports' => array_keys($this->userExports)], JSON_THROW_ON_ERROR);
|
||||||
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
|
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
|
||||||
$stream = fopen('php://memory', 'w+b');
|
$stream = fopen('php://memory', 'w+b');
|
||||||
fwrite($stream, $encrypted);
|
fwrite($stream, $encrypted);
|
||||||
|
|||||||
@@ -91,7 +91,8 @@ final class Repository {
|
|||||||
$this->catalog = new Catalog($this->backend, $this->cipher, $this->repositoryId, $anchor);
|
$this->catalog = new Catalog($this->backend, $this->cipher, $this->repositoryId, $anchor);
|
||||||
$this->catalog->load(
|
$this->catalog->load(
|
||||||
fn () => array_map('basename', $this->backend->list('snapshots')),
|
fn () => array_map('basename', $this->backend->list('snapshots')),
|
||||||
fn () => array_map('basename', $this->backend->list('trash/info')));
|
fn () => array_map('basename', $this->backend->list('trash/info')),
|
||||||
|
fn () => $this->backend->list('users'));
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Refuse uploads that would make the stored data larger than this (null = no limit). */
|
/** Refuse uploads that would make the stored data larger than this (null = no limit). */
|
||||||
@@ -189,8 +190,15 @@ final class Repository {
|
|||||||
return ['snapshot' => $snapshotId] + $stats;
|
return ['snapshot' => $snapshotId] + $stats;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Restore a snapshot (or only paths starting with $prefix) into a local directory. */
|
/**
|
||||||
public function restore(string $snapshotId, string $target, string $prefix = ''): int {
|
* Restore a snapshot (or only paths starting with $prefix) into a local directory.
|
||||||
|
*
|
||||||
|
* @param bool $stripPrefix write "$target/<path without $prefix>" instead of
|
||||||
|
* "$target/<full logical path>" (disaster recovery: restoring e.g. "data/" or
|
||||||
|
* "config/" directly into the real data/config directory, not a subdirectory
|
||||||
|
* named after the prefix). Ignored when $prefix is '' (nothing to strip).
|
||||||
|
*/
|
||||||
|
public function restore(string $snapshotId, string $target, string $prefix = '', bool $stripPrefix = false): int {
|
||||||
if ($prefix !== '' && !self::isSafePath(rtrim($prefix, '/'))) {
|
if ($prefix !== '' && !self::isSafePath(rtrim($prefix, '/'))) {
|
||||||
throw new RepositoryException('Invalid path prefix');
|
throw new RepositoryException('Invalid path prefix');
|
||||||
}
|
}
|
||||||
@@ -199,7 +207,8 @@ final class Repository {
|
|||||||
if ($prefix !== '' && !str_starts_with($entry['p'], $prefix)) {
|
if ($prefix !== '' && !str_starts_with($entry['p'], $prefix)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
$dest = rtrim($target, '/') . '/' . $entry['p'];
|
$relPath = ($stripPrefix && $prefix !== '') ? substr($entry['p'], strlen(rtrim($prefix, '/')) + 1) : $entry['p'];
|
||||||
|
$dest = rtrim($target, '/') . '/' . $relPath;
|
||||||
if (!is_dir(dirname($dest))) {
|
if (!is_dir(dirname($dest))) {
|
||||||
mkdir(dirname($dest), 0700, true);
|
mkdir(dirname($dest), 0700, true);
|
||||||
}
|
}
|
||||||
@@ -374,6 +383,50 @@ final class Repository {
|
|||||||
$this->backend->delete('trash/info/' . $snapshotId);
|
$this->backend->delete('trash/info/' . $snapshotId);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Record a user_migration export's manifest in the catalog (rollback-protected like a snapshot). */
|
||||||
|
public function recordUserExport(string $manifestPath): void {
|
||||||
|
$this->catalog->userExports[$manifestPath] = true;
|
||||||
|
$this->catalog->write();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* User export manifests according to the verified catalog, newest first. A manifest the
|
||||||
|
* catalog lists but the location no longer has means it was deleted or hidden behind NG
|
||||||
|
* Backup's back, same as a missing snapshot (see listSnapshots()).
|
||||||
|
*
|
||||||
|
* @return list<string>
|
||||||
|
*/
|
||||||
|
public function listUserExports(?string $uid = null): array {
|
||||||
|
$prefix = $uid !== null ? 'users/' . $uid . '/' : 'users/';
|
||||||
|
$out = [];
|
||||||
|
foreach (array_keys($this->catalog->userExports) as $path) {
|
||||||
|
if (!str_starts_with($path, $prefix)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (!$this->backend->exists($path)) {
|
||||||
|
throw new RepositoryException("User export $path is missing on the location (deleted or hidden outside NG Backup)");
|
||||||
|
}
|
||||||
|
$out[] = $path;
|
||||||
|
}
|
||||||
|
sort($out, SORT_STRING);
|
||||||
|
return array_reverse($out);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether $manifestPath is a user export the verified catalog actually recorded. */
|
||||||
|
public function isRecordedUserExport(string $manifestPath): bool {
|
||||||
|
return isset($this->catalog->userExports[$manifestPath]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
||||||
|
public function forgetUserExport(string $manifestPath): void {
|
||||||
|
if (!isset($this->catalog->userExports[$manifestPath])) {
|
||||||
|
throw new RepositoryException("$manifestPath is not a recorded user export");
|
||||||
|
}
|
||||||
|
$this->backend->delete($manifestPath);
|
||||||
|
unset($this->catalog->userExports[$manifestPath]);
|
||||||
|
$this->catalog->write();
|
||||||
|
}
|
||||||
|
|
||||||
/** @return array<string, array{forgottenAt:int, by:string, db:?string, dbInTrash:?string, time:?string, label:string}> */
|
/** @return array<string, array{forgottenAt:int, by:string, db:?string, dbInTrash:?string, time:?string, label:string}> */
|
||||||
public function trash(): array {
|
public function trash(): array {
|
||||||
$out = [];
|
$out = [];
|
||||||
@@ -609,6 +662,117 @@ final class Repository {
|
|||||||
return $this->streamEntries($snapshotId);
|
return $this->streamEntries($snapshotId);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checksum audit of a snapshot: every blob it references (files and the database dump, when
|
||||||
|
* the snapshot has one) must be in the index, freshly re-downloaded and re-authenticated from
|
||||||
|
* the location rather than trusted from the local cache (same reasoning as prune()), and its
|
||||||
|
* pack must still be present. With $deep, each blob is also downloaded and decrypted, which
|
||||||
|
* verifies its AEAD authentication tag and its content hash against the blob id (loadBlob()
|
||||||
|
* already does both on every real read -- this just does it for everything in the snapshot,
|
||||||
|
* deliberately, rather than relying on it to come up during a later restore). A blob
|
||||||
|
* referenced by more than one file is only checked once; a missing database manifest is a
|
||||||
|
* failure, not something to silently skip over.
|
||||||
|
*
|
||||||
|
* @param ?callable $heartbeat called after every blob (shallow) or every blob read (deep);
|
||||||
|
* callers use it to keep a time-boxed lease alive during a large audit
|
||||||
|
* @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list<string>, failed:list<string>}
|
||||||
|
*/
|
||||||
|
public function verify(string $snapshotId, bool $deep = false, ?callable $heartbeat = null): array {
|
||||||
|
$heartbeat ??= static function (): void {
|
||||||
|
};
|
||||||
|
$this->index->load(true);
|
||||||
|
$stats = ['filesChecked' => 0, 'blobsChecked' => 0, 'bytesChecked' => 0, 'missing' => [], 'failed' => []];
|
||||||
|
$seen = [];
|
||||||
|
$packSeen = []; // pack id => exists on the backend; avoids one exists() call per blob
|
||||||
|
foreach ($this->streamEntries($snapshotId) as $entry) {
|
||||||
|
$stats['filesChecked']++;
|
||||||
|
foreach ($entry['b'] as $id) {
|
||||||
|
if (isset($seen[$id])) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
$seen[$id] = true;
|
||||||
|
$this->verifyBlob($id, $deep, $stats, $packSeen);
|
||||||
|
$heartbeat();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$meta = $this->snapshotMeta($snapshotId);
|
||||||
|
if (isset($meta['db']) && is_string($meta['db'])) {
|
||||||
|
if (!$this->backend->exists($meta['db'])) {
|
||||||
|
$stats['missing'][] = $meta['db'];
|
||||||
|
} else {
|
||||||
|
$dbManifest = json_decode($this->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
foreach ($dbManifest['tables'] as $info) {
|
||||||
|
foreach ($info['blobs'] as $id) {
|
||||||
|
if (isset($seen[$id])) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
$seen[$id] = true;
|
||||||
|
$this->verifyBlob($id, $deep, $stats, $packSeen);
|
||||||
|
$heartbeat();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return $stats;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Same checks as verify(), for a user export: every blob its manifest references must be in
|
||||||
|
* the freshly re-authenticated index with its pack present; $deep also decrypts each.
|
||||||
|
*
|
||||||
|
* @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list<string>, failed:list<string>}
|
||||||
|
*/
|
||||||
|
public function verifyUserExport(string $manifestPath, bool $deep = false, ?callable $heartbeat = null): array {
|
||||||
|
if (!$this->isRecordedUserExport($manifestPath)) {
|
||||||
|
throw new RepositoryException("$manifestPath is not a recorded user export for this location");
|
||||||
|
}
|
||||||
|
$heartbeat ??= static function (): void {
|
||||||
|
};
|
||||||
|
$this->index->load(true);
|
||||||
|
$stats = ['filesChecked' => 0, 'blobsChecked' => 0, 'bytesChecked' => 0, 'missing' => [], 'failed' => []];
|
||||||
|
$seen = [];
|
||||||
|
$packSeen = [];
|
||||||
|
$manifest = json_decode($this->getObject($manifestPath), true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
foreach ($manifest['entries'] as $entry) {
|
||||||
|
if (($entry['t'] ?? '') !== 'f') {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
$stats['filesChecked']++;
|
||||||
|
foreach ($entry['b'] ?? [] as $id) {
|
||||||
|
if (isset($seen[$id])) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
$seen[$id] = true;
|
||||||
|
$this->verifyBlob($id, $deep, $stats, $packSeen);
|
||||||
|
$heartbeat();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return $stats;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param array<string, bool> $packSeen */
|
||||||
|
private function verifyBlob(string $id, bool $deep, array &$stats, array &$packSeen): void {
|
||||||
|
if (!$this->index->has($id)) {
|
||||||
|
$stats['missing'][] = $id;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
[$pack] = $this->index->get($id);
|
||||||
|
$packExists = $packSeen[$pack] ??= $this->backend->exists('packs/' . substr($pack, 0, 2) . '/' . $pack);
|
||||||
|
if (!$packExists) {
|
||||||
|
$stats['missing'][] = $id;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
$stats['blobsChecked']++;
|
||||||
|
if (!$deep) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
$stats['bytesChecked'] += strlen($this->loadBlob($id));
|
||||||
|
} catch (\Throwable) {
|
||||||
|
$stats['failed'][] = $id;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** The meta record (first line) of a snapshot. */
|
/** The meta record (first line) of a snapshot. */
|
||||||
public function snapshotMeta(string $snapshotId): array {
|
public function snapshotMeta(string $snapshotId): array {
|
||||||
$plain = fopen('php://temp/maxmemory:' . (4 * 1048576), 'w+b');
|
$plain = fopen('php://temp/maxmemory:' . (4 * 1048576), 'w+b');
|
||||||
|
|||||||
@@ -71,6 +71,84 @@ final class KeyService {
|
|||||||
$this->cached = $keys;
|
$this->cached = $keys;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Disaster recovery on a fresh installation: adopt the master key from a recovery kit
|
||||||
|
* (occ backup:key:kit on the original installation) instead of generating a new one.
|
||||||
|
* Afterwards occ backup:target:add opens the existing repository on the location with this
|
||||||
|
* key, instead of creating a new, empty one.
|
||||||
|
*
|
||||||
|
* @param array<string, mixed> $kit as produced by recoveryKit()
|
||||||
|
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase, or a damaged/foreign kit
|
||||||
|
*/
|
||||||
|
public function importRecoveryKit(array $kit, #[\SensitiveParameter] string $passphrase): void {
|
||||||
|
if ($this->isInitialized()) {
|
||||||
|
throw new \RuntimeException('A backup key already exists');
|
||||||
|
}
|
||||||
|
if (!isset($kit['wrapped_key'])) {
|
||||||
|
throw new \InvalidArgumentException('Not a NG Backup recovery kit file');
|
||||||
|
}
|
||||||
|
$keys = KeyRing::unwrapAny($kit['wrapped_key'], $passphrase);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
|
||||||
|
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
|
||||||
|
$this->cached = $keys;
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode($kit['wrapped_key'], JSON_THROW_ON_ERROR), true, true);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY, self::DEFAULT_DELETE_DELAY, true);
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, (int)($kit['kit_version'] ?? 1), true);
|
||||||
|
// Having and successfully using a real kit file is stronger proof than the normal
|
||||||
|
// download-and-confirm flow it replaces.
|
||||||
|
$this->confirmRecoveryKit('cli-import');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The master key and the recovery kit code are stored encrypted with Nextcloud's own
|
||||||
|
* instance secret (ICrypto), not with themselves; the wrapped key is not (it is sealed with
|
||||||
|
* the admin's passphrase instead), but Nextcloud's own AppConfig additionally encrypts any
|
||||||
|
* "sensitive" value with the instance secret regardless, underneath whatever we store. So
|
||||||
|
* changing that secret (occ backup:restore:full merging config.php) would otherwise leave
|
||||||
|
* all three permanently undecryptable. Call $applySecretChange (which must be the only thing
|
||||||
|
* that changes the secret) through here instead: all three are decrypted first, then
|
||||||
|
* re-encrypted under whatever secret is active once $applySecretChange returns.
|
||||||
|
*
|
||||||
|
* Not crash-safe: a process kill between the delete and the re-save of one of these keys
|
||||||
|
* would leave it missing rather than merely stale. Disaster recovery is run once, by hand,
|
||||||
|
* right after occ maintenance:install; a dedicated two-phase write is not justified yet for
|
||||||
|
* that narrow a window.
|
||||||
|
*/
|
||||||
|
public function reencryptForSecretRotation(callable $applySecretChange): mixed {
|
||||||
|
$master = $this->appConfig->getValueString(Application::APP_ID, self::KEY_MASTER, '', true);
|
||||||
|
$wrapped = $this->appConfig->getValueString(Application::APP_ID, self::KEY_WRAPPED, '', true);
|
||||||
|
$kitCodeRaw = $this->appConfig->getValueString(Application::APP_ID, self::KEY_KIT_CODE, '', true);
|
||||||
|
$kitCode = $kitCodeRaw !== '' ? json_decode($kitCodeRaw, true, 512, JSON_THROW_ON_ERROR) : null;
|
||||||
|
$masterPlain = $master !== '' ? $this->crypto->decrypt($master) : null;
|
||||||
|
$kitCodePlain = $kitCode !== null && isset($kitCode['code']) ? $this->crypto->decrypt($kitCode['code']) : null;
|
||||||
|
|
||||||
|
$result = $applySecretChange();
|
||||||
|
$this->cached = null; // the master key is about to be re-saved; do not keep the pre-rotation copy
|
||||||
|
|
||||||
|
// Nextcloud's own AppConfig additionally encrypts a "sensitive" value itself (tied to
|
||||||
|
// the same secret, underneath our own encrypt() above); setValueString() on an existing
|
||||||
|
// key reads the old value first to compare, which would try to decrypt it with the new
|
||||||
|
// secret and fail the same way. Delete first so there is nothing to compare against.
|
||||||
|
if ($masterPlain !== null) {
|
||||||
|
$this->appConfig->deleteKey(Application::APP_ID, self::KEY_MASTER);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER, $this->crypto->encrypt($masterPlain), true, true);
|
||||||
|
}
|
||||||
|
if ($wrapped !== '') {
|
||||||
|
// Already plaintext JSON here: getValueString() above transparently undid
|
||||||
|
// AppConfig's own sensitive-value encryption; there is no encrypt() of our own to
|
||||||
|
// redo (wrapping is sealed with the passphrase, not the instance secret).
|
||||||
|
$this->appConfig->deleteKey(Application::APP_ID, self::KEY_WRAPPED);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, $wrapped, true, true);
|
||||||
|
}
|
||||||
|
if ($kitCodePlain !== null) {
|
||||||
|
$kitCode['code'] = $this->crypto->encrypt($kitCodePlain);
|
||||||
|
$this->appConfig->deleteKey(Application::APP_ID, self::KEY_KIT_CODE);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_KIT_CODE, json_encode($kitCode, JSON_THROW_ON_ERROR), true, true);
|
||||||
|
}
|
||||||
|
return $result;
|
||||||
|
}
|
||||||
|
|
||||||
public function keyRing(): KeyRing {
|
public function keyRing(): KeyRing {
|
||||||
if ($this->cached === null) {
|
if ($this->cached === null) {
|
||||||
$enc = $this->appConfig->getValueString(Application::APP_ID, self::KEY_MASTER, '', true);
|
$enc = $this->appConfig->getValueString(Application::APP_ID, self::KEY_MASTER, '', true);
|
||||||
|
|||||||
@@ -98,6 +98,83 @@ final class PruneService {
|
|||||||
return 'ng_backup/target/' . $target->getId();
|
return 'ng_backup/target/' . $target->getId();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A separate, short-lived exclusive lock serializing writes to the catalog itself. The
|
||||||
|
* per-target lock above only keeps readers/writers out while a cleanup runs; it does not
|
||||||
|
* stop two SHARED holders (e.g. two concurrent per-user backups, or one alongside a full
|
||||||
|
* backup) from both reading the same catalog generation and racing to write the next one,
|
||||||
|
* silently dropping whichever's change loses. The full-backup path avoids this today only
|
||||||
|
* because a Run row makes concurrent runs for the same target mutually exclusive; per-user
|
||||||
|
* backup/restore has no such row, so it takes this lock around the catalog write itself.
|
||||||
|
*/
|
||||||
|
public static function catalogLockKey(Target $target): string {
|
||||||
|
return self::lockKey($target) . '/catalog';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** How many of each user's own exports (occ backup:user:backup) to keep, newest first. */
|
||||||
|
public function userExportRetention(): int {
|
||||||
|
return $this->appConfig->getValueInt(Application::APP_ID, 'user_export_retention', 3);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function setUserExportRetention(int $keep): void {
|
||||||
|
if ($keep < 1) {
|
||||||
|
throw new \InvalidArgumentException('Keep at least the last export');
|
||||||
|
}
|
||||||
|
$old = $this->userExportRetention();
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, 'user_export_retention', $keep);
|
||||||
|
if ($old !== $keep) {
|
||||||
|
$this->alerts->securityEvent('user_export_retention_changed', ['keep' => (string)$keep]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Forgets a user's older exports once more than userExportRetention() exist for them,
|
||||||
|
* newest first; no trash, unlike a full snapshot (occ backup:user:backup is on-demand, not
|
||||||
|
* yet the primary safety net the deletion delay protects).
|
||||||
|
*
|
||||||
|
* @return array{kept: array<string, list<string>>, forgotten: list<string>}
|
||||||
|
*/
|
||||||
|
public function applyUserExports(Target $target, bool $dryRun = false): array {
|
||||||
|
if ($target->getAppendOnly()) {
|
||||||
|
throw new \RuntimeException('Location ' . $target->getName() . ' is append-only: NG Backup does not delete anything there');
|
||||||
|
}
|
||||||
|
$lease = $this->lockExclusive($target, 1800);
|
||||||
|
try {
|
||||||
|
$repo = $this->targets->repository($target);
|
||||||
|
$keep = $this->userExportRetention();
|
||||||
|
$byUser = [];
|
||||||
|
foreach ($repo->listUserExports() as $path) {
|
||||||
|
$uid = explode('/', $path)[1] ?? '';
|
||||||
|
$manifest = json_decode($repo->getObject($path), true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
$byUser[$uid][$path] = (int)strtotime((string)($manifest['time'] ?? '@0'));
|
||||||
|
}
|
||||||
|
$kept = [];
|
||||||
|
$forget = [];
|
||||||
|
foreach ($byUser as $uid => $times) {
|
||||||
|
arsort($times);
|
||||||
|
$i = 0;
|
||||||
|
foreach (array_keys($times) as $path) {
|
||||||
|
if ($i++ < $keep) {
|
||||||
|
$kept[$uid][] = $path;
|
||||||
|
} else {
|
||||||
|
$forget[] = $path;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!$dryRun) {
|
||||||
|
foreach ($forget as $path) {
|
||||||
|
$repo->forgetUserExport($path);
|
||||||
|
}
|
||||||
|
if ($forget !== []) {
|
||||||
|
$this->alerts->securityEvent('user_exports_forgotten', ['target' => $target->getName(), 'count' => (string)count($forget)], false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ['kept' => $kept, 'forgotten' => $forget];
|
||||||
|
} finally {
|
||||||
|
$this->leases->release($lease);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public function policy(): RetentionPolicy {
|
public function policy(): RetentionPolicy {
|
||||||
return RetentionPolicy::fromArray(json_decode($this->appConfig->getValueString(Application::APP_ID, 'retention', '{}'), true) ?: []);
|
return RetentionPolicy::fromArray(json_decode($this->appConfig->getValueString(Application::APP_ID, 'retention', '{}'), true) ?: []);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,225 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Db\DbDumper;
|
||||||
|
use OCA\NgBackup\Db\Target;
|
||||||
|
use OCP\App\IAppManager;
|
||||||
|
use OCP\IConfig;
|
||||||
|
use OCP\IDBConnection;
|
||||||
|
use OCP\Lock\LockedException;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Disaster recovery on a fresh Nextcloud installation (same major version): replace the
|
||||||
|
* database, restore the data directory and merge the backed-up config.php, see PLAN.md 3.7.
|
||||||
|
*
|
||||||
|
* Deliberately NOT handled yet (phase 3): restoring in place over a live, already-used
|
||||||
|
* installation. The admin is expected to run this once, right after occ maintenance:install and
|
||||||
|
* occ backup:key:import/backup:target:add, before anyone else touches the instance.
|
||||||
|
*/
|
||||||
|
final class RestoreFullService {
|
||||||
|
/**
|
||||||
|
* Config keys that must keep pointing at THIS installation's actual, already-working
|
||||||
|
* database and data directory (set by occ maintenance:install), never at the old server's.
|
||||||
|
*/
|
||||||
|
private const CONNECTION_KEYS = [
|
||||||
|
'dbtype', 'dbhost', 'dbport', 'dbname', 'dbuser', 'dbpassword', 'dbpersistent',
|
||||||
|
'dbtableprefix', 'dbdriveroptions', 'datadirectory',
|
||||||
|
];
|
||||||
|
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private RestoreService $restore,
|
||||||
|
private LeaseService $leases,
|
||||||
|
private IConfig $config,
|
||||||
|
private IDBConnection $db,
|
||||||
|
private IAppManager $appManager,
|
||||||
|
private KeyService $keys,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether this installation can receive $snapshotId: same Nextcloud major version, and every
|
||||||
|
* app that was enabled at backup time already installed here (its tables must exist before
|
||||||
|
* the database restore can write into them).
|
||||||
|
*
|
||||||
|
* @return array{ok:bool, currentMajor:string, snapshotMajor:string, missingApps:list<string>}
|
||||||
|
*/
|
||||||
|
public function checkCompatibility(Target $target, string $snapshotId): array {
|
||||||
|
$meta = $this->targets->repository($target)->snapshotMeta($snapshotId);
|
||||||
|
$currentMajor = explode('.', $this->config->getSystemValueString('version'))[0];
|
||||||
|
$snapshotMajor = explode('.', (string)($meta['nextcloud'] ?? ''))[0];
|
||||||
|
$enabled = array_flip($this->appManager->getEnabledApps());
|
||||||
|
$missingApps = array_values(array_diff(array_keys($meta['apps'] ?? []), array_keys($enabled)));
|
||||||
|
return [
|
||||||
|
'ok' => $currentMajor === $snapshotMajor && $missingApps === [],
|
||||||
|
'currentMajor' => $currentMajor,
|
||||||
|
'snapshotMajor' => $snapshotMajor,
|
||||||
|
'missingApps' => $missingApps,
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array{dbReport:array, dataFiles:int, configKeys:int} */
|
||||||
|
public function restoreFull(Target $target, string $snapshotId): array {
|
||||||
|
$check = $this->checkCompatibility($target, $snapshotId);
|
||||||
|
if (!$check['ok']) {
|
||||||
|
throw new \RuntimeException($this->describeBlockers($check));
|
||||||
|
}
|
||||||
|
return $this->withSharedLock($target, function () use ($target, $snapshotId) {
|
||||||
|
$repo = $this->targets->repository($target);
|
||||||
|
$meta = $repo->snapshotMeta($snapshotId);
|
||||||
|
|
||||||
|
$dbReport = null;
|
||||||
|
if (isset($meta['db'])) {
|
||||||
|
$manifest = json_decode($repo->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
$prefix = $this->config->getSystemValueString('dbtableprefix', 'oc_');
|
||||||
|
// ng_backup's own tables (and its own rows in the shared appconfig table) must
|
||||||
|
// survive this restore untouched: they describe the backup/key/target that is
|
||||||
|
// actively performing it right now, on THIS installation, not the one that was
|
||||||
|
// backed up. Overwriting them mid-restore would pull the rug from under it.
|
||||||
|
foreach (array_keys($manifest['tables']) as $table) {
|
||||||
|
/** @var string $table */
|
||||||
|
if (str_starts_with($table, $prefix . 'ngb_')) {
|
||||||
|
unset($manifest['tables'][$table]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Sequences are keyed separately from tables; excluding a table does not exclude
|
||||||
|
// its sequence, which would otherwise rewind a retained ngb_* table's sequence
|
||||||
|
// below its current ids (PostgreSQL only -- see DbDumper::restoreSequences()).
|
||||||
|
foreach (array_keys($manifest['sequences'] ?? []) as $seq) {
|
||||||
|
/** @var string $seq */
|
||||||
|
if (str_starts_with((string)($manifest['sequences'][$seq]['table'] ?? ''), $prefix . 'ngb_')) {
|
||||||
|
unset($manifest['sequences'][$seq]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$ownAppConfig = $this->preserveOwnAppConfig();
|
||||||
|
$dumper = new DbDumper($this->db->getInner(), $prefix);
|
||||||
|
try {
|
||||||
|
$dumper->restore($repo, $manifest, $dbReport);
|
||||||
|
} finally {
|
||||||
|
$this->restoreOwnAppConfig($ownAppConfig);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read (but do not yet apply) the backed-up config.php before touching the data
|
||||||
|
// directory: once applied, 'secret' may differ from what ng_backup's own Target
|
||||||
|
// options were encrypted with (see restoreFull()'s docblock), and every further
|
||||||
|
// repository access through $this->targets would fail to decrypt them.
|
||||||
|
$backupConfig = $this->fetchBackupConfig($target, $snapshotId);
|
||||||
|
|
||||||
|
$dataDir = rtrim($this->config->getSystemValueString('datadirectory'), '/');
|
||||||
|
$dataFiles = $this->restore->restoreToDirectory($target, $snapshotId, 'data', $dataDir, stripPrefix: true);
|
||||||
|
|
||||||
|
// Last step: nothing after this may need $this->targets->repository($target) again.
|
||||||
|
if ($backupConfig !== []) {
|
||||||
|
$this->targets->reencryptOptionsAround(
|
||||||
|
fn (): mixed => $this->keys->reencryptForSecretRotation(fn (): mixed => $this->config->setSystemValues($backupConfig)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return ['dbReport' => $dbReport ?? ['skipped' => [], 'restored' => []], 'dataFiles' => $dataFiles, 'configKeys' => count($backupConfig)];
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array<string, mixed> config.php's $CONFIG from the backup, minus the connection keys */
|
||||||
|
private function fetchBackupConfig(Target $target, string $snapshotId): array {
|
||||||
|
$tmp = sys_get_temp_dir() . '/ngb-restore-config-' . bin2hex(random_bytes(6));
|
||||||
|
try {
|
||||||
|
$this->restore->restoreToDirectory($target, $snapshotId, 'config', $tmp, stripPrefix: true);
|
||||||
|
$path = $tmp . '/config.php';
|
||||||
|
if (!is_file($path)) {
|
||||||
|
return []; // nothing to merge; the live config.php stays as occ maintenance:install made it
|
||||||
|
}
|
||||||
|
$backupConfig = (static function () use ($path): array {
|
||||||
|
$CONFIG = [];
|
||||||
|
include $path;
|
||||||
|
return $CONFIG;
|
||||||
|
})();
|
||||||
|
foreach (self::CONNECTION_KEYS as $key) {
|
||||||
|
unset($backupConfig[$key]);
|
||||||
|
}
|
||||||
|
return $backupConfig;
|
||||||
|
} finally {
|
||||||
|
$this->rmrf($tmp);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return list<array<string, mixed>> raw appconfig rows (appid, configkey, configvalue, type, lazy) */
|
||||||
|
private function preserveOwnAppConfig(): array {
|
||||||
|
$qb = $this->db->getQueryBuilder();
|
||||||
|
$qb->select('*')->from('appconfig')
|
||||||
|
->where($qb->expr()->eq('appid', $qb->createNamedParameter(\OCA\NgBackup\AppInfo\Application::APP_ID)));
|
||||||
|
/** @var list<array<string, mixed>> */
|
||||||
|
return $qb->executeQuery()->fetchAll();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param list<array<string, mixed>> $rows raw appconfig rows, as returned by preserveOwnAppConfig()
|
||||||
|
*
|
||||||
|
* Runs in its own transaction (DbDumper::restore() already committed its own by the time
|
||||||
|
* this is called): not a full guarantee against a crash spanning both, but means this step
|
||||||
|
* itself cannot leave ng_backup's appconfig rows half deleted, half reinserted.
|
||||||
|
*/
|
||||||
|
private function restoreOwnAppConfig(array $rows): void {
|
||||||
|
$this->db->beginTransaction();
|
||||||
|
try {
|
||||||
|
$qb = $this->db->getQueryBuilder();
|
||||||
|
$qb->delete('appconfig')
|
||||||
|
->where($qb->expr()->eq('appid', $qb->createNamedParameter(\OCA\NgBackup\AppInfo\Application::APP_ID)));
|
||||||
|
$qb->executeStatement();
|
||||||
|
foreach ($rows as $row) {
|
||||||
|
$ins = $this->db->getQueryBuilder();
|
||||||
|
$ins->insert('appconfig')->values([
|
||||||
|
'appid' => $ins->createNamedParameter($row['appid']),
|
||||||
|
'configkey' => $ins->createNamedParameter($row['configkey']),
|
||||||
|
'configvalue' => $ins->createNamedParameter($row['configvalue']),
|
||||||
|
'type' => $ins->createNamedParameter($row['type'], \OCP\DB\QueryBuilder\IQueryBuilder::PARAM_INT),
|
||||||
|
'lazy' => $ins->createNamedParameter($row['lazy'], \OCP\DB\QueryBuilder\IQueryBuilder::PARAM_INT),
|
||||||
|
]);
|
||||||
|
$ins->executeStatement();
|
||||||
|
}
|
||||||
|
$this->db->commit();
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$this->db->rollBack();
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private function rmrf(string $dir): void {
|
||||||
|
if (!is_dir($dir)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
foreach (new \RecursiveIteratorIterator(new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS), \RecursiveIteratorIterator::CHILD_FIRST) as $f) {
|
||||||
|
$f->isDir() ? rmdir($f->getPathname()) : unlink($f->getPathname());
|
||||||
|
}
|
||||||
|
rmdir($dir);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function describeBlockers(array $check): string {
|
||||||
|
$parts = [];
|
||||||
|
if ($check['currentMajor'] !== $check['snapshotMajor']) {
|
||||||
|
$parts[] = "this installation is Nextcloud {$check['currentMajor']}, the backup is from {$check['snapshotMajor']} (same major version required)";
|
||||||
|
}
|
||||||
|
if ($check['missingApps'] !== []) {
|
||||||
|
$parts[] = 'install these apps first (their tables must exist before the database restore can use them): '
|
||||||
|
. implode(', ', array_map(fn (string $a) => "occ app:install $a", $check['missingApps']));
|
||||||
|
}
|
||||||
|
return implode('; ', $parts);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function withSharedLock(Target $target, callable $fn): mixed {
|
||||||
|
try {
|
||||||
|
// No progress from DbDumper::restore()/Repository::restore() to refresh this lease
|
||||||
|
// against; a very large instance could in theory outrun this TTL (same limitation as
|
||||||
|
// RestoreService::restoreToDirectory(), see also UserBackupService).
|
||||||
|
return $this->leases->with(PruneService::lockKey($target), LeaseService::SHARED, 7200, $fn);
|
||||||
|
} catch (LockedException) {
|
||||||
|
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is in progress; try again later');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -149,15 +149,21 @@ final class RestoreService {
|
|||||||
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
|
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Raw restore of any snapshot path to a local directory (administrators, disaster recovery). */
|
/**
|
||||||
public function restoreToDirectory(Target $target, string $snapshotId, string $prefix, string $directory): int {
|
* Raw restore of any snapshot path to a local directory (administrators, disaster recovery).
|
||||||
|
*
|
||||||
|
* @param bool $stripPrefix write into $directory itself instead of $directory/$prefix (see
|
||||||
|
* Repository::restore()); used to restore "data/" or "config/" straight into the
|
||||||
|
* real data/config directory for occ backup:restore:full.
|
||||||
|
*/
|
||||||
|
public function restoreToDirectory(Target $target, string $snapshotId, string $prefix, string $directory, bool $stripPrefix = false): int {
|
||||||
if ($prefix !== '' && !Repository::isSafePath($prefix)) {
|
if ($prefix !== '' && !Repository::isSafePath($prefix)) {
|
||||||
throw new \InvalidArgumentException('Invalid path: ' . $prefix);
|
throw new \InvalidArgumentException('Invalid path: ' . $prefix);
|
||||||
}
|
}
|
||||||
if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) {
|
if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) {
|
||||||
throw new \RuntimeException("Cannot create $directory");
|
throw new \RuntimeException("Cannot create $directory");
|
||||||
}
|
}
|
||||||
return $this->withSharedLock($target, fn () => $this->targets->repository($target)->restore($snapshotId, $directory, trim($prefix, '/') === '' ? '' : trim($prefix, '/') . '/'), 3600);
|
return $this->withSharedLock($target, fn () => $this->targets->repository($target)->restore($snapshotId, $directory, trim($prefix, '/') === '' ? '' : trim($prefix, '/') . '/', $stripPrefix), 3600);
|
||||||
}
|
}
|
||||||
|
|
||||||
private function writeFile(Repository $repo, Folder $dest, string $relPath, array $entry): void {
|
private function writeFile(Repository $repo, Folder $dest, string $relPath, array $entry): void {
|
||||||
|
|||||||
@@ -10,10 +10,13 @@ namespace OCA\NgBackup\Service;
|
|||||||
use OCA\NgBackup\Backend\ExternalStorageFactory;
|
use OCA\NgBackup\Backend\ExternalStorageFactory;
|
||||||
use OCA\NgBackup\Backend\IBackend;
|
use OCA\NgBackup\Backend\IBackend;
|
||||||
use OCA\NgBackup\Backend\StorageBackend;
|
use OCA\NgBackup\Backend\StorageBackend;
|
||||||
|
use OCA\NgBackup\Db\Snapshot;
|
||||||
|
use OCA\NgBackup\Db\SnapshotMapper;
|
||||||
use OCA\NgBackup\Db\Target;
|
use OCA\NgBackup\Db\Target;
|
||||||
use OCA\NgBackup\Db\TargetMapper;
|
use OCA\NgBackup\Db\TargetMapper;
|
||||||
use OCA\NgBackup\Repository\Repository;
|
use OCA\NgBackup\Repository\Repository;
|
||||||
use OCP\App\IAppManager;
|
use OCP\App\IAppManager;
|
||||||
|
use OCP\AppFramework\Db\DoesNotExistException;
|
||||||
use OCP\Security\ICrypto;
|
use OCP\Security\ICrypto;
|
||||||
use OCP\Server;
|
use OCP\Server;
|
||||||
|
|
||||||
@@ -30,6 +33,7 @@ final class TargetService {
|
|||||||
private IAppManager $appManager,
|
private IAppManager $appManager,
|
||||||
private \OCA\NgBackup\Db\DbIndexCache $indexCache,
|
private \OCA\NgBackup\Db\DbIndexCache $indexCache,
|
||||||
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
|
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
|
||||||
|
private SnapshotMapper $snapshots,
|
||||||
) {
|
) {
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -49,6 +53,12 @@ final class TargetService {
|
|||||||
if (!preg_match('/^[A-Za-z0-9][A-Za-z0-9 _.-]{0,63}$/', $name)) {
|
if (!preg_match('/^[A-Za-z0-9][A-Za-z0-9 _.-]{0,63}$/', $name)) {
|
||||||
throw new \InvalidArgumentException('Name: letters, digits, space, _ . - (max 64)');
|
throw new \InvalidArgumentException('Name: letters, digits, space, _ . - (max 64)');
|
||||||
}
|
}
|
||||||
|
// Without the native extension files_external falls back to the smbclient binary, which
|
||||||
|
// registers fine but cannot seek: the first backup then fails with "Seek failed". Refuse
|
||||||
|
// up front instead of letting the admin find out from a failed run.
|
||||||
|
if ($backend === 'smb' && !extension_loaded('smbclient')) {
|
||||||
|
throw new \InvalidArgumentException('The SMB backend needs the PHP smbclient extension (pecl install smbclient); the smbclient binary alone cannot seek and every backup would fail');
|
||||||
|
}
|
||||||
try {
|
try {
|
||||||
$this->mapper->findByName($name);
|
$this->mapper->findByName($name);
|
||||||
throw new \InvalidArgumentException("A target named '$name' already exists");
|
throw new \InvalidArgumentException("A target named '$name' already exists");
|
||||||
@@ -78,7 +88,62 @@ final class TargetService {
|
|||||||
$target->setBasePath(trim($basePath, '/'));
|
$target->setBasePath(trim($basePath, '/'));
|
||||||
$target->setRepositoryId($repo->id());
|
$target->setRepositoryId($repo->id());
|
||||||
$target->setCreatedAt(time());
|
$target->setCreatedAt(time());
|
||||||
return ['target' => $this->mapper->insert($target), 'created' => $created];
|
$target = $this->mapper->insert($target);
|
||||||
|
if (!$created) {
|
||||||
|
// Disaster recovery: occ backup:list reads this local cache, not the repository
|
||||||
|
// directly, so a snapshot made by another installation must be backfilled here too.
|
||||||
|
$this->adoptSnapshots($target, $repo);
|
||||||
|
}
|
||||||
|
return ['target' => $target, 'created' => $created];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A target's options are encrypted with Nextcloud's own instance secret (ICrypto), not
|
||||||
|
* ng_backup's key, so changing that secret (occ backup:restore:full merging config.php)
|
||||||
|
* would otherwise leave every target permanently undecryptable. Call $applySecretChange
|
||||||
|
* (which must be the only thing that changes the secret) through here instead: every
|
||||||
|
* target's options are decrypted first, then re-encrypted under whatever secret is active
|
||||||
|
* once $applySecretChange returns.
|
||||||
|
*/
|
||||||
|
public function reencryptOptionsAround(callable $applySecretChange): mixed {
|
||||||
|
$plain = [];
|
||||||
|
foreach ($this->mapper->findAll() as $target) {
|
||||||
|
$plain[$target->getId()] = $this->options($target);
|
||||||
|
}
|
||||||
|
$result = $applySecretChange();
|
||||||
|
foreach ($this->mapper->findAll() as $target) {
|
||||||
|
if (!isset($plain[$target->getId()])) {
|
||||||
|
continue; // added by $applySecretChange itself; nothing to re-encrypt
|
||||||
|
}
|
||||||
|
$target->setOptions($this->crypto->encrypt(json_encode($plain[$target->getId()], JSON_THROW_ON_ERROR)));
|
||||||
|
$this->mapper->update($target);
|
||||||
|
}
|
||||||
|
return $result;
|
||||||
|
}
|
||||||
|
|
||||||
|
private function adoptSnapshots(Target $target, Repository $repo): void {
|
||||||
|
foreach ($repo->listSnapshots() as $snapshotId) {
|
||||||
|
try {
|
||||||
|
$this->snapshots->findOne($target->getId(), $snapshotId);
|
||||||
|
continue; // already known locally
|
||||||
|
} catch (DoesNotExistException) {
|
||||||
|
}
|
||||||
|
// One unreadable/corrupt snapshot must not stop the rest of the location's
|
||||||
|
// snapshots from being backfilled, nor abort adding the target itself.
|
||||||
|
try {
|
||||||
|
$meta = $repo->snapshotMeta($snapshotId);
|
||||||
|
$s = new Snapshot();
|
||||||
|
$s->setTargetId($target->getId());
|
||||||
|
$s->setSnapshotId($snapshotId);
|
||||||
|
$s->setKind((string)($meta['kind'] ?? 'full'));
|
||||||
|
$s->setLabel(($meta['label'] ?? '') !== '' ? $meta['label'] : null);
|
||||||
|
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
||||||
|
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
||||||
|
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
||||||
|
$this->snapshots->insert($s);
|
||||||
|
} catch (\Throwable) {
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** @return list<Target> */
|
/** @return list<Target> */
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Db\Target;
|
||||||
|
use OCA\NgBackup\UserMigration\RepositoryExportDestination;
|
||||||
|
use OCA\NgBackup\UserMigration\UserMigrationServiceLocator;
|
||||||
|
use OCP\IUser;
|
||||||
|
use OCP\Lock\LockedException;
|
||||||
|
use OCP\UserMigration\UserMigrationException;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-user backup via user_migration: every registered migrator's export streams straight into
|
||||||
|
* the encrypted repository (no zip, no copy in the user's own storage, see
|
||||||
|
* RepositoryExportDestination). Shares blobs with the full backup; prune() keeps a user export's
|
||||||
|
* blobs referenced through its manifest under users/<uid>/.
|
||||||
|
*
|
||||||
|
* user_migration is an optional dependency (unlike files_external): without it, only this
|
||||||
|
* per-user backup/restore is unavailable, the full backup still works.
|
||||||
|
*/
|
||||||
|
final class UserBackupService {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private LeaseService $leases,
|
||||||
|
private UserMigrationServiceLocator $locator,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @throws UserMigrationException when user_migration is not enabled, or the export itself fails */
|
||||||
|
public function backupUser(Target $target, IUser $user): string {
|
||||||
|
$service = $this->locator->get();
|
||||||
|
return $this->withSharedLock($target, function () use ($target, $user, $service) {
|
||||||
|
$repo = $this->targets->repository($target);
|
||||||
|
$dest = new RepositoryExportDestination($repo, $user->getUID());
|
||||||
|
$service->export($dest, $user);
|
||||||
|
// A short exclusive lock around the catalog write only: the shared lock above
|
||||||
|
// allows another concurrent backup (full or per-user) to be reading the same
|
||||||
|
// catalog generation at the same time, and both writing the next one would let
|
||||||
|
// one silently overwrite the other's addition.
|
||||||
|
$this->withCatalogLock($target, fn () => $repo->recordUserExport($dest->manifestPath()));
|
||||||
|
return $dest->manifestPath();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private function withSharedLock(Target $target, callable $fn): mixed {
|
||||||
|
try {
|
||||||
|
// UserMigrationService::export() exposes no progress, so unlike
|
||||||
|
// RestoreService::restoreUserFiles() this lease cannot be refreshed mid-call; a very
|
||||||
|
// large account could in theory outrun this TTL. Resumable, step-based per-user
|
||||||
|
// backup (like the main BackupService run) is a possible future improvement.
|
||||||
|
return $this->leases->with(PruneService::lockKey($target), LeaseService::SHARED, 3600, $fn);
|
||||||
|
} catch (LockedException) {
|
||||||
|
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is in progress; try again later');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private function withCatalogLock(Target $target, callable $fn): mixed {
|
||||||
|
try {
|
||||||
|
return $this->leases->with(PruneService::catalogLockKey($target), LeaseService::EXCLUSIVE, 30, $fn);
|
||||||
|
} catch (LockedException) {
|
||||||
|
throw new \RuntimeException('Another catalog write for ' . $target->getName() . ' is in progress; try again');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Db\Target;
|
||||||
|
use OCA\NgBackup\UserMigration\RepositoryExportDestination;
|
||||||
|
use OCA\NgBackup\UserMigration\RepositoryImportSource;
|
||||||
|
use OCA\NgBackup\UserMigration\UserMigrationServiceLocator;
|
||||||
|
use OCP\EventDispatcher\IEventDispatcher;
|
||||||
|
use OCP\Files\Events\Node\FilesystemTornDownEvent;
|
||||||
|
use OCP\IUser;
|
||||||
|
use OCP\IUserManager;
|
||||||
|
use OCP\Lock\LockedException;
|
||||||
|
use OCP\Security\ISecureRandom;
|
||||||
|
use OCP\UserMigration\UserMigrationException;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Restoring a user from a user_migration export (see UserBackupService), never over an existing
|
||||||
|
* account:
|
||||||
|
* - replace: a safety export of the current state goes into the repository first, then the
|
||||||
|
* account is deleted and recreated fresh from the backup, under the same uid;
|
||||||
|
* - as-backup: the backup is restored into a new account "<uid>-bak" (or "-bak-2", ...) instead,
|
||||||
|
* so an admin can compare and transfer by hand. Nextcloud cannot rename a uid, so the *backup*
|
||||||
|
* gets the new name, never the existing account.
|
||||||
|
*/
|
||||||
|
final class UserRestoreService {
|
||||||
|
public const MODE_REPLACE = 'replace';
|
||||||
|
public const MODE_AS_BACKUP = 'as-backup';
|
||||||
|
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private LeaseService $leases,
|
||||||
|
private UserMigrationServiceLocator $locator,
|
||||||
|
private IUserManager $userManager,
|
||||||
|
private ISecureRandom $random,
|
||||||
|
private IEventDispatcher $events,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @throws UserMigrationException when user_migration is not enabled, or the restore itself fails */
|
||||||
|
public function restoreUser(Target $target, string $manifestPath, string $mode = self::MODE_REPLACE): IUser {
|
||||||
|
$service = $this->locator->get();
|
||||||
|
return $this->withSharedLock($target, function () use ($target, $manifestPath, $mode, $service): IUser {
|
||||||
|
$repo = $this->targets->repository($target);
|
||||||
|
// Only ever read a manifest the verified catalog actually recorded: a path alone
|
||||||
|
// (even one this app generated before) proves nothing about it still being current
|
||||||
|
// or genuine once the catalog's rollback protection is supposed to vouch for it.
|
||||||
|
if (!$repo->isRecordedUserExport($manifestPath)) {
|
||||||
|
throw new \InvalidArgumentException("$manifestPath is not a recorded user export for this location");
|
||||||
|
}
|
||||||
|
$source = new RepositoryImportSource($repo, $manifestPath);
|
||||||
|
$originalUid = $source->getOriginalUid();
|
||||||
|
|
||||||
|
if ($mode === self::MODE_AS_BACKUP) {
|
||||||
|
$backupUid = $this->freeBackupUid($originalUid);
|
||||||
|
$user = $this->userManager->createUser($backupUid, $this->random->generate(32));
|
||||||
|
if ($user === false) {
|
||||||
|
throw new \RuntimeException("Could not create the account $backupUid");
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return $service->import($source, $user);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
// A partially-imported, never-used account left behind is strictly worse
|
||||||
|
// than no account at all; the original source account was never touched.
|
||||||
|
$user->delete();
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($mode !== self::MODE_REPLACE) {
|
||||||
|
throw new \InvalidArgumentException("Unknown restore mode: $mode");
|
||||||
|
}
|
||||||
|
$existing = $this->userManager->get($originalUid);
|
||||||
|
$safetyManifestPath = null;
|
||||||
|
if ($existing !== null) {
|
||||||
|
// Never import over a live account: export its current state first, so nothing
|
||||||
|
// is lost if the restore turns out to be the wrong choice.
|
||||||
|
$safety = new RepositoryExportDestination($repo, $originalUid);
|
||||||
|
$service->export($safety, $existing);
|
||||||
|
$safetyManifestPath = $safety->manifestPath();
|
||||||
|
// Same race as UserBackupService::backupUser(): serialize the catalog write
|
||||||
|
// itself against any other concurrent backup/restore for this target.
|
||||||
|
$this->withCatalogLock($target, fn () => $repo->recordUserExport($safetyManifestPath));
|
||||||
|
if (!$existing->delete()) {
|
||||||
|
throw new \RuntimeException("Could not delete the existing account $originalUid before restoring; its safety export is at $safetyManifestPath");
|
||||||
|
}
|
||||||
|
// IRootFolder caches a Folder per uid for the lifetime of this process; without
|
||||||
|
// this it would still hand the just-deleted account's (now storage-less) folder
|
||||||
|
// to the import below, and every write would fail.
|
||||||
|
$this->events->dispatchTyped(new FilesystemTornDownEvent());
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return $service->import($source, null);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
// The original account is already gone at this point (or never existed); a
|
||||||
|
// failure here can only be reported, not undone in place. Its safety export
|
||||||
|
// (if any) is the recovery path, already recorded above.
|
||||||
|
throw new UserMigrationException(
|
||||||
|
"Could not restore $originalUid" . ($safetyManifestPath !== null ? "; its safety export is at $safetyManifestPath" : '') . ': ' . $e->getMessage(),
|
||||||
|
0,
|
||||||
|
$e,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private function freeBackupUid(string $uid): string {
|
||||||
|
$candidate = $uid . '-bak';
|
||||||
|
for ($n = 2; $this->userManager->userExists($candidate); $n++) {
|
||||||
|
$candidate = $uid . '-bak-' . $n;
|
||||||
|
}
|
||||||
|
return $candidate;
|
||||||
|
}
|
||||||
|
|
||||||
|
private function withSharedLock(Target $target, callable $fn): mixed {
|
||||||
|
try {
|
||||||
|
// See UserBackupService::withSharedLock(): UserMigrationService exposes no progress,
|
||||||
|
// so this lease cannot be refreshed mid-call.
|
||||||
|
return $this->leases->with(PruneService::lockKey($target), LeaseService::SHARED, 3600, $fn);
|
||||||
|
} catch (LockedException) {
|
||||||
|
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is in progress; try again later');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private function withCatalogLock(Target $target, callable $fn): mixed {
|
||||||
|
try {
|
||||||
|
return $this->leases->with(PruneService::catalogLockKey($target), LeaseService::EXCLUSIVE, 30, $fn);
|
||||||
|
} catch (LockedException) {
|
||||||
|
throw new \RuntimeException('Another catalog write for ' . $target->getName() . ' is in progress; try again');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Db\Target;
|
||||||
|
use OCP\Lock\LockedException;
|
||||||
|
|
||||||
|
/** Checksum audit of a snapshot (occ backup:verify), see Repository::verify(). */
|
||||||
|
final class VerifyService {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private LeaseService $leases,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list<string>, failed:list<string>} */
|
||||||
|
public function verify(Target $target, string $snapshotId, bool $deep = false): array {
|
||||||
|
return $this->run($target, $deep, fn (\OCA\NgBackup\Repository\Repository $repo, callable $heartbeat): array => $repo->verify($snapshotId, $deep, $heartbeat));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list<string>, failed:list<string>} */
|
||||||
|
public function verifyUserExport(Target $target, string $manifestPath, bool $deep = false): array {
|
||||||
|
return $this->run($target, $deep, fn (\OCA\NgBackup\Repository\Repository $repo, callable $heartbeat): array => $repo->verifyUserExport($manifestPath, $deep, $heartbeat));
|
||||||
|
}
|
||||||
|
|
||||||
|
private function run(Target $target, bool $deep, callable $fn): array {
|
||||||
|
$ttl = $deep ? 600 : 300;
|
||||||
|
try {
|
||||||
|
return $this->leases->with(PruneService::lockKey($target), LeaseService::SHARED, $ttl, function (callable $refresh) use ($target, $deep, $ttl, $fn): array {
|
||||||
|
$last = time();
|
||||||
|
$heartbeat = function () use ($refresh, $ttl, &$last): void {
|
||||||
|
if (time() - $last >= intdiv($ttl, 2)) {
|
||||||
|
$refresh();
|
||||||
|
$last = time();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
return $fn($this->targets->repository($target), $heartbeat);
|
||||||
|
});
|
||||||
|
} catch (LockedException) {
|
||||||
|
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' is in progress; try again later');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -65,11 +65,14 @@ final class RepositoryExportDestination implements IExportDestination {
|
|||||||
if ($stream === false) {
|
if ($stream === false) {
|
||||||
throw new \RuntimeException('Cannot read ' . $node->getPath());
|
throw new \RuntimeException('Cannot read ' . $node->getPath());
|
||||||
}
|
}
|
||||||
|
try {
|
||||||
$this->addFileAsStream($target, $stream);
|
$this->addFileAsStream($target, $stream);
|
||||||
|
} finally {
|
||||||
fclose($stream);
|
fclose($stream);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public function setMigratorVersions(array $versions): void {
|
public function setMigratorVersions(array $versions): void {
|
||||||
$this->versions = $versions;
|
$this->versions = $versions;
|
||||||
|
|||||||
@@ -109,11 +109,18 @@ final class RepositoryImportSource implements IImportSource {
|
|||||||
$destination->newFile($name, $stream);
|
$destination->newFile($name, $stream);
|
||||||
}
|
}
|
||||||
} finally {
|
} finally {
|
||||||
|
// putContent()/newFile() already close the stream once they've consumed it:
|
||||||
|
// despite the @return resource docblock above, $stream is no longer a
|
||||||
|
// resource by the time we get here (verified against a real storage; a
|
||||||
|
// double fclose() throws a TypeError on PHP 8).
|
||||||
|
/** @psalm-suppress RedundantConditionGivenDocblockType */
|
||||||
|
if (is_resource($stream)) {
|
||||||
fclose($stream);
|
fclose($stream);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public function getMigratorVersions(): array {
|
public function getMigratorVersions(): array {
|
||||||
return $this->manifest['versions'];
|
return $this->manifest['versions'];
|
||||||
|
|||||||
@@ -0,0 +1,35 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\UserMigration;
|
||||||
|
|
||||||
|
use OCP\App\IAppManager;
|
||||||
|
use OCP\Server;
|
||||||
|
use OCP\UserMigration\UserMigrationException;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolves user_migration's own (internal, non-public) UserMigrationService on demand.
|
||||||
|
*
|
||||||
|
* Unlike files_external, user_migration is an optional dependency: without it, only per-user
|
||||||
|
* backup/restore is unavailable, the full backup still works. The class is never autoloaded by
|
||||||
|
* composer; everything that touches it is behind this one class.
|
||||||
|
*/
|
||||||
|
final class UserMigrationServiceLocator {
|
||||||
|
public function __construct(
|
||||||
|
private IAppManager $appManager,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @throws UserMigrationException when user_migration is not enabled anywhere */
|
||||||
|
public function get(): object {
|
||||||
|
if (!$this->appManager->isEnabledForAnyone('user_migration')) {
|
||||||
|
throw new UserMigrationException('The user_migration app is not enabled; per-user backup/restore is unavailable');
|
||||||
|
}
|
||||||
|
$this->appManager->loadApp('user_migration');
|
||||||
|
return Server::get(\OCA\UserMigration\Service\UserMigrationService::class);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -49,6 +49,11 @@
|
|||||||
<code><![CDATA[ask]]></code>
|
<code><![CDATA[ask]]></code>
|
||||||
</UndefinedInterfaceMethod>
|
</UndefinedInterfaceMethod>
|
||||||
</file>
|
</file>
|
||||||
|
<file src="lib/Command/KeyImport.php">
|
||||||
|
<UndefinedInterfaceMethod>
|
||||||
|
<code><![CDATA[ask]]></code>
|
||||||
|
</UndefinedInterfaceMethod>
|
||||||
|
</file>
|
||||||
<file src="lib/Command/KeyInit.php">
|
<file src="lib/Command/KeyInit.php">
|
||||||
<UndefinedInterfaceMethod>
|
<UndefinedInterfaceMethod>
|
||||||
<code><![CDATA[ask]]></code>
|
<code><![CDATA[ask]]></code>
|
||||||
@@ -455,6 +460,17 @@
|
|||||||
<code><![CDATA[array{kept: array<string, list<string>>, forgotten: list<string>, purged: list<string>, prune: array}]]></code>
|
<code><![CDATA[array{kept: array<string, list<string>>, forgotten: list<string>, purged: list<string>, prune: array}]]></code>
|
||||||
</MoreSpecificReturnType>
|
</MoreSpecificReturnType>
|
||||||
</file>
|
</file>
|
||||||
|
<file src="lib/Service/RestoreFullService.php">
|
||||||
|
<EmptyArrayAccess>
|
||||||
|
<code><![CDATA[$backupConfig[$key]]]></code>
|
||||||
|
</EmptyArrayAccess>
|
||||||
|
<UndefinedInterfaceMethod>
|
||||||
|
<code><![CDATA[getInner]]></code>
|
||||||
|
</UndefinedInterfaceMethod>
|
||||||
|
<UnresolvableInclude>
|
||||||
|
<code><![CDATA[include $path]]></code>
|
||||||
|
</UnresolvableInclude>
|
||||||
|
</file>
|
||||||
<file src="lib/Service/RestoreService.php">
|
<file src="lib/Service/RestoreService.php">
|
||||||
<LessSpecificReturnStatement>
|
<LessSpecificReturnStatement>
|
||||||
<code><![CDATA[array_values($children)]]></code>
|
<code><![CDATA[array_values($children)]]></code>
|
||||||
|
|||||||
@@ -31,8 +31,17 @@
|
|||||||
<errorLevel type="suppress">
|
<errorLevel type="suppress">
|
||||||
<!-- Provided by files_external at runtime; not a composer dependency (PLAN.md 3.5). -->
|
<!-- Provided by files_external at runtime; not a composer dependency (PLAN.md 3.5). -->
|
||||||
<referencedClass name="OCA\Files_External\**" />
|
<referencedClass name="OCA\Files_External\**" />
|
||||||
|
<!-- Provided by user_migration at runtime, an optional dependency (PLAN.md 3.8). -->
|
||||||
|
<referencedClass name="OCA\UserMigration\**" />
|
||||||
</errorLevel>
|
</errorLevel>
|
||||||
</UndefinedClass>
|
</UndefinedClass>
|
||||||
|
<UndefinedDocblockClass>
|
||||||
|
<errorLevel type="suppress">
|
||||||
|
<!-- Server::get()'s templated return type resolving the users_migration class-string
|
||||||
|
constant below; same reason as the UndefinedClass suppression above. -->
|
||||||
|
<referencedClass name="OCA\UserMigration\**" />
|
||||||
|
</errorLevel>
|
||||||
|
</UndefinedDocblockClass>
|
||||||
<!-- The #[\Override] attribute convention (PHP 8.3+) adds no behaviour; this app supports
|
<!-- The #[\Override] attribute convention (PHP 8.3+) adds no behaviour; this app supports
|
||||||
PHP 8.1+, and applying it to every interface-implementing method project-wide is pure
|
PHP 8.1+, and applying it to every interface-implementing method project-wide is pure
|
||||||
ceremony with no bug-catching value, so it's suppressed rather than papered over. -->
|
ceremony with no bug-catching value, so it's suppressed rather than papered over. -->
|
||||||
|
|||||||
@@ -0,0 +1,109 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Repository\ICatalogAnchor;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Repository\RepositoryException;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Repository::recordUserExport/listUserExports/forgetUserExport: the catalog side of a
|
||||||
|
* user_migration export (see RepositoryExportDestination/RepositoryImportSource, which this
|
||||||
|
* does not exercise directly since they implement OCP\UserMigration interfaces not available to
|
||||||
|
* plain PHPUnit; that round trip is covered by tests/integration/user-roundtrip.sh instead).
|
||||||
|
*/
|
||||||
|
class UserExportTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private LocalBackend $backend;
|
||||||
|
private KeyRing $keys;
|
||||||
|
private ICatalogAnchor $anchor;
|
||||||
|
private string $root;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->root = $this->tempDir();
|
||||||
|
$this->backend = new LocalBackend($this->root);
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
$this->anchor = new class implements ICatalogAnchor {
|
||||||
|
public array $data = [];
|
||||||
|
public function get(string $r): ?array { return $this->data[$r] ?? null; }
|
||||||
|
public function set(string $r, int $g, string $h): void { $this->data[$r] = ['gen' => $g, 'hash' => $h]; }
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private function open(): Repository {
|
||||||
|
return Repository::openWithKey($this->backend, $this->keys, null, $this->anchor);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Stands in for an export destination's close(): a manifest object plus a catalog record. */
|
||||||
|
private function putManifest(Repository $repo, string $path): void {
|
||||||
|
$repo->putObject($path, json_encode(['uid' => 'alice']));
|
||||||
|
$repo->recordUserExport($path);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRecordAndListSurviveAReopen(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
|
||||||
|
$again = $this->open();
|
||||||
|
$this->assertSame(['users/alice/20261005T120000Z-aaaa0001'], $again->listUserExports());
|
||||||
|
$this->assertSame(['users/alice/20261005T120000Z-aaaa0001'], $again->listUserExports('alice'));
|
||||||
|
$this->assertSame([], $again->listUserExports('bob'));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testListIsNewestFirst(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
$this->putManifest($repo, 'users/alice/20261006T120000Z-aaaa0002');
|
||||||
|
|
||||||
|
$this->assertSame(
|
||||||
|
['users/alice/20261006T120000Z-aaaa0002', 'users/alice/20261005T120000Z-aaaa0001'],
|
||||||
|
$repo->listUserExports('alice'),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testForgetRemovesItFromTheCatalogAndLocation(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
|
||||||
|
$repo->forgetUserExport($path);
|
||||||
|
|
||||||
|
$this->assertSame([], $repo->listUserExports());
|
||||||
|
$this->assertFalse($this->backend->exists($path));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testForgetRejectsAPathNotInTheCatalog(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$repo->forgetUserExport('users/nobody/not-a-real-export');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testIsRecordedUserExport(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
|
||||||
|
$this->assertTrue($repo->isRecordedUserExport($path));
|
||||||
|
$this->assertFalse($repo->isRecordedUserExport('users/alice/not-the-real-one'));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testListingDetectsAManifestRemovedBehindItsBack(): void {
|
||||||
|
// A catalog-listed export whose object disappeared from the location (deleted or
|
||||||
|
// hidden outside NG Backup) must be detected, the same as a missing snapshot.
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
$this->backend->delete($path);
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage('missing on the location');
|
||||||
|
$repo->listUserExports();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
class VerifyTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private LocalBackend $backend;
|
||||||
|
private KeyRing $keys;
|
||||||
|
private string $root;
|
||||||
|
private string $src;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->root = $this->tempDir();
|
||||||
|
$this->backend = new LocalBackend($this->root);
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
$this->src = $this->tempDir();
|
||||||
|
file_put_contents("$this->src/a", random_bytes(100000));
|
||||||
|
file_put_contents("$this->src/b", random_bytes(100000));
|
||||||
|
}
|
||||||
|
|
||||||
|
private function open(): Repository {
|
||||||
|
return Repository::openWithKey($this->backend, $this->keys, null, null);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testShallowVerifyPassesOnAHealthyRepository(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$s = $repo->backupDirectory($this->src);
|
||||||
|
|
||||||
|
$result = $this->open()->verify($s['snapshot']);
|
||||||
|
|
||||||
|
$this->assertSame(2, $result['filesChecked']);
|
||||||
|
$this->assertGreaterThanOrEqual(1, $result['blobsChecked']);
|
||||||
|
$this->assertSame(0, $result['bytesChecked']); // not downloaded without --deep
|
||||||
|
$this->assertSame([], $result['missing']);
|
||||||
|
$this->assertSame([], $result['failed']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testDeepVerifyDownloadsAndCountsBytes(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$s = $repo->backupDirectory($this->src);
|
||||||
|
|
||||||
|
$result = $this->open()->verify($s['snapshot'], deep: true);
|
||||||
|
|
||||||
|
$this->assertSame(200000, $result['bytesChecked']);
|
||||||
|
$this->assertSame([], $result['missing']);
|
||||||
|
$this->assertSame([], $result['failed']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testShallowVerifyDetectsAMissingPack(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$s = $repo->backupDirectory($this->src);
|
||||||
|
foreach ($this->backend->list('packs') as $pack) {
|
||||||
|
$this->backend->delete($pack);
|
||||||
|
}
|
||||||
|
|
||||||
|
$result = $this->open()->verify($s['snapshot']);
|
||||||
|
|
||||||
|
$this->assertNotSame([], $result['missing']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testVerifyFailsOnAMissingDatabaseManifest(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$repo->writeSnapshot('snap-no-db', ['time' => gmdate('c'), 'db' => 'db/does-not-exist'], []);
|
||||||
|
|
||||||
|
$result = $this->open()->verify('snap-no-db');
|
||||||
|
|
||||||
|
$this->assertSame(['db/does-not-exist'], $result['missing']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testDeepVerifyDetectsATamperedPack(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$s = $repo->backupDirectory($this->src);
|
||||||
|
$packs = $this->backend->list('packs');
|
||||||
|
$this->assertNotEmpty($packs);
|
||||||
|
$path = $this->root . '/' . $packs[0];
|
||||||
|
$bytes = file_get_contents($path);
|
||||||
|
// Flip a byte well inside the file, away from any header, so decryption's own
|
||||||
|
// authentication tag catches it (this must fail loudly, not just "do nothing").
|
||||||
|
$bytes[(int)(strlen($bytes) / 2)] = chr(~ord($bytes[(int)(strlen($bytes) / 2)]) & 0xFF);
|
||||||
|
file_put_contents($path, $bytes);
|
||||||
|
|
||||||
|
$result = $this->open()->verify($s['snapshot'], deep: true);
|
||||||
|
|
||||||
|
$this->assertNotSame([], $result['failed']);
|
||||||
|
}
|
||||||
|
}
|
||||||
Executable
+86
@@ -0,0 +1,86 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Disaster recovery: occ backup:key:import + backup:target:add + backup:restore:full on a
|
||||||
|
# SECOND, fresh Nextcloud instance that never saw the first one, sharing only the backup
|
||||||
|
# location (a local directory, bind-mounted into both containers) -- see PLAN.md 3.7.
|
||||||
|
# Usage: full-restore-roundtrip.sh <nextcloud-image-tag>
|
||||||
|
# e.g. full-restore-roundtrip.sh 35-apache
|
||||||
|
set -euo pipefail
|
||||||
|
IMAGE="nextcloud:${1:-35-apache}"
|
||||||
|
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
||||||
|
NET=ngb-frt-net
|
||||||
|
SHARED=$(mktemp -d)
|
||||||
|
chmod 777 "$SHARED"
|
||||||
|
fail() { echo "FAIL: $*"; exit 1; }
|
||||||
|
ok() { echo "ok $*"; }
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
docker rm -fv ngb-frt-origin ngb-frt-restore >/dev/null 2>&1 || true
|
||||||
|
docker network rm "$NET" >/dev/null 2>&1 || true
|
||||||
|
docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
|
||||||
|
rmdir "$SHARED" 2>/dev/null || true
|
||||||
|
rm -f /tmp/ngb-frt-kit.json /tmp/ngb-frt-pass
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
|
||||||
|
|
||||||
|
docker network create "$NET" >/dev/null
|
||||||
|
docker run -d --name ngb-frt-origin --network "$NET" -e SQLITE_DATABASE=nextcloud \
|
||||||
|
-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin \
|
||||||
|
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
|
||||||
|
for i in $(seq 1 90); do run_on ngb-frt-origin status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
|
||||||
|
docker exec ngb-frt-origin chown www-data:www-data /var/www/html/custom_apps
|
||||||
|
run_on ngb-frt-origin app:enable ng_backup >/dev/null
|
||||||
|
ok "origin instance up"
|
||||||
|
|
||||||
|
docker exec ngb-frt-origin sh -c "printf 'full restore roundtrip passphrase 2026\n' > /tmp/pass"
|
||||||
|
run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI >/dev/null || fail "key init"
|
||||||
|
run_on ngb-frt-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||||
|
CODE=$(run_on ngb-frt-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
run_on ngb-frt-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||||
|
run_on ngb-frt-origin backup:target:add offsite local -a null::null -o datadir=/mnt/shared --path=repo >/dev/null || fail "target add"
|
||||||
|
ok "key, kit and location on the origin"
|
||||||
|
|
||||||
|
NC_PASS=$(openssl rand -hex 16)
|
||||||
|
docker exec -u www-data -e OC_PASS="$NC_PASS" ngb-frt-origin php occ user:add --password-from-env frtest >/dev/null
|
||||||
|
docker exec ngb-frt-origin sh -c "mkdir -p /var/www/html/data/frtest/files/Docs && head -c 200000 /dev/urandom > /var/www/html/data/frtest/files/Docs/report.bin && echo 'hello disaster recovery' > /var/www/html/data/frtest/files/Docs/note.txt && chown -R www-data:www-data /var/www/html/data/frtest"
|
||||||
|
run_on ngb-frt-origin files:scan frtest >/dev/null
|
||||||
|
RUN_OUT=$(run_on ngb-frt-origin backup:run offsite -l disaster-test)
|
||||||
|
SNAP=$(printf '%s\n' "$RUN_OUT" | head -1 | awk -F'snapshot ' '{print $2}' | tr -d ',')
|
||||||
|
[ -n "$SNAP" ] || fail "no snapshot id from backup:run"
|
||||||
|
ok "test data backed up, snapshot $SNAP"
|
||||||
|
|
||||||
|
docker cp ngb-frt-origin:/tmp/kit.json /tmp/ngb-frt-kit.json
|
||||||
|
docker exec ngb-frt-origin sh -c "cat /tmp/pass" > /tmp/ngb-frt-pass
|
||||||
|
docker run -d --name ngb-frt-restore --network "$NET" -e SQLITE_DATABASE=nextcloud \
|
||||||
|
-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin \
|
||||||
|
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
|
||||||
|
for i in $(seq 1 90); do run_on ngb-frt-restore status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
|
||||||
|
docker exec ngb-frt-restore chown www-data:www-data /var/www/html/custom_apps
|
||||||
|
run_on ngb-frt-restore app:enable ng_backup >/dev/null
|
||||||
|
docker cp /tmp/ngb-frt-kit.json ngb-frt-restore:/tmp/kit.json
|
||||||
|
docker exec ngb-frt-restore chmod 644 /tmp/kit.json
|
||||||
|
docker cp /tmp/ngb-frt-pass ngb-frt-restore:/tmp/pass
|
||||||
|
ok "fresh restore instance up"
|
||||||
|
|
||||||
|
run_on ngb-frt-restore backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "key import"
|
||||||
|
run_on ngb-frt-restore backup:target:add offsite local -a null::null -o datadir=/mnt/shared --path=repo >/dev/null || fail "target add on the restore instance"
|
||||||
|
run_on ngb-frt-restore backup:list offsite | grep -q "$SNAP" || fail "the origin's snapshot is not listed after connecting the location"
|
||||||
|
ok "recovery kit imported, existing repository adopted, snapshot visible"
|
||||||
|
|
||||||
|
run_on ngb-frt-restore backup:restore:full offsite "$SNAP" >/dev/null || fail "restore:full"
|
||||||
|
ok "backup:restore:full completed"
|
||||||
|
|
||||||
|
# Everything below must work in a FRESH occ process (a new PHP process per docker exec), since
|
||||||
|
# earlier bugs here only showed up after config.php's secret was rewritten mid-restore.
|
||||||
|
run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after restore (own key/target undecryptable?)"
|
||||||
|
run_on ngb-frt-restore user:list | grep -q frtest || fail "the backed-up user is missing from the restored database"
|
||||||
|
docker exec ngb-frt-restore test -f /var/www/html/data/frtest/files/Docs/note.txt || fail "restored file is missing"
|
||||||
|
[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt)" = "hello disaster recovery" ] || fail "restored file content differs"
|
||||||
|
run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after restore"
|
||||||
|
ok "database, files and ng_backup's own state are all intact and usable after the restore"
|
||||||
|
|
||||||
|
run_on ngb-frt-restore backup:run offsite -l post-restore-check >/dev/null || fail "a new backup after the restore fails"
|
||||||
|
ok "the restored instance can run a new backup of its own"
|
||||||
|
|
||||||
|
echo "FULL RESTORE ROUNDTRIP PASS"
|
||||||
@@ -49,6 +49,11 @@ SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xa
|
|||||||
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
||||||
ok "restore new-folder and replace: files identical, extra file in the trash"
|
ok "restore new-folder and replace: files identical, extra file in the trash"
|
||||||
|
|
||||||
|
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
|
||||||
|
occ backup:verify ci "$SNAP" | grep -q "^OK$" || fail "shallow verify"
|
||||||
|
occ backup:verify ci "$SNAP" --deep | grep -q "^OK$" || fail "deep verify"
|
||||||
|
ok "shallow and deep checksum audit"
|
||||||
|
|
||||||
# Database dump/restore round trip (all tables, checksums, sequences on PostgreSQL).
|
# Database dump/restore round trip (all tables, checksums, sequences on PostgreSQL).
|
||||||
$X php -d memory_limit=512M "$APPS/ng_backup/spikes/03-db-dump.php" /tmp/ngb-rt-db | grep "ALL PASS" >/dev/null || fail "database round trip"
|
$X php -d memory_limit=512M "$APPS/ng_backup/spikes/03-db-dump.php" /tmp/ngb-rt-db | grep "ALL PASS" >/dev/null || fail "database round trip"
|
||||||
ok "database dump and restore"
|
ok "database dump and restore"
|
||||||
|
|||||||
@@ -1,17 +1,25 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Round trip of NG Backup against real S3 (s3mock) and SFTP backends, through files_external's own
|
# Round trip of NG Backup against real S3 (s3mock), SFTP, FTP and SMB backends, through
|
||||||
# backend/auth classes, same as a production location would use. Backend correctness does not
|
# files_external's own backend/auth classes, same as a production location would use. Backend
|
||||||
# depend on the Nextcloud version or database, so this runs once (not matrixed across NC/DB),
|
# correctness does not depend on the Nextcloud version or database, so this runs once (not
|
||||||
# on a recent Nextcloud with sqlite. Each backend gets its own fresh Nextcloud container, since
|
# matrixed across NC/DB), on a recent Nextcloud with sqlite. Each backend gets its own fresh
|
||||||
# NG Backup's key can only be initialised once per instance.
|
# Nextcloud container, since NG Backup's key can only be initialised once per instance.
|
||||||
# Usage: tests/integration/targets-matrix.sh [nextcloud-version]
|
# Usage: tests/integration/targets-matrix.sh [nextcloud-version]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
V=${1:-35}
|
V=${1:-35}
|
||||||
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
||||||
NET=ngb-targets-net
|
NET=ngb-targets-net
|
||||||
|
SMB_IMAGE=ngb-targets-smb-nc
|
||||||
|
SMB_BUILD_DIR=$(mktemp -d)
|
||||||
RESULTS=()
|
RESULTS=()
|
||||||
|
|
||||||
cleanup() { docker rm -fv ngb-targets-nc-s3 ngb-targets-nc-sftp ngb-targets-s3 ngb-targets-sftp >/dev/null 2>&1; docker network rm "$NET" >/dev/null 2>&1; }
|
cleanup() {
|
||||||
|
docker rm -fv ngb-targets-nc-s3 ngb-targets-nc-sftp ngb-targets-nc-ftp ngb-targets-nc-smb \
|
||||||
|
ngb-targets-s3 ngb-targets-sftp ngb-targets-ftp ngb-targets-smb >/dev/null 2>&1 || true
|
||||||
|
docker rmi "$SMB_IMAGE" >/dev/null 2>&1 || true
|
||||||
|
docker network rm "$NET" >/dev/null 2>&1 || true
|
||||||
|
rm -rf "$SMB_BUILD_DIR"
|
||||||
|
}
|
||||||
trap cleanup EXIT
|
trap cleanup EXIT
|
||||||
|
|
||||||
wait_tcp() { # host port
|
wait_tcp() { # host port
|
||||||
@@ -19,11 +27,11 @@ wait_tcp() { # host port
|
|||||||
|| { echo "FAIL: $1:$2 never came up"; exit 1; }
|
|| { echo "FAIL: $1:$2 never came up"; exit 1; }
|
||||||
}
|
}
|
||||||
|
|
||||||
start_nc() { # container-name
|
start_nc() { # container-name [image]
|
||||||
docker run -d --name "$1" --network "$NET" \
|
docker run -d --name "$1" --network "$NET" \
|
||||||
-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin-targets-pw -e NEXTCLOUD_TRUSTED_DOMAINS=localhost \
|
-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin-targets-pw -e NEXTCLOUD_TRUSTED_DOMAINS=localhost \
|
||||||
-e SQLITE_DATABASE=nextcloud \
|
-e SQLITE_DATABASE=nextcloud \
|
||||||
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" nextcloud:$V-apache >/dev/null
|
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" "${2:-nextcloud:$V-apache}" >/dev/null
|
||||||
for i in $(seq 1 90); do docker exec -u www-data "$1" php occ status 2>/dev/null | grep -q "installed: true" && break; sleep 4; done
|
for i in $(seq 1 90); do docker exec -u www-data "$1" php occ status 2>/dev/null | grep -q "installed: true" && break; sleep 4; done
|
||||||
docker exec "$1" chown www-data:www-data /var/www/html/custom_apps
|
docker exec "$1" chown www-data:www-data /var/www/html/custom_apps
|
||||||
echo "=== NC $V / sqlite ($1): $(docker exec -u www-data "$1" php occ -V 2>/dev/null)"
|
echo "=== NC $V / sqlite ($1): $(docker exec -u www-data "$1" php occ -V 2>/dev/null)"
|
||||||
@@ -51,8 +59,37 @@ docker run -d --name ngb-targets-s3 --network "$NET" \
|
|||||||
docker run -d --name ngb-targets-sftp --network "$NET" \
|
docker run -d --name ngb-targets-sftp --network "$NET" \
|
||||||
atmoz/sftp ngbackup:ngbackuppw:::upload >/dev/null
|
atmoz/sftp ngbackup:ngbackuppw:::upload >/dev/null
|
||||||
|
|
||||||
|
# stilliard/pure-ftpd: delfer/alpine-ftp-server (vsftpd) was tried first and rejects the absolute
|
||||||
|
# paths files_external's FTP backend always builds, inside this server's chroot specifically --
|
||||||
|
# not a files_external bug (confirmed with plain php ftp_mkdir()), but pure-ftpd sidesteps it.
|
||||||
|
docker run -d --name ngb-targets-ftp --network "$NET" \
|
||||||
|
-e "FTP_USER_NAME=ngbackup" -e "FTP_USER_PASS=ngbackuppw" -e "FTP_USER_HOME=/home/ngbackup" \
|
||||||
|
-e "PUBLICHOST=ngb-targets-ftp" \
|
||||||
|
stilliard/pure-ftpd >/dev/null
|
||||||
|
|
||||||
|
# dperson/samba. The *native* smbclient PECL extension is required: files_external's shell-based
|
||||||
|
# fallback (just the smbclient CLI binary) still registers this backend, but its stream cannot
|
||||||
|
# seek, and ng_backup reads blobs back by byte range while backing up -- confirmed live, that
|
||||||
|
# fallback fails the very first (small) backup with "Seek failed in packs/...". The extension is
|
||||||
|
# not in the stock nextcloud image (needs libsmbclient-dev + a compile), so this backend gets its
|
||||||
|
# own small image, built once here, in its own directory (never /tmp directly: that would send
|
||||||
|
# every unrelated file under /tmp to the docker daemon as build context).
|
||||||
|
cat > "$SMB_BUILD_DIR/Dockerfile" <<EOF
|
||||||
|
FROM nextcloud:$V-apache
|
||||||
|
RUN apt-get update -qq && apt-get install -y -qq libsmbclient-dev >/dev/null \\
|
||||||
|
&& pecl install smbclient >/dev/null 2>&1 \\
|
||||||
|
&& docker-php-ext-enable smbclient \\
|
||||||
|
&& apt-get clean
|
||||||
|
EOF
|
||||||
|
docker build -q -t "$SMB_IMAGE" "$SMB_BUILD_DIR" >/dev/null
|
||||||
|
docker run -d --name ngb-targets-smb --network "$NET" \
|
||||||
|
-e USERID=1000 -e GROUPID=1000 \
|
||||||
|
dperson/samba -p -u "ngbackup;ngbackuppw" -s "backup;/share;yes;no;no;ngbackup" >/dev/null
|
||||||
|
|
||||||
wait_tcp ngb-targets-s3 9090
|
wait_tcp ngb-targets-s3 9090
|
||||||
wait_tcp ngb-targets-sftp 22
|
wait_tcp ngb-targets-sftp 22
|
||||||
|
wait_tcp ngb-targets-ftp 21
|
||||||
|
wait_tcp ngb-targets-smb 445
|
||||||
|
|
||||||
start_nc ngb-targets-nc-s3
|
start_nc ngb-targets-nc-s3
|
||||||
run "S3 (s3mock)" ngb-targets-nc-s3 "amazons3 -a amazons3::accesskey -o bucket=ngbackups -o hostname=ngb-targets-s3 -o port=9090 -o region=us-east-1 -o use_ssl=false -o use_path_style=true -o key=test -o secret=test"
|
run "S3 (s3mock)" ngb-targets-nc-s3 "amazons3 -a amazons3::accesskey -o bucket=ngbackups -o hostname=ngb-targets-s3 -o port=9090 -o region=us-east-1 -o use_ssl=false -o use_path_style=true -o key=test -o secret=test"
|
||||||
@@ -60,6 +97,12 @@ run "S3 (s3mock)" ngb-targets-nc-s3 "amazons3 -a amazons3::accesskey -o bucket=n
|
|||||||
start_nc ngb-targets-nc-sftp
|
start_nc ngb-targets-nc-sftp
|
||||||
run "SFTP" ngb-targets-nc-sftp "sftp -a password::password -o host=ngb-targets-sftp -o root=/upload -o user=ngbackup -o password=ngbackuppw"
|
run "SFTP" ngb-targets-nc-sftp "sftp -a password::password -o host=ngb-targets-sftp -o root=/upload -o user=ngbackup -o password=ngbackuppw"
|
||||||
|
|
||||||
|
start_nc ngb-targets-nc-ftp
|
||||||
|
run "FTP" ngb-targets-nc-ftp "ftp -a password::password -o host=ngb-targets-ftp -o user=ngbackup -o password=ngbackuppw"
|
||||||
|
|
||||||
|
start_nc ngb-targets-nc-smb "$SMB_IMAGE"
|
||||||
|
run "SMB" ngb-targets-nc-smb "smb -a password::password -o host=ngb-targets-smb -o share=backup -o user=ngbackup -o password=ngbackuppw"
|
||||||
|
|
||||||
printf '%s\n' "${RESULTS[@]}"
|
printf '%s\n' "${RESULTS[@]}"
|
||||||
printf '%s\n' "${RESULTS[@]}" | grep -q FAIL && exit 1
|
printf '%s\n' "${RESULTS[@]}" | grep -q FAIL && exit 1
|
||||||
exit 0
|
exit 0
|
||||||
|
|||||||
Executable
+71
@@ -0,0 +1,71 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Per-user backup/restore round trip via user_migration (optional dependency, see PLAN.md 3.8):
|
||||||
|
# occ backup:user:backup / occ backup:user:restore --mode=as-backup|replace.
|
||||||
|
# Usage: user-roundtrip.sh "<command prefix to run occ/php as the web user>" <datadir> [target-args]
|
||||||
|
# e.g. user-roundtrip.sh "docker exec -u www-data ngb-nc35-pgsql" /var/www/html/data
|
||||||
|
set -euo pipefail
|
||||||
|
X="$1"; DATA="$2"
|
||||||
|
TARGET_ARGS="${3:-local -a null::null -o datadir=/tmp/ngb-urt-target}"
|
||||||
|
occ() { $X php occ "$@"; }
|
||||||
|
fail() { echo "FAIL: $*"; exit 1; }
|
||||||
|
ok() { echo "ok $*"; }
|
||||||
|
U=ngbutest-$RANDOM
|
||||||
|
|
||||||
|
occ app:enable ng_backup >/dev/null
|
||||||
|
if ! occ app:install user_migration >/dev/null 2>&1 && ! occ app:enable user_migration >/dev/null 2>&1; then
|
||||||
|
echo "SKIP: user_migration is not available (no network, or the app store is unreachable) -- per-user backup/restore not tested"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
sleep 8 # APCu app config cache
|
||||||
|
|
||||||
|
$X sh -c "printf 'user roundtrip passphrase 2026\n' > /tmp/ngb-urt-pass && mkdir -p /tmp/ngb-urt-target"
|
||||||
|
occ backup:key:init --passphrase-file=/tmp/ngb-urt-pass --label=CI --delete-delay-days=7 >/dev/null || fail "key init"
|
||||||
|
CODE=$(occ backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
occ backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||||
|
ok "key, kit code and confirmation"
|
||||||
|
|
||||||
|
occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add"
|
||||||
|
ok "location added"
|
||||||
|
|
||||||
|
$X sh -c "OC_PASS='User-roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
|
||||||
|
$X sh -c "mkdir -p $DATA/$U/files/Docs && for i in 1 2 3; do head -c 100000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done"
|
||||||
|
occ files:scan "$U" >/dev/null
|
||||||
|
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||||
|
|
||||||
|
MANIFEST=$(occ backup:user:backup ci "$U" | sed -n 's/.* to \(.*\)\.$/\1/p')
|
||||||
|
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
||||||
|
ok "backed up $U to $MANIFEST"
|
||||||
|
|
||||||
|
# as-backup: restore into <uid>-bak; the original account must stay untouched.
|
||||||
|
occ backup:user:restore ci "$MANIFEST" --mode=as-backup >/dev/null || fail "restore as-backup"
|
||||||
|
occ files:scan "$U-bak" >/dev/null
|
||||||
|
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||||
|
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
||||||
|
occ user:delete "$U-bak" >/dev/null
|
||||||
|
$X sh -c "[ -d '$DATA/$U/files/Docs' ]" || fail "the original account should be untouched by an as-backup restore"
|
||||||
|
ok "as-backup restores into <uid>-bak, byte-identical, original account untouched"
|
||||||
|
|
||||||
|
# replace: damage the account, then restore it in place under the same uid.
|
||||||
|
$X sh -c "rm '$DATA/$U/files/Docs/f1.bin' && echo damaged > '$DATA/$U/files/Docs/f1.bin'"
|
||||||
|
occ files:scan "$U" >/dev/null
|
||||||
|
occ backup:user:restore ci "$MANIFEST" --mode=replace >/dev/null || fail "restore replace"
|
||||||
|
occ files:scan "$U" >/dev/null
|
||||||
|
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||||
|
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
||||||
|
ok "replace restores the original content under the same uid, byte-identical"
|
||||||
|
|
||||||
|
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
||||||
|
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
||||||
|
# must forget exactly the 2 oldest.
|
||||||
|
sleep 1.1 # exportId is time-based to the second; force each one to be distinct
|
||||||
|
occ backup:user:backup ci "$U" >/dev/null || fail "third backup:user:backup"
|
||||||
|
sleep 1.1
|
||||||
|
occ backup:user:backup ci "$U" >/dev/null || fail "fourth backup:user:backup"
|
||||||
|
occ backup:retention --user-last=2 >/dev/null || fail "retention --user-last"
|
||||||
|
PRUNE_OUT=$(occ backup:prune ci)
|
||||||
|
echo "$PRUNE_OUT" | grep -q "forgot 2 user export(s) for 1 user(s)" || fail "prune did not forget the expected two user exports"
|
||||||
|
ok "user-export retention forgets exactly the exports over the keep count"
|
||||||
|
|
||||||
|
occ user:delete "$U" >/dev/null
|
||||||
|
$X sh -c "rm -rf /tmp/ngb-urt-target /tmp/ngb-urt-pass"
|
||||||
|
echo "USER ROUNDTRIP PASS"
|
||||||
Reference in New Issue
Block a user