Restore without the recovery kit: the key from the location, with a passphrase
On a server without a backup key, backup:target:add finds the backups at the location and takes their key from its config with one of the old server's passphrases. Backups then wait until a new recovery kit is confirmed. The deletion delay and the repository id are now stored on the location, sealed with the key, so the delay survives a restore without a kit and --repair-config can write a lost config again from the kit. The key page warns against creating a new key when restoring a server. New test: tests/integration/recovery-scenarios.sh, and a no-kit mode in full-restore-roundtrip.sh.
This commit is contained in:
@@ -29,6 +29,15 @@ shown under 1.0.0. The git tag of a release candidate goes after the date.
|
||||
background scan) pause while a backup runs, at most 6 hours in a row, then they get an hour. No app
|
||||
is turned off, and a job that is running goes on. On by default; turn it off under Backups or with
|
||||
`occ backup:pause-jobs off`.
|
||||
- Disaster recovery without the recovery kit: on a new server without a backup key,
|
||||
`occ backup:target:add` finds the backups on the location and takes their key from there with one
|
||||
of the old server's passphrases. Backups then wait until a new recovery kit is confirmed. The
|
||||
deletion delay is now also stored on the location, sealed with the key, from its first backup
|
||||
with this version; a location backed up only by an older version gives the default of 7 days.
|
||||
- `occ backup:target:add --repair-config`: when the config file of a location is gone, the recovery
|
||||
kit writes it again (for locations backed up by this version or newer). Before, the message said
|
||||
this was possible, but it was not.
|
||||
- The key page warns against creating a new key when you are restoring a server.
|
||||
|
||||
### Changed
|
||||
- Locations are backed up one after the other: a backup that has to wait shows as queued. A restore
|
||||
|
||||
@@ -592,6 +592,13 @@ in README "Known limitations":
|
||||
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
||||
`target:add` en sleutelimport (het gevoeligste pad), daarom niet in rc1. Daarna de
|
||||
DR-test op PVE02 herhalen zonder kit.
|
||||
**Gebouwd (2026-11-07):** `backup:target:add` op een server zonder sleutel kijkt eerst of de
|
||||
locatie een repository heeft, vraagt dan een wachtwoordzin (of `--passphrase-file`), opent de
|
||||
slots uit `config`, controleert de sleutel tegen de `check` en slaat hem pas dan op; geen
|
||||
kitbevestiging, dus backups wachten op een nieuwe kit. Nieuw bestand `settings` op de locatie,
|
||||
versleuteld met de hoofdsleutel en eenmalig geschreven (bij `target:add` en bij het begin van
|
||||
een backup): de verwijdertermijn, anders de standaard 7 dagen. Getest: unit-tests en
|
||||
`full-restore-roundtrip.sh 35-apache pgsql no-kit`. Nog open: de DR-test op PVE02 zonder kit.
|
||||
2. Instelling "zware achtergrondjobs pauzeren tijdens een backup" (standaard aan, uit te zetten):
|
||||
Memories, files_antivirus BackgroundScanner, Recognize, Preview Generator, fulltextsearch.
|
||||
Via de reservering/last_run van die jobs (geen apps uitzetten), crashbestendig, maximaal
|
||||
|
||||
@@ -87,8 +87,9 @@ All backups of an installation are encrypted with one master key.
|
||||
- The **recovery kit** is your own copy of that encrypted key. Still keep it off the server and
|
||||
off the location: an attacker who gets write access to the location can damage or delete its
|
||||
`config` file on purpose, and without the kit that makes every backup there unreadable.
|
||||
- For a **restore on a new server** (disaster recovery) you currently need the **recovery kit**
|
||||
and **one passphrase**.
|
||||
- For a **restore on a new server** (disaster recovery) **one passphrase** is enough: the new
|
||||
server takes the key from the location. With the **recovery kit** you can restore even when the
|
||||
location's `config` file is damaged or gone.
|
||||
- Planned: an option to not store the key on the location at all. Then the kit is the only copy,
|
||||
and losing it means losing every backup; that option will come with an explicit warning.
|
||||
- Up to **three passphrases** ("slots") can open the key, so one person leaving or dying does not
|
||||
|
||||
@@ -5,7 +5,8 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
||||
|
||||
## What you need
|
||||
|
||||
- [ ] The **recovery kit** file (`ng_backup-kit.json`) and **one passphrase** of it.
|
||||
- [ ] **One passphrase** of the old server, and if you have it, the **recovery kit** file
|
||||
(`ng_backup-kit.json`).
|
||||
- [ ] The **connection details of the backup location**: host, user, password or keys, bucket, folder.
|
||||
- [ ] A new server with the **same Nextcloud major version** as the old one (for example 34),
|
||||
a database (PostgreSQL, MySQL/MariaDB or SQLite; it may differ from the old one) and room for
|
||||
@@ -15,8 +16,10 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
||||
- [ ] A list of the apps that are **not in the app store** (your own builds, apps copied in by
|
||||
hand). Keep their packages somewhere safe beforehand.
|
||||
|
||||
No kit? If the location still holds its `config` file, the key is there too, but this version
|
||||
can only import it from a kit. Keep the kit off the server, next to the passphrase.
|
||||
No kit? Since version 1.1.0 the passphrase is enough: the key is also stored on the location (in
|
||||
its `config` file), locked with the same passphrases. Skip step 4. The kit is still the safer
|
||||
copy: it helps when that file on the location is damaged. Keep it off the server, next to the
|
||||
passphrase.
|
||||
|
||||
## Steps
|
||||
|
||||
@@ -40,7 +43,8 @@ This keeps the new server quiet (no mail syncs, no notifications, no backups) un
|
||||
|
||||
**3. Install NG Backup** (the same version or newer) and enable it.
|
||||
|
||||
**4. Import the backup key.** Copy the kit to the new server so the web server user can read it.
|
||||
**4. Import the backup key** (with a kit; without one, go to step 5). Copy the kit to the new
|
||||
server so the web server user can read it.
|
||||
|
||||
From your own computer:
|
||||
|
||||
@@ -67,13 +71,20 @@ the index and shows how far it is, for example
|
||||
|
||||
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=... -o root=... --path=...
|
||||
|
||||
If it says "No backup key yet", the key is not imported. Run step 4:
|
||||
Without a kit (step 4 skipped), it finds the backups and asks for a passphrase of the old server;
|
||||
you type it yourself. It then shows the key fingerprint and the deletion delay of the old server.
|
||||
A wrong passphrase changes nothing: run the command again. Backups made before version 1.1.0 did
|
||||
not store the deletion delay on the location; it is then set to 7 days.
|
||||
|
||||
occ backup:key:import /tmp/ng_backup-kit.json
|
||||
If it says that the location's **config file is gone**, the key is only in the recovery kit:
|
||||
import it (step 4), then add the location with `--repair-config` at the end of the same command.
|
||||
That writes the config again from the kit; it works for locations backed up by version 1.1.0 or
|
||||
newer.
|
||||
|
||||
Do **not** run `backup:key:init`, even though the message suggests it: that makes a brand new key,
|
||||
and your backups were encrypted with the old one. NG Backup then refuses the location ("created
|
||||
with a different key").
|
||||
If it says "No backup key yet, and no backups at this location", the connection details or
|
||||
`--path` differ from the old server's. Check them; do **not** run `backup:key:init`: that makes a
|
||||
brand new key, and your backups were encrypted with the old one. NG Backup then refuses the
|
||||
location ("created with a different key").
|
||||
|
||||
**6. List the restore points** and pick one, normally the newest:
|
||||
|
||||
@@ -164,6 +175,10 @@ The command then ends with exit code 2 instead of 0. The same option exists for
|
||||
**11. Make a backup of the new server** to the same location (`occ backup:run offsite`). The old
|
||||
server must stay off: two servers must never write to the same location.
|
||||
|
||||
Restored without a kit? Then backups wait until you have a new one: download it on the backup
|
||||
page (or `occ backup:key:kit --output=...`), store it off the server and confirm it
|
||||
(`occ backup:key:confirm`).
|
||||
|
||||
## Never
|
||||
|
||||
- Never let the old server write to the location again if it comes back. Switch its backups off
|
||||
|
||||
@@ -157,6 +157,10 @@ occ backup:list offsite # the snapshots must be lis
|
||||
occ backup:restore:full offsite <snapshot>
|
||||
```
|
||||
|
||||
Without the kit, leave out `backup:key:import`: `backup:target:add` finds the backups and takes
|
||||
the key from the location with a passphrase (`--passphrase-file=/path/passphrase`, or it asks).
|
||||
Backups then wait until a new kit is downloaded and confirmed.
|
||||
|
||||
The command refuses on an installation that already has more than one user, and asks before it
|
||||
starts; `--force` skips both (also needed to rerun it after it stopped halfway).
|
||||
|
||||
|
||||
@@ -32,6 +32,9 @@ For a fresh installation only (occ backup:key:init must not have been run here y
|
||||
of the kit's passphrase slots to recover the original master key, so occ backup:target:add opens
|
||||
the existing repository on the location instead of creating a new, empty one.
|
||||
|
||||
No kit? Skip this command: occ backup:target:add then takes the key from the location with one
|
||||
of the old server's passphrases. The kit still helps when the location's config file is damaged.
|
||||
|
||||
Example:
|
||||
occ backup:key:import /root/ng_backup-recovery-kit.json
|
||||
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=...
|
||||
|
||||
@@ -13,7 +13,9 @@ use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Input\InputOption;
|
||||
use Symfony\Component\Console\Helper\QuestionHelper;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
use Symfony\Component\Console\Question\Question;
|
||||
|
||||
final class TargetAdd extends Command {
|
||||
public function __construct(
|
||||
@@ -31,10 +33,20 @@ final class TargetAdd extends Command {
|
||||
->addOption('auth', 'a', InputOption::VALUE_REQUIRED, 'Auth mechanism id (see backup:target:backends)')
|
||||
->addOption('option', 'o', InputOption::VALUE_REQUIRED | InputOption::VALUE_IS_ARRAY, 'Backend/auth option key=value (repeatable)')
|
||||
->addOption('path', 'p', InputOption::VALUE_REQUIRED, 'Folder on the location for this repository', 'ng_backup')
|
||||
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Disaster recovery without the recovery kit: read the passphrase from this file instead of asking')
|
||||
->addOption('repair-config', null, InputOption::VALUE_NONE, 'The location\'s config file is gone: write it again with the key of this server or of the imported recovery kit (locations backed up by 1.1.0 or newer)')
|
||||
->setHelp(<<<'HELP'
|
||||
The location is reached with files_external's own connection code, but it is never added as an
|
||||
external storage: it does not appear in Files for anyone. Credentials are stored encrypted.
|
||||
|
||||
Disaster recovery without the recovery kit: on a server without a backup key, a location that
|
||||
holds backups gives its key with one of the old server's passphrases. Run this command first
|
||||
(no occ backup:key:import), then occ backup:list and occ backup:restore:full.
|
||||
|
||||
With the recovery kit: occ backup:key:import first; this command then asks for nothing more.
|
||||
If the location's config file is gone, only the kit has the key: import it, then add the
|
||||
location with --repair-config.
|
||||
|
||||
Examples:
|
||||
occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket=backups -o hostname=s3.example.com -o region=eu-central-1 -o use_ssl=true -o use_path_style=true -o key=AKIA... -o secret=... --path=nextcloud
|
||||
|
||||
@@ -43,13 +55,6 @@ HELP);
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
if (!$this->keys->isInitialized()) {
|
||||
$output->writeln('<error>No backup key yet.</error>');
|
||||
$output->writeln('Restoring from a location that already has backups (disaster recovery)? Import the key from the recovery kit first:');
|
||||
$output->writeln(' occ backup:key:import /path/to/ng_backup-kit.json');
|
||||
$output->writeln('Only for a new location without backups: create a new key with occ backup:key:init. A new key cannot read existing backups.');
|
||||
return 1;
|
||||
}
|
||||
$options = [];
|
||||
foreach ($input->getOption('option') as $pair) {
|
||||
if (!str_contains($pair, '=')) {
|
||||
@@ -63,24 +68,102 @@ HELP);
|
||||
$output->writeln('<error>--auth is required (see occ backup:target:backends)</error>');
|
||||
return 1;
|
||||
}
|
||||
$name = (string)$input->getArgument('name');
|
||||
$backend = (string)$input->getArgument('backend');
|
||||
$auth = (string)$input->getOption('auth');
|
||||
$path = (string)$input->getOption('path');
|
||||
$passphrase = null;
|
||||
$repair = (bool)$input->getOption('repair-config');
|
||||
if (!$this->keys->isInitialized()) {
|
||||
try {
|
||||
$state = $this->targets->locationState($backend, $auth, $options, $path);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
if ($state === TargetService::LOCATION_EMPTY) {
|
||||
$output->writeln('<error>No backup key yet, and no backups at this location.</error>');
|
||||
$output->writeln('Restoring a server (disaster recovery)? Check the connection details and --path: they must be the same as on the old server.');
|
||||
$output->writeln('Only for a new location: create a new key first with occ backup:key:init. A new key cannot read existing backups.');
|
||||
return 1;
|
||||
}
|
||||
if ($state === TargetService::LOCATION_DATA_WITHOUT_CONFIG) {
|
||||
$output->writeln('<error>This location holds backups, but its config file is gone, and the key was stored in it.</error>');
|
||||
$output->writeln('Now only the recovery kit has the key. Import it, then run this command again with --repair-config:');
|
||||
$output->writeln(' occ backup:key:import /path/to/ng_backup-kit.json');
|
||||
$output->writeln('Do not run occ backup:key:init: a new key cannot read these backups.');
|
||||
return 1;
|
||||
}
|
||||
if ($repair) {
|
||||
$output->writeln('<error>The config at this location is there: leave out --repair-config.</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln('No backup key on this server yet, and this location holds backups. Their key is stored there too,');
|
||||
$output->writeln('locked with the passphrases of the old server: one of them unlocks it (the recovery kit is not needed).');
|
||||
$passphrase = $this->passphrase($input, $output);
|
||||
if ($passphrase === null) {
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
// Adopting an existing repository reads its whole index first, which takes a while on a
|
||||
// large backup or a slow line; show how far it is instead of looking stuck.
|
||||
$output->writeln('Connecting to the location; with existing backups this reads their index first ...');
|
||||
try {
|
||||
$r = $this->targets->add($input->getArgument('name'), $input->getArgument('backend'), $input->getOption('auth'), $options,
|
||||
(string)$input->getOption('path'), self::progress($output));
|
||||
$r = $passphrase === null
|
||||
? $this->targets->add($name, $backend, $auth, $options, $path, self::progress($output), $repair)
|
||||
: $this->targets->addWithPassphrase($name, $backend, $auth, $options, $path, $passphrase, self::progress($output));
|
||||
} catch (\OCA\NgBackup\Crypto\CryptoException $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
if (!$this->keys->isInitialized()) {
|
||||
$output->writeln('Nothing was changed. Run the command again to try another passphrase, or use the recovery kit (occ backup:key:import).');
|
||||
}
|
||||
return 1;
|
||||
} catch (\Throwable $e) {
|
||||
if ($output->isDecorated()) {
|
||||
$output->write("\r\033[K"); // clear an unfinished progress line
|
||||
}
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
if (str_contains($e->getMessage(), 'different key')) {
|
||||
$output->writeln('The backup key here (fingerprint ' . $this->keys->fingerprint() . ') is not the one these backups were made with.');
|
||||
$output->writeln('Restoring a server? The recovery kit must be the old server\'s: its fingerprint is in the kit file. Nothing was changed.');
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
if ($repair) {
|
||||
$output->writeln('The config of the location was written again from the recovery kit.');
|
||||
}
|
||||
$output->writeln(sprintf('Location <info>%s</info> added (id %d), repository %s %s.', $r['target']->getName(), $r['target']->getId(),
|
||||
$r['target']->getRepositoryId(), $r['created'] ? 'created' : 'found and adopted'));
|
||||
if ($passphrase !== null) {
|
||||
$output->writeln('Backup key taken from the location, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
||||
$output->writeln(array_key_exists('deleteDelayDays', $r) && $r['deleteDelayDays'] !== null
|
||||
? 'Deletion delay for forgotten snapshots: <info>' . $this->keys->deleteDelayDays() . '</info> days (from the location).'
|
||||
: 'Deletion delay for forgotten snapshots: <info>' . $this->keys->deleteDelayDays() . '</info> days (the default: the location has none stored, its backups were made before NG Backup 1.1.0).');
|
||||
$output->writeln('Next: <info>occ backup:list</info> and <info>occ backup:restore:full</info>. Before backups run again, download a new recovery kit and confirm it (<info>occ backup:key:kit</info>, <info>occ backup:key:confirm</info>).');
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
private function passphrase(InputInterface $input, OutputInterface $output): ?string {
|
||||
$file = $input->getOption('passphrase-file');
|
||||
if ($file !== null) {
|
||||
$raw = @file_get_contents((string)$file);
|
||||
if ($raw === false) {
|
||||
$output->writeln('<error>Cannot read the passphrase file</error>');
|
||||
return null;
|
||||
}
|
||||
return rtrim($raw, "\r\n");
|
||||
}
|
||||
if (!$input->isInteractive()) {
|
||||
$output->writeln('<error>Give the passphrase with --passphrase-file, or run the command without --no-interaction to type it.</error>');
|
||||
return null;
|
||||
}
|
||||
$q = (new Question('Passphrase of the old server (one of its slots): '))->setHidden(true)->setHiddenFallback(false);
|
||||
/** @var QuestionHelper $helper */
|
||||
$helper = $this->getHelper('question');
|
||||
return (string)$helper->ask($input, $output, $q);
|
||||
}
|
||||
|
||||
/**
|
||||
* Progress of reading the index: on a terminal one line that is updated in place, otherwise
|
||||
* (a log, a pipe) a line every 30 seconds. The time left is estimated from the rate so far.
|
||||
|
||||
@@ -18,6 +18,7 @@ use OCA\NgBackup\Crypto\StreamCipher;
|
||||
*
|
||||
* Layout on the backend:
|
||||
* config plaintext JSON: format version, repository id, wrapped master key
|
||||
* settings encrypted JSON: settings fixed at installation (deletion delay)
|
||||
* packs/xx/<packId> encrypted blobs + encrypted pack header
|
||||
* index/<packId> encrypted list of the blobs in that pack
|
||||
* snapshots/<snapId> encrypted JSON lines: one meta line, then one line per file
|
||||
@@ -28,6 +29,7 @@ use OCA\NgBackup\Crypto\StreamCipher;
|
||||
final class Repository {
|
||||
public const FORMAT = 1;
|
||||
public const BLOB_SIZE = 4 * 1048576;
|
||||
private const SETTINGS = 'settings';
|
||||
private const FLAG_DEFLATE = 1;
|
||||
/** Blobs from this size on get a quick compression test before the full compression. */
|
||||
private const DEFLATE_PROBE_FROM = 256 * 1024;
|
||||
@@ -79,7 +81,7 @@ final class Repository {
|
||||
// opens this data; refuse instead.
|
||||
foreach (['index', 'snapshots', 'trash/snapshots', 'catalog'] as $dir) {
|
||||
if ($backend->listFiles($dir) !== []) {
|
||||
throw new RepositoryException("This location holds backup data ($dir/) but no repository config could be read: not creating a new repository over it. Check the connection and the folder; a missing config can be restored from the recovery kit.");
|
||||
throw new RepositoryException("This location holds backup data ($dir/) but no repository config could be read: not creating a new repository over it. Check the connection and the folder. If the config is really gone, occ backup:target:add with --repair-config writes it again with this server's key (on a new server: import the recovery kit first, occ backup:key:import).");
|
||||
}
|
||||
}
|
||||
$id = bin2hex(random_bytes(16));
|
||||
@@ -95,12 +97,7 @@ final class Repository {
|
||||
/** @param ?callable(int, int): void $progress index files downloaded so far, and in all (see BlobIndex::load()) */
|
||||
public static function openWithKey(IBackend $backend, KeyRing $keys, ?IIndexCache $cache = null, ?ICatalogAnchor $anchor = null, ?callable $progress = null): self {
|
||||
$config = self::readConfig($backend);
|
||||
try {
|
||||
$ok = isset($config['check']) && (new StreamCipher($keys))->decryptString((string)base64_decode($config['check'], true), 'config-check') === $config['id'];
|
||||
} catch (\OCA\NgBackup\Crypto\CryptoException) {
|
||||
$ok = false;
|
||||
}
|
||||
if (!$ok) {
|
||||
if (!self::configMatches($config, $keys)) {
|
||||
throw new RepositoryException('This repository was created with a different key');
|
||||
}
|
||||
$repo = new self($backend, $keys, $config['id']);
|
||||
@@ -196,6 +193,97 @@ final class Repository {
|
||||
return $config;
|
||||
}
|
||||
|
||||
private static function configMatches(array $config, KeyRing $keys): bool {
|
||||
try {
|
||||
return isset($config['check'], $config['id'])
|
||||
&& (new StreamCipher($keys))->decryptString((string)base64_decode((string)$config['check'], true), 'config-check') === $config['id'];
|
||||
} catch (\OCA\NgBackup\Crypto\CryptoException) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The passphrase-wrapped key slots in the config at this location: disaster recovery without
|
||||
* the recovery kit, where any one of their passphrases gives back the master key.
|
||||
*/
|
||||
public static function wrappedKeyAt(IBackend $backend): array {
|
||||
$config = self::readConfig($backend);
|
||||
if (!isset($config['key']) || !is_array($config['key'])) {
|
||||
throw new RepositoryException('The repository config at this location holds no key');
|
||||
}
|
||||
return $config['key'];
|
||||
}
|
||||
|
||||
/** Whether the repository at this location was created with this key. Does not read the index. */
|
||||
public static function madeWithKey(IBackend $backend, KeyRing $keys): bool {
|
||||
return self::configMatches(self::readConfig($backend), $keys);
|
||||
}
|
||||
|
||||
/**
|
||||
* Settings of the installation that never change (the deletion delay), sealed with the
|
||||
* master key so that, unlike the config and the recovery kit, nobody can alter them unseen.
|
||||
* Empty when the location has none yet (made before 1.1.0, until its next backup) or they
|
||||
* cannot be read.
|
||||
*
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
public static function settingsAt(IBackend $backend, KeyRing $keys): array {
|
||||
try {
|
||||
if (!$backend->exists(self::SETTINGS)) {
|
||||
return [];
|
||||
}
|
||||
$fh = $backend->get(self::SETTINGS);
|
||||
$raw = (string)stream_get_contents($fh);
|
||||
fclose($fh);
|
||||
$settings = json_decode((new StreamCipher($keys))->decryptString($raw, 'settings'), true, 16, JSON_THROW_ON_ERROR);
|
||||
return is_array($settings) ? $settings : [];
|
||||
} catch (BackendException|\OCA\NgBackup\Crypto\CryptoException|\JsonException) {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Write the settings once; a location that has them keeps them. The repository id goes with
|
||||
* them: with it and the key from the recovery kit, a lost config can be written again.
|
||||
*/
|
||||
public function recordSettings(array $settings): void {
|
||||
if ($this->backend->exists(self::SETTINGS)) {
|
||||
return;
|
||||
}
|
||||
$settings['repository'] = $this->repositoryId;
|
||||
self::putString($this->backend, self::SETTINGS, $this->cipher->encryptString(json_encode($settings, JSON_THROW_ON_ERROR), 'settings'));
|
||||
}
|
||||
|
||||
/** Whether the location holds backup data, with or without a config. */
|
||||
public static function holdsData(IBackend $backend): bool {
|
||||
foreach (['index', 'snapshots', 'catalog'] as $dir) {
|
||||
if ($backend->listFiles($dir) !== []) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Write a lost config again, from the key (recovery kit) and the repository id in the
|
||||
* sealed settings. Only where there is no config and the settings open with this key, so
|
||||
* nothing that works is ever overwritten and a wrong key writes nothing.
|
||||
*/
|
||||
public static function repairConfig(IBackend $backend, KeyRing $keys, array $wrappedKey): void {
|
||||
if ($backend->exists('config')) {
|
||||
throw new RepositoryException('This location has a config: nothing to repair, leave out --repair-config');
|
||||
}
|
||||
$id = self::settingsAt($backend, $keys)['repository'] ?? null;
|
||||
if (!is_string($id) || !preg_match('/^[0-9a-f]{32}$/', $id)) {
|
||||
throw new RepositoryException($backend->exists(self::SETTINGS)
|
||||
? 'The settings at this location do not open with this key: the recovery kit is of another server, or the settings are damaged. The config cannot be written again.'
|
||||
: 'This location has no settings file (it was backed up only by NG Backup before 1.1.0), so its config cannot be written again.');
|
||||
}
|
||||
$config = ['format' => self::FORMAT, 'id' => $id, 'created' => gmdate('c'), 'key' => $wrappedKey,
|
||||
'check' => base64_encode((new StreamCipher($keys))->encryptString($id, 'config-check')), 'repaired' => true];
|
||||
self::putString($backend, 'config', json_encode($config, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR));
|
||||
}
|
||||
|
||||
/** Replace the passphrase-wrapped key slots in this repository's config (after a slot change). */
|
||||
public function updateWrappedKey(array $wrappedKey): void {
|
||||
$config = self::readConfig($this->backend);
|
||||
|
||||
@@ -225,6 +225,8 @@ final class BackupService {
|
||||
if (!isset($state['dump'])) {
|
||||
// It may have been queued for hours: the restore point has the time of its data.
|
||||
$state['files']['started'] = gmdate('c');
|
||||
// Locations from before 1.1.0 get their sealed settings at their next backup.
|
||||
$this->targets->recordSettings($repo);
|
||||
}
|
||||
$dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_'));
|
||||
$dbStats = $state['dbStats'] ?? [];
|
||||
|
||||
@@ -91,20 +91,38 @@ final class KeyService {
|
||||
throw new \InvalidArgumentException('Not a NG Backup recovery kit file');
|
||||
}
|
||||
$keys = KeyRing::unwrapAny($kit['wrapped_key'], $passphrase);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
|
||||
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
|
||||
$this->cached = $keys;
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode($kit['wrapped_key'], JSON_THROW_ON_ERROR), true, true);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
|
||||
// Kits from before this field existed fall back to the default.
|
||||
$delay = (int)($kit['delete_delay_days'] ?? self::DEFAULT_DELETE_DELAY);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY, ($delay >= 1 && $delay <= 365) ? $delay : self::DEFAULT_DELETE_DELAY, true);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, (int)($kit['kit_version'] ?? 1), true);
|
||||
$this->store($keys, $kit['wrapped_key'], (int)($kit['delete_delay_days'] ?? self::DEFAULT_DELETE_DELAY), (int)($kit['kit_version'] ?? 1));
|
||||
// Having and successfully using a real kit file is stronger proof than the normal
|
||||
// download-and-confirm flow it replaces.
|
||||
$this->confirmRecoveryKit('cli-import');
|
||||
}
|
||||
|
||||
/**
|
||||
* Disaster recovery without the recovery kit: adopt the master key that a passphrase opened
|
||||
* from the key slots in a location's config (TargetService::addWithPassphrase). There is no
|
||||
* kit, so none is confirmed: backups wait until a new kit is downloaded and confirmed.
|
||||
*
|
||||
* @param ?int $deleteDelayDays from the location's sealed settings, null when it has none
|
||||
*/
|
||||
public function adoptKey(KeyRing $keys, array $wrappedKey, ?int $deleteDelayDays): void {
|
||||
if ($this->isInitialized()) {
|
||||
throw new \RuntimeException('A backup key already exists');
|
||||
}
|
||||
$this->store($keys, $wrappedKey, $deleteDelayDays ?? self::DEFAULT_DELETE_DELAY, 1);
|
||||
}
|
||||
|
||||
private function store(KeyRing $keys, array $wrappedKey, int $deleteDelayDays, int $kitVersion): void {
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
|
||||
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
|
||||
$this->cached = $keys;
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode($wrappedKey, JSON_THROW_ON_ERROR), true, true);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY,
|
||||
($deleteDelayDays >= 1 && $deleteDelayDays <= 365) ? $deleteDelayDays : self::DEFAULT_DELETE_DELAY, true);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, max(1, $kitVersion), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* The master key and the recovery kit code are stored encrypted with Nextcloud's own
|
||||
* instance secret (ICrypto), not with themselves; the wrapped key is not (it is sealed with
|
||||
|
||||
@@ -52,11 +52,14 @@ final class TargetService {
|
||||
* @param ?callable(int, int): void $progress reading an existing repository's index: files done, and in all
|
||||
* @return array{target: Target, created: bool}
|
||||
*/
|
||||
public function add(string $name, string $backend, string $auth, array $options, string $basePath, ?callable $progress = null): array {
|
||||
public function add(string $name, string $backend, string $auth, array $options, string $basePath, ?callable $progress = null, bool $repairConfig = false): array {
|
||||
$this->validateName($name);
|
||||
$this->requireUsableBackend($backend);
|
||||
$keys = $this->keys->keyRing();
|
||||
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath);
|
||||
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath, !$repairConfig);
|
||||
if ($repairConfig) {
|
||||
Repository::repairConfig($storageBackend, $keys, $this->keys->wrappedKey());
|
||||
}
|
||||
if ($storageBackend->exists('config')) {
|
||||
$repo = Repository::openWithKey($storageBackend, $keys, $this->indexCache, $this->anchor, $progress);
|
||||
$created = false;
|
||||
@@ -64,6 +67,7 @@ final class TargetService {
|
||||
$repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey(), $this->anchor);
|
||||
$created = true;
|
||||
}
|
||||
$this->recordSettings($repo);
|
||||
|
||||
foreach ($this->mapper->findAll() as $existing) {
|
||||
if ($existing->getRepositoryId() === $repo->id()) {
|
||||
@@ -93,6 +97,65 @@ final class TargetService {
|
||||
return ['target' => $target, 'created' => $created];
|
||||
}
|
||||
|
||||
public const LOCATION_REPOSITORY = 'repository';
|
||||
public const LOCATION_DATA_WITHOUT_CONFIG = 'data-without-config';
|
||||
public const LOCATION_EMPTY = 'empty';
|
||||
|
||||
/**
|
||||
* What these settings lead to: a repository, backup data whose config is gone, or nothing
|
||||
* of NG Backup. Creates nothing.
|
||||
*
|
||||
* @return self::LOCATION_*
|
||||
*/
|
||||
public function locationState(string $backend, string $auth, array $options, string $basePath): string {
|
||||
$this->requireUsableBackend($backend);
|
||||
$storage = $this->buildBackend($backend, $auth, $options, $basePath, false);
|
||||
if ($storage->exists('config')) {
|
||||
return self::LOCATION_REPOSITORY;
|
||||
}
|
||||
return Repository::holdsData($storage) ? self::LOCATION_DATA_WITHOUT_CONFIG : self::LOCATION_EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* Disaster recovery without the recovery kit, on an installation without a backup key: take
|
||||
* the key from the location's config with one of its passphrases, then add the location as
|
||||
* usual. The key is only kept when it really is the one the repository was made with.
|
||||
*
|
||||
* @param array<string, mixed> $options
|
||||
* @param ?callable(int, int): void $progress
|
||||
* @return array{target: Target, created: bool, deleteDelayDays: ?int} the deletion delay the location had stored, if any
|
||||
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase
|
||||
*/
|
||||
public function addWithPassphrase(string $name, string $backend, string $auth, array $options, string $basePath,
|
||||
#[\SensitiveParameter] string $passphrase, ?callable $progress = null): array {
|
||||
if ($this->keys->isInitialized()) {
|
||||
throw new \RuntimeException('A backup key already exists here');
|
||||
}
|
||||
$this->validateName($name);
|
||||
$this->requireUsableBackend($backend);
|
||||
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath, false);
|
||||
if (!$storageBackend->exists('config')) {
|
||||
throw new \InvalidArgumentException("No NG Backup repository found in folder '" . trim($basePath, '/') . "'");
|
||||
}
|
||||
$wrapped = Repository::wrappedKeyAt($storageBackend);
|
||||
$keys = \OCA\NgBackup\Crypto\KeyRing::unwrapAny($wrapped, $passphrase);
|
||||
if (!Repository::madeWithKey($storageBackend, $keys)) {
|
||||
throw new \RuntimeException('The key in the config at this location does not open its backups: the config is damaged or was replaced. Only the recovery kit can help now: occ backup:key:import <kit file>, then this command again.');
|
||||
}
|
||||
$delay = Repository::settingsAt($storageBackend, $keys)['delete_delay_days'] ?? null;
|
||||
$delay = is_int($delay) ? $delay : null;
|
||||
$this->keys->adoptKey($keys, $wrapped, $delay);
|
||||
return $this->add($name, $backend, $auth, $options, $basePath, $progress) + ['deleteDelayDays' => $delay];
|
||||
}
|
||||
|
||||
/** Seal the settings fixed at installation on the location, once. A failure is tried again at the next backup. */
|
||||
public function recordSettings(Repository $repo): void {
|
||||
try {
|
||||
$repo->recordSettings(['delete_delay_days' => $this->keys->deleteDelayDays()]);
|
||||
} catch (\Throwable) {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Change how an existing location is reached (credentials, host, folder, even the connection
|
||||
* type) or its name. The new settings must lead to the same repository: the history stays
|
||||
|
||||
@@ -4,8 +4,11 @@
|
||||
-->
|
||||
<template>
|
||||
<NcSettingsSection :name="t('ng_backup', 'Backup key and recovery kit')"
|
||||
:description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups. To restore on a new server you need the recovery kit and one of the passphrases.')">
|
||||
:description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups. To restore on a new server you need one of the passphrases; the recovery kit also helps when the key on the backup location is damaged.')">
|
||||
<!-- 1. Create the key -->
|
||||
<NcNoteCard v-if="!status.key.initialized" type="warning">
|
||||
{{ t('ng_backup', 'Restoring a server after a crash? Do not create a key here: a new key cannot read your backups. Follow the disaster recovery guide instead (occ backup:target:add asks for one of your passphrases, or import the recovery kit with occ backup:key:import).') }}
|
||||
</NcNoteCard>
|
||||
<form v-if="!status.key.initialized" class="ngb-form" @submit.prevent="createKey">
|
||||
<NcTextField v-model="label" :label="t('ng_backup', 'Who keeps this passphrase (e.g. Safe, CTO)')" />
|
||||
<NcPasswordField v-model="pass1" :label="t('ng_backup', 'Passphrase (at least 16 characters)')" autocomplete="new-password" />
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||
|
||||
use OCA\NgBackup\Backend\LocalBackend;
|
||||
use OCA\NgBackup\Crypto\CryptoException;
|
||||
use OCA\NgBackup\Crypto\KeyRing;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** Disaster recovery without the recovery kit: the location gives its key with a passphrase. */
|
||||
class KeyFromLocationTest extends TestCase {
|
||||
use TempDirTrait;
|
||||
|
||||
private string $dir;
|
||||
private LocalBackend $backend;
|
||||
private KeyRing $keys;
|
||||
|
||||
protected function setUp(): void {
|
||||
$this->dir = $this->tempDir();
|
||||
$this->backend = new LocalBackend($this->dir);
|
||||
$this->keys = KeyRing::generate();
|
||||
$slots = ['slots' => [
|
||||
['slot' => 1, 'label' => 'A', 'created' => 0, 'key' => $this->keys->wrap('first passphrase')],
|
||||
['slot' => 2, 'label' => 'B', 'created' => 0, 'key' => $this->keys->wrap('second passphrase')],
|
||||
]];
|
||||
Repository::initWithKey($this->backend, $this->keys, $slots)->recordSettings(['delete_delay_days' => 30]);
|
||||
}
|
||||
|
||||
public function testEachPassphraseGivesTheKeyThatOpensTheBackups(): void {
|
||||
foreach (['first passphrase', 'second passphrase'] as $passphrase) {
|
||||
$keys = KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), $passphrase);
|
||||
$this->assertTrue(Repository::madeWithKey($this->backend, $keys));
|
||||
$this->assertSame($this->keys->exportMasterKey(), $keys->exportMasterKey());
|
||||
}
|
||||
}
|
||||
|
||||
public function testAWrongPassphraseGivesNothing(): void {
|
||||
$this->expectException(CryptoException::class);
|
||||
KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), 'not the passphrase');
|
||||
}
|
||||
|
||||
public function testAnotherKeyIsNotTakenForTheRightOne(): void {
|
||||
$this->assertFalse(Repository::madeWithKey($this->backend, KeyRing::generate()));
|
||||
}
|
||||
|
||||
public function testTheSettingsAreSealedAndWrittenOnce(): void {
|
||||
$this->assertSame(30, Repository::settingsAt($this->backend, $this->keys)['delete_delay_days']);
|
||||
Repository::openWithKey($this->backend, $this->keys)->recordSettings(['delete_delay_days' => 1]);
|
||||
$this->assertSame(30, Repository::settingsAt($this->backend, $this->keys)['delete_delay_days']);
|
||||
$this->assertSame([], Repository::settingsAt($this->backend, KeyRing::generate()));
|
||||
}
|
||||
|
||||
public function testAlteredSettingsAreNotBelieved(): void {
|
||||
$file = $this->dir . '/settings';
|
||||
$raw = file_get_contents($file);
|
||||
$raw[strlen($raw) - 1] = $raw[strlen($raw) - 1] ^ "\x01";
|
||||
file_put_contents($file, $raw);
|
||||
$this->assertSame([], Repository::settingsAt($this->backend, $this->keys));
|
||||
}
|
||||
|
||||
public function testTheKitWritesALostConfigAgainAndTheBackupsOpen(): void {
|
||||
$src = $this->tempDir();
|
||||
file_put_contents("$src/a", 'still here');
|
||||
$repo = Repository::openWithKey($this->backend, $this->keys);
|
||||
$id = $repo->id();
|
||||
$snapshot = $repo->backupDirectory($src)['snapshot'];
|
||||
unlink($this->dir . '/config');
|
||||
$this->assertTrue(Repository::holdsData($this->backend));
|
||||
|
||||
Repository::repairConfig($this->backend, $this->keys, $this->keys->wrap('kit passphrase'));
|
||||
$repo = Repository::openWithKey($this->backend, $this->keys);
|
||||
$this->assertSame($id, $repo->id());
|
||||
$this->assertSame([$snapshot], $repo->listSnapshots());
|
||||
$out = $this->tempDir();
|
||||
$repo->restore($snapshot, $out);
|
||||
$this->assertSame('still here', file_get_contents("$out/a"));
|
||||
$this->assertSame($this->keys->exportMasterKey(), KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), 'kit passphrase')->exportMasterKey());
|
||||
}
|
||||
|
||||
public function testAnotherKeyWritesNoConfig(): void {
|
||||
unlink($this->dir . '/config');
|
||||
$other = KeyRing::generate();
|
||||
try {
|
||||
Repository::repairConfig($this->backend, $other, $other->wrap('x'));
|
||||
$this->fail('a config was written with another key');
|
||||
} catch (\OCA\NgBackup\Repository\RepositoryException $e) {
|
||||
$this->assertStringContainsString('another server', $e->getMessage());
|
||||
}
|
||||
$this->assertFileDoesNotExist($this->dir . '/config');
|
||||
}
|
||||
|
||||
public function testAConfigThatIsThereIsNeverRepaired(): void {
|
||||
$before = file_get_contents($this->dir . '/config');
|
||||
try {
|
||||
Repository::repairConfig($this->backend, $this->keys, $this->keys->wrap('x'));
|
||||
$this->fail('an existing config was replaced');
|
||||
} catch (\OCA\NgBackup\Repository\RepositoryException) {
|
||||
}
|
||||
$this->assertSame($before, file_get_contents($this->dir . '/config'));
|
||||
}
|
||||
|
||||
public function testALocationWithoutSettingsHasNone(): void {
|
||||
$other = new LocalBackend($this->tempDir());
|
||||
Repository::initWithKey($other, $this->keys, $this->keys->wrap('x'));
|
||||
$this->assertSame([], Repository::settingsAt($other, $this->keys));
|
||||
}
|
||||
}
|
||||
@@ -2,8 +2,10 @@
|
||||
# Disaster recovery: occ backup:key:import + backup:target:add + backup:restore:full on a
|
||||
# SECOND, fresh Nextcloud instance that never saw the first one, sharing only the backup
|
||||
# location (a local directory, bind-mounted into both containers) -- see PLAN.md 3.7.
|
||||
# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|salvage|salvage-files|to-pgsql|to-mysql|to-sqlite]
|
||||
# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|salvage|salvage-files|to-pgsql|to-mysql|to-sqlite|no-kit]
|
||||
# e.g. full-restore-roundtrip.sh 35-apache pgsql
|
||||
# With "no-kit" the recovery kit is gone: backup:target:add takes the key from the location with
|
||||
# the passphrase, and backups wait until a new kit is confirmed.
|
||||
# With "salvage-files" one pack is damaged and another is gone from the location. The restore must
|
||||
# stop and point at --salvage; run again as the guide says, it restores the database and every file
|
||||
# that can be read, and lists the rest (exit 2). NGB_FRT_DATA_MB sets the amount of data (default 200).
|
||||
@@ -93,7 +95,10 @@ run_on ngb-frt-origin app:enable ng_backup >/dev/null
|
||||
ok "origin instance up ($DB${LEGACY:+, NG Backup 0.9.7})"
|
||||
|
||||
docker exec ngb-frt-origin sh -c "printf 'full restore roundtrip passphrase 2026\n' > /tmp/pass"
|
||||
run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI >/dev/null || fail "key init"
|
||||
DELAY_OPT=()
|
||||
[ "$MODE" = no-kit ] && DELAY_OPT=(--delete-delay-days=21)
|
||||
run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI "${DELAY_OPT[@]}" >/dev/null || fail "key init"
|
||||
ORIGIN_FP=$(run_on ngb-frt-origin backup:status | grep -o 'Key: [0-9a-f-]*' | cut -d' ' -f2)
|
||||
run_on ngb-frt-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||
CODE=$(run_on ngb-frt-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-frt-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||
@@ -187,8 +192,29 @@ docker exec ngb-frt-restore chmod 644 /tmp/kit.json
|
||||
docker cp /tmp/ngb-frt-pass ngb-frt-restore:/tmp/pass
|
||||
ok "fresh restore instance up"
|
||||
|
||||
run_on ngb-frt-restore backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "key import"
|
||||
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo 2>&1) || { printf '%s\n' "$ADD_OUT"; fail "target add on the restore instance"; }
|
||||
if [ "$MODE" = no-kit ]; then
|
||||
# No recovery kit: backup:target:add takes the key from the location with a passphrase
|
||||
docker exec ngb-frt-restore rm -f /tmp/kit.json
|
||||
docker exec ngb-frt-restore sh -c "printf 'not the passphrase of the old server\n' > /tmp/wrongpass && chmod 644 /tmp/wrongpass"
|
||||
OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo --passphrase-file=/tmp/wrongpass 2>&1) \
|
||||
&& { printf '%s\n' "$OUT"; fail "a wrong passphrase was accepted"; }
|
||||
printf '%s\n' "$OUT" | grep -q "Nothing was changed" || { printf '%s\n' "$OUT"; fail "a wrong passphrase did not say that nothing was changed"; }
|
||||
STATUS_OUT=$(run_on ngb-frt-restore backup:status 2>&1 || true)
|
||||
printf '%s\n' "$STATUS_OUT" | grep -q "Key: not created" || { printf '%s\n' "$STATUS_OUT"; fail "a wrong passphrase left a key behind"; }
|
||||
[ -z "$(run_on ngb-frt-restore backup:target:list 2>&1 | grep offsite)" ] || fail "a wrong passphrase left a location behind"
|
||||
OUT=$(run_on ngb-frt-restore backup:target:add other "${LOCATION[@]}" --path=nothing-here --passphrase-file=/tmp/pass 2>&1) \
|
||||
&& { printf '%s\n' "$OUT"; fail "a folder without backups was added without a key"; }
|
||||
printf '%s\n' "$OUT" | grep -q "no backups at this location" || { printf '%s\n' "$OUT"; fail "a folder without backups did not say so"; }
|
||||
ok "without a kit: a wrong passphrase and a folder without backups change nothing"
|
||||
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo --passphrase-file=/tmp/pass 2>&1) \
|
||||
|| { printf '%s\n' "$ADD_OUT"; fail "target add with the passphrase on the restore instance"; }
|
||||
printf '%s\n' "$ADD_OUT" | grep -q "fingerprint $ORIGIN_FP" || { printf '%s\n' "$ADD_OUT"; fail "the key from the location is not the origin's ($ORIGIN_FP)"; }
|
||||
printf '%s\n' "$ADD_OUT" | grep -q "21 days (from the location)" || { printf '%s\n' "$ADD_OUT"; fail "the deletion delay did not come from the location"; }
|
||||
ok "without a kit: the key and the deletion delay came from the location"
|
||||
else
|
||||
run_on ngb-frt-restore backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "key import"
|
||||
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo 2>&1) || { printf '%s\n' "$ADD_OUT"; fail "target add on the restore instance"; }
|
||||
fi
|
||||
LIST_OUT=$(run_on ngb-frt-restore backup:list offsite 2>&1)
|
||||
printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
|
||||
printf 'target:add said:\n%s\nbackup:list said:\n%s\n' "$ADD_OUT" "$LIST_OUT"
|
||||
@@ -196,7 +222,8 @@ printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
|
||||
docker exec ngb-frt-restore sh -c 'ls -la /mnt/shared/repo /mnt/shared/repo/snapshots' || true
|
||||
fail "the origin's snapshot is not listed after connecting the location"
|
||||
}
|
||||
ok "recovery kit imported, existing repository adopted, snapshot visible"
|
||||
if [ "$MODE" = no-kit ]; then KEY_FROM="key from the location"; else KEY_FROM="recovery kit imported"; fi
|
||||
ok "$KEY_FROM, existing repository adopted, snapshot visible"
|
||||
|
||||
if [ "$MODE" = salvage ]; then
|
||||
set +e
|
||||
@@ -371,6 +398,13 @@ if [ "$MODE" = sftp ]; then
|
||||
ok "over SFTP: read ahead on connections of its own ($LOGINS SFTP logins during the restore), large file identical, no spool left"
|
||||
fi
|
||||
|
||||
if [ "$MODE" = no-kit ]; then
|
||||
OUT=$(run_on ngb-frt-restore backup:run offsite -l post-restore-check 2>&1) && { printf '%s\n' "$OUT"; fail "a backup ran without a confirmed recovery kit"; }
|
||||
printf '%s\n' "$OUT" | grep -q "Confirm the recovery kit" || { printf '%s\n' "$OUT"; fail "the refused backup does not point at the recovery kit"; }
|
||||
CODE=$(run_on ngb-frt-restore backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-frt-restore backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm a new kit after the restore"
|
||||
ok "without a kit: backups wait for a new recovery kit to be confirmed"
|
||||
fi
|
||||
run_on ngb-frt-restore backup:run offsite -l post-restore-check >/dev/null || fail "a new backup after the restore fails"
|
||||
ok "the restored instance can run a new backup of its own"
|
||||
|
||||
|
||||
Executable
+203
@@ -0,0 +1,203 @@
|
||||
#!/bin/bash
|
||||
# Getting the backup key onto a new server, with and without the recovery kit, and what the
|
||||
# administrator is told in each case. Every scenario starts on a fresh Nextcloud (SQLite) with a
|
||||
# copy of the origin's location; nothing is restored (full-restore-roundtrip.sh does that).
|
||||
# Usage: recovery-scenarios.sh [nextcloud-image-tag] e.g. recovery-scenarios.sh 35-apache
|
||||
#
|
||||
# kit kit + passphrase: backup:key:import, then backup:target:add asks nothing
|
||||
# kit-wrong kit with a wrong passphrase: refused, no key left behind
|
||||
# no-kit passphrase only: target:add asks for it (or --passphrase-file), a wrong one
|
||||
# changes nothing; backups wait for a new kit
|
||||
# no-kit-old as no-kit, on a location backed up only before 1.1.0 (no settings file):
|
||||
# the deletion delay is the default, and the admin is told so
|
||||
# no-config-no-kit the location's config is gone and there is no kit: told that only the kit helps
|
||||
# no-config-kit config gone, kit there: --repair-config writes it again, the backups open
|
||||
# other-kit the kit of another server: refused, told to use the old server's kit
|
||||
# empty nothing at the location: told to check the details, not to make a new key
|
||||
set -euo pipefail
|
||||
IMAGE="nextcloud:${1:-35-apache}"
|
||||
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
||||
SHARED=$(mktemp -d)
|
||||
chmod 777 "$SHARED"
|
||||
PASS='recovery scenarios passphrase 2026'
|
||||
fail() { echo "FAIL: $*"; exit 1; }
|
||||
ok() { echo "ok $*"; }
|
||||
|
||||
cleanup() {
|
||||
docker rm -fv ngb-rs-origin ngb-rs-other ngb-rs-new >/dev/null 2>&1 || true
|
||||
docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
|
||||
rmdir "$SHARED" 2>/dev/null || true
|
||||
return 0
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
|
||||
# Output of a command that must fail; the scenario fails when it succeeds.
|
||||
must_fail() { # container what args...
|
||||
local out
|
||||
if out=$(run_on "$1" "${@:3}" 2>&1); then printf '%s\n' "$out"; fail "$2: the command succeeded"; fi
|
||||
printf '%s\n' "$out"
|
||||
}
|
||||
expect() { # output text what
|
||||
printf '%s\n' "$1" | grep -qF -- "$2" || { printf '%s\n' "$1"; fail "$3: the output does not say \"$2\""; }
|
||||
}
|
||||
not_expect() {
|
||||
printf '%s\n' "$1" | grep -qF -- "$2" && { printf '%s\n' "$1"; fail "$3: the output says \"$2\""; }
|
||||
return 0
|
||||
}
|
||||
|
||||
start() { # name
|
||||
docker rm -fv "$1" >/dev/null 2>&1 || true
|
||||
docker run -d --name "$1" -e SQLITE_DATABASE=nextcloud -e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin \
|
||||
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
|
||||
for i in $(seq 1 90); do run_on "$1" status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
|
||||
docker exec "$1" chown www-data:www-data /var/www/html/custom_apps
|
||||
run_on "$1" app:enable ng_backup >/dev/null
|
||||
docker exec "$1" sh -c "printf '%s\n' '$PASS' > /tmp/pass; printf 'not the passphrase at all\n' > /tmp/wrongpass; chmod 644 /tmp/pass /tmp/wrongpass"
|
||||
}
|
||||
key_of() { run_on "$1" backup:status 2>&1 | sed -n 's/^Key: \([0-9a-f-]*\),.*/\1/p'; }
|
||||
no_key() { # container what
|
||||
local s
|
||||
s=$(run_on "$1" backup:status 2>&1 || true)
|
||||
expect "$s" "Key: not created" "$2: a key was left behind"
|
||||
[ -z "$(run_on "$1" backup:target:list 2>&1 | grep offsite || true)" ] || fail "$2: a location was left behind"
|
||||
}
|
||||
location() { echo "local -a null::null -o datadir=/mnt/shared/$1 --path=repo"; }
|
||||
# A fresh server with its own copy of the origin's location
|
||||
scenario() { # name
|
||||
docker exec ngb-rs-origin sh -c "rm -rf /mnt/shared/$1 && mkdir -p /mnt/shared/$1 && cp -a /mnt/shared/origin/repo /mnt/shared/$1/ && chown -R www-data /mnt/shared/$1"
|
||||
start ngb-rs-new
|
||||
docker exec -i ngb-rs-new sh -c 'cat > /tmp/kit.json && chmod 644 /tmp/kit.json' < "$SHARED/kit.json"
|
||||
}
|
||||
|
||||
# The origin: a key with a deletion delay of its own, a kit, one backup
|
||||
start ngb-rs-origin
|
||||
run_on ngb-rs-origin backup:key:init --passphrase-file=/tmp/pass --label=Safe --delete-delay-days=21 >/dev/null || fail "key init"
|
||||
run_on ngb-rs-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||
CODE=$(run_on ngb-rs-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-rs-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||
docker exec ngb-rs-origin sh -c 'mkdir -p /mnt/shared/origin && chown www-data /mnt/shared/origin'
|
||||
# shellcheck disable=SC2046
|
||||
run_on ngb-rs-origin backup:target:add offsite $(location origin) >/dev/null || fail "target add on the origin"
|
||||
run_on ngb-rs-origin backup:run offsite >/dev/null || fail "backup on the origin"
|
||||
SNAP=$(run_on ngb-rs-origin backup:list offsite | grep -o '[0-9a-f]\{32\}' | head -1)
|
||||
FP=$(key_of ngb-rs-origin)
|
||||
docker exec ngb-rs-origin cat /tmp/kit.json > "$SHARED/kit.json"
|
||||
docker exec ngb-rs-origin test -f /mnt/shared/origin/repo/settings || fail "the origin's location has no settings file"
|
||||
ok "origin: key $FP, kit, snapshot $SNAP, settings on the location"
|
||||
|
||||
# kit: the passphrase once, for the kit; target:add asks nothing more
|
||||
scenario kit
|
||||
OUT=$(run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "kit: key import"; }
|
||||
expect "$OUT" "fingerprint $FP" "kit"
|
||||
expect "$OUT" "21 days (from the kit)" "kit"
|
||||
expect "$OUT" "backup:target:add" "kit: the next step"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location kit) --no-interaction 2>&1) || { printf '%s\n' "$OUT"; fail "kit: target add"; }
|
||||
not_expect "$OUT" "Passphrase" "kit: target:add asked for a passphrase"
|
||||
expect "$OUT" "found and adopted" "kit"
|
||||
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "kit: the snapshot is not listed"
|
||||
run_on ngb-rs-new backup:run offsite >/dev/null || fail "kit: a backup after the kit import"
|
||||
ok "kit: imported with the passphrase, location added without asking again, backups run"
|
||||
|
||||
# kit-wrong
|
||||
scenario kit-wrong
|
||||
OUT=$(must_fail ngb-rs-new "kit-wrong" backup:key:import /tmp/kit.json --passphrase-file=/tmp/wrongpass)
|
||||
expect "$OUT" "Wrong passphrase" "kit-wrong"
|
||||
no_key ngb-rs-new "kit-wrong"
|
||||
ok "kit-wrong: refused, nothing left behind"
|
||||
|
||||
# no-kit
|
||||
scenario no-kit
|
||||
docker exec ngb-rs-new rm /tmp/kit.json
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-kit without a passphrase" backup:target:add offsite $(location no-kit) --no-interaction)
|
||||
expect "$OUT" "this location holds backups" "no-kit: explains where the key comes from"
|
||||
expect "$OUT" "--passphrase-file" "no-kit: says how to give the passphrase"
|
||||
no_key ngb-rs-new "no-kit without a passphrase"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-kit wrong passphrase" backup:target:add offsite $(location no-kit) --passphrase-file=/tmp/wrongpass)
|
||||
expect "$OUT" "Wrong passphrase" "no-kit wrong passphrase"
|
||||
expect "$OUT" "Nothing was changed" "no-kit wrong passphrase"
|
||||
no_key ngb-rs-new "no-kit wrong passphrase"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit: target add"; }
|
||||
expect "$OUT" "fingerprint $FP" "no-kit"
|
||||
expect "$OUT" "21 days (from the location)" "no-kit"
|
||||
expect "$OUT" "download a new recovery kit" "no-kit: asks for a new kit"
|
||||
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "no-kit: the snapshot is not listed"
|
||||
OUT=$(must_fail ngb-rs-new "no-kit backup before a new kit" backup:run offsite)
|
||||
expect "$OUT" "Confirm the recovery kit" "no-kit: a backup before a new kit"
|
||||
CODE=$(run_on ngb-rs-new backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-rs-new backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "no-kit: confirm the new kit"
|
||||
run_on ngb-rs-new backup:run offsite >/dev/null || fail "no-kit: a backup after the new kit"
|
||||
ok "no-kit: passphrase asked, wrong one changes nothing, key and delay from the location, backups wait for a new kit"
|
||||
|
||||
# no-kit-old: a location from before 1.1.0 has no settings file
|
||||
scenario no-kit-old
|
||||
docker exec ngb-rs-new rm /tmp/kit.json /mnt/shared/no-kit-old/repo/settings
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit-old) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit-old: target add"; }
|
||||
expect "$OUT" "fingerprint $FP" "no-kit-old"
|
||||
expect "$OUT" "7 days (the default" "no-kit-old: the default delay and why"
|
||||
docker exec ngb-rs-new test -f /mnt/shared/no-kit-old/repo/settings || fail "no-kit-old: the settings were not written when the location was added"
|
||||
ok "no-kit-old: key from the location, default delay with the reason, settings written now"
|
||||
|
||||
# no-config-no-kit
|
||||
scenario no-config-no-kit
|
||||
docker exec ngb-rs-new rm /tmp/kit.json /mnt/shared/no-config-no-kit/repo/config
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-config-no-kit" backup:target:add offsite $(location no-config-no-kit) --passphrase-file=/tmp/pass)
|
||||
expect "$OUT" "its config file is gone" "no-config-no-kit"
|
||||
expect "$OUT" "only the recovery kit has the key" "no-config-no-kit"
|
||||
expect "$OUT" "--repair-config" "no-config-no-kit"
|
||||
no_key ngb-rs-new "no-config-no-kit"
|
||||
docker exec ngb-rs-new test ! -f /mnt/shared/no-config-no-kit/repo/config || fail "no-config-no-kit: a config was written"
|
||||
ok "no-config-no-kit: told the kit is needed, nothing written"
|
||||
|
||||
# no-config-kit
|
||||
scenario no-config-kit
|
||||
docker exec ngb-rs-new rm /mnt/shared/no-config-kit/repo/config
|
||||
run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "no-config-kit: key import"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-config-kit without --repair-config" backup:target:add offsite $(location no-config-kit))
|
||||
expect "$OUT" "--repair-config" "no-config-kit: points at --repair-config"
|
||||
docker exec ngb-rs-new test ! -f /mnt/shared/no-config-kit/repo/config || fail "no-config-kit: a config was written without --repair-config"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-config-kit) --repair-config 2>&1) || { printf '%s\n' "$OUT"; fail "no-config-kit: --repair-config"; }
|
||||
expect "$OUT" "written again" "no-config-kit"
|
||||
expect "$OUT" "found and adopted" "no-config-kit"
|
||||
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "no-config-kit: the snapshot is not listed after the repair"
|
||||
run_on ngb-rs-new backup:browse offsite "$SNAP" >/dev/null 2>&1 || fail "no-config-kit: the snapshot cannot be read after the repair"
|
||||
ok "no-config-kit: refused without --repair-config, config written again with it, the backups open"
|
||||
|
||||
# other-kit: the kit of another server
|
||||
start ngb-rs-other
|
||||
run_on ngb-rs-other backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "other: key init"
|
||||
run_on ngb-rs-other backup:key:kit --output=/tmp/other.json >/dev/null
|
||||
docker exec ngb-rs-other cat /tmp/other.json > "$SHARED/other.json"
|
||||
docker rm -fv ngb-rs-other >/dev/null
|
||||
scenario other-kit
|
||||
docker exec ngb-rs-new rm /tmp/kit.json
|
||||
docker exec -i ngb-rs-new sh -c 'cat > /tmp/other.json && chmod 644 /tmp/other.json' < "$SHARED/other.json"
|
||||
run_on ngb-rs-new backup:key:import /tmp/other.json --passphrase-file=/tmp/pass >/dev/null || fail "other-kit: key import"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "other-kit" backup:target:add offsite $(location other-kit))
|
||||
expect "$OUT" "different key" "other-kit"
|
||||
expect "$OUT" "must be the old server's" "other-kit: points at the right kit"
|
||||
[ -z "$(run_on ngb-rs-new backup:target:list 2>&1 | grep offsite || true)" ] || fail "other-kit: a location was left behind"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "other-kit repair" backup:target:add offsite $(location other-kit) --repair-config)
|
||||
expect "$OUT" "leave out --repair-config" "other-kit: a config that is there is not repaired"
|
||||
ok "other-kit: refused, told to use the old server's kit, the config untouched"
|
||||
|
||||
# empty
|
||||
scenario empty
|
||||
docker exec ngb-rs-new sh -c 'mkdir -p /mnt/shared/nothing && chown www-data /mnt/shared/nothing'
|
||||
OUT=$(must_fail ngb-rs-new "empty" backup:target:add offsite local -a null::null -o datadir=/mnt/shared/nothing --path=repo --passphrase-file=/tmp/pass)
|
||||
expect "$OUT" "no backups at this location" "empty"
|
||||
expect "$OUT" "Check the connection details and --path" "empty"
|
||||
no_key ngb-rs-new "empty"
|
||||
ok "empty: told to check the details, no key made"
|
||||
|
||||
echo "RECOVERY SCENARIOS PASS"
|
||||
Reference in New Issue
Block a user