Full restore creates missing tables and suspends foreign keys; --install-apps; clearer messages

Backups now record each table's structure in the dump manifest. The full
restore creates tables that do not exist on the new instance (left behind
by apps that were removed or are switched off), lists all problems at
once, and no longer stops on apps that are missing. Foreign keys are not
checked while rows are loaded in name order, and checked again before the
restore commits (PostgreSQL: dropped and added back in the transaction).

--install-apps installs or enables the apps that were enabled at backup
time; --skip-missing-tables leaves out tables of older backups whose
structure is unknown. nextcloud-aio is never required.

backup:target:add without a key points to backup:key:import, and says that
connecting can take a while. Removing a location drops its cached index;
a migration removes the cache of locations that are gone.

The full-restore test now also runs on PostgreSQL with Mail (foreign keys)
and a disabled app.

.github/workflows/ci.yml

@@ -104,3 +104,5 @@ jobs:
           persist-credentials: false
       - name: Full restore round trip
         run: tests/integration/full-restore-roundtrip.sh 35-apache
+      - name: Full restore round trip on PostgreSQL, with Mail (foreign keys) and a disabled app
+        run: tests/integration/full-restore-roundtrip.sh 35-apache pgsql

PLAN.md

@@ -430,6 +430,9 @@ in README "Known limitations":
       terwijl OpenSSH `sftp` 16,6 MiB/s haalt. De SFTP-stream van files_external (phpseclib) wacht na
       elk klein blok op antwoord. Oplossing: pack-ranges bij SFTP met phpseclib's eigen `get()`
       (veel verzoeken tegelijk) lezen in plaats van via de stream, zoals bij S3. 1 TB duurt nu ~2 weken.
+      Na meten: de grootste oorzaak was het netwerk (cubic, zie 14): met BBR gaat de restore van 0,75
+      naar 2,3 MiB/s, een hele pack lezen 4-5 MiB/s. Meer verzoeken in één stream (DEPTH 16/32) gaf geen
+      duidelijke winst. Verder komen vraagt meerdere verbindingen tegelijk: naar **1.1.0**.
   14. **Uploaden over een lange lijn**: backup naar OVH S3 Warschau vanaf Canada (RTT 114 ms) haalt
       ~1 MiB/s met één pack tegelijk; `aws s3 cp` (multipart, parallel) 6 MiB/s. Meerdere packs tegelijk
       uploaden of S3 multipart. Minder dringend dan 13 (restore).

lib/Command/RestoreFull.php

@@ -23,6 +23,7 @@ final class RestoreFull extends Command {
 		private TargetService $targets,
 		private RestoreFullService $restoreFull,
 		private IUserManager $userManager,
+		private \OCP\App\IAppManager $appManager,
 	) {
 		parent::__construct();
 	}
@@ -33,14 +34,19 @@ final class RestoreFull extends Command {
 			->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
 			->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
 			->addOption('force', null, InputOption::VALUE_NONE, 'Also on an installation that already has more than one user, and without asking')
+			->addOption('install-apps', null, InputOption::VALUE_NONE, 'First install (from the app store) or enable the apps that were enabled at backup time and are missing here')
+			->addOption('skip-missing-tables', null, InputOption::VALUE_NONE, 'Older backups only: leave out tables that do not exist here and whose structure the backup does not record')
 			->setHelp(<<<'HELP'
 For a FRESH Nextcloud installation only, same major version as the backup, right after
 occ maintenance:install, occ backup:key:import and occ backup:target:add -- before anyone else
 uses this instance.
 
-Every app that was enabled when the snapshot was taken must already be installed here (its
-database tables must exist before the restore can write into them); the command lists what is
-missing and refuses to continue rather than guess.
+Apps: --install-apps first installs (from the app store) or enables every app that was enabled
+when the snapshot was taken. Apps that are not in the app store (own builds) must be copied into
+custom_apps by hand; the command lists what is still missing. Tables of apps that are missing or
+were removed long ago are created from the structure recorded in the backup (backups made with
+1.0.0-rc.2 or later), so they do not block the restore. Older backups do not record it: the
+command then lists those tables per app, and --skip-missing-tables leaves them out.
 
 Steps: the database is replaced (consistent, in one transaction); the data directory is restored
 directly in place; config.php is merged in -- everything from the backup except the database
@@ -75,7 +81,14 @@ HELP);
 		try {
 			$target = $this->targets->get($input->getArgument('target'));
 			$snapshot = (string)$input->getArgument('snapshot');
-			$result = $this->restoreFull->restoreFull($target, $snapshot);
+			$skipMissingTables = (bool)$input->getOption('skip-missing-tables');
+			if ($input->getOption('install-apps')) {
+				$check = $this->restoreFull->checkCompatibility($target, $snapshot, $skipMissingTables);
+				foreach ($check['missingApps'] as $app) {
+					$this->installApp($app, $output);
+				}
+			}
+			$result = $this->restoreFull->restoreFull($target, $snapshot, $skipMissingTables);
 		} catch (\Throwable $e) {
 			$output->writeln('<error>' . $e->getMessage() . '</error>');
 			$output->writeln('If this happened after the restore had started, the instance may be partly restored: fix the cause and run the same command again with --force.');
@@ -88,10 +101,43 @@ HELP);
 			$result['configKeys'],
 			$result['dataFiles'],
 		));
+		if ($result['dbReport']['created'] !== []) {
+			$output->writeln(sprintf('Created <info>%d</info> tables that did not exist here (apps that are missing or were removed): %s',
+				count($result['dbReport']['created']), implode(', ', $result['dbReport']['created'])));
+		}
+		if ($result['dbReport']['missing'] !== []) {
+			$output->writeln('<comment>Left out (structure unknown): ' . implode(', ', $result['dbReport']['missing']) . '</comment>');
+		}
+		if ($result['missingApps'] !== []) {
+			$output->writeln('<comment>These apps were enabled on the old server but are not installed here. Their data is restored; install them so they work again:</comment>');
+			foreach ($result['missingApps'] as $app) {
+				$output->writeln("  occ app:install $app   (not in the app store? copy it into custom_apps and run occ app:enable $app)");
+			}
+		}
 		if (!$result['dbConsistent']) {
 			$output->writeln('<comment>Warning: this snapshot\'s database dump was made over several steps (web cron), so it is not one consistent point in time. Check the instance carefully; prefer a snapshot made with system cron.</comment>');
 		}
 		$output->writeln('Next: check trusted_domains and overwritehost/overwriteprotocol match this server, then occ maintenance:repair.');
 		return 0;
 	}
+
+	/** Enable an app that is here but switched off, or install it from the app store. */
+	private function installApp(string $app, OutputInterface $output): void {
+		try {
+			$this->appManager->getAppPath($app);
+			$command = 'app:enable'; // here, but switched off
+		} catch (\OCP\App\AppPathNotFoundException) {
+			$command = 'app:install';
+		}
+		$output->writeln("Running occ $command $app ...");
+		try {
+			$code = $this->getApplication()?->find($command)->run(new \Symfony\Component\Console\Input\ArrayInput(['app-id' => $command === 'app:enable' ? [$app] : $app]), $output);
+		} catch (\Throwable $e) {
+			$code = 1;
+			$output->writeln('<comment>' . $e->getMessage() . '</comment>');
+		}
+		if ($code !== 0) {
+			$output->writeln("<comment>Could not $command $app (not in the app store?). Copy it into custom_apps and enable it, or restore without it.</comment>");
+		}
+	}
 }

lib/Command/TargetAdd.php

@@ -44,7 +44,10 @@ HELP);
 
 	protected function execute(InputInterface $input, OutputInterface $output): int {
 		if (!$this->keys->isInitialized()) {
-			$output->writeln('<error>No backup key yet: run occ backup:key:init first.</error>');
+			$output->writeln('<error>No backup key yet.</error>');
+			$output->writeln('Restoring from a location that already has backups (disaster recovery)? Import the key from the recovery kit first:');
+			$output->writeln('  occ backup:key:import /path/to/ng_backup-kit.json');
+			$output->writeln('Only for a new location without backups: create a new key with occ backup:key:init. A new key cannot read existing backups.');
 			return 1;
 		}
 		$options = [];
@@ -60,6 +63,9 @@ HELP);
 			$output->writeln('<error>--auth is required (see occ backup:target:backends)</error>');
 			return 1;
 		}
+		// Adopting an existing repository reads its whole index first, which takes a while on a
+		// large backup or a slow line; say so instead of looking stuck.
+		$output->writeln('Connecting to the location; with existing backups this reads their index first and can take a while ...');
 		try {
 			$r = $this->targets->add($input->getArgument('name'), $input->getArgument('backend'), $input->getOption('auth'), $options, (string)$input->getOption('path'));
 		} catch (\Throwable $e) {

lib/Db/DbDumper.php

@@ -23,8 +23,12 @@ use OCA\NgBackup\Repository\Repository;
  *   sets, so a single big SELECT would not stream).
  * - Each table becomes its own blob stream of JSON lines: a header line, then one JSON array per
  *   row. A table that did not change produces the same blobs and costs nothing in the next run.
- * - The schema itself is not dumped: it comes from Nextcloud's migrations on the instance that is
- *   restored into (same Nextcloud and app versions). The dump records those versions.
+ * - The structure of each table (columns, types, primary key, indexes) is recorded in the manifest,
+ *   so a restore can create a table that does not exist on the new instance, for example one left
+ *   behind by an app that was removed long ago. Tables that exist keep the structure the new
+ *   instance's migrations gave them.
+ * - Foreign keys are not checked while rows are loaded (tables are restored in name order, not in
+ *   dependency order), and are checked again before the restore commits.
  *
  * Phase 0 uses the Doctrine connection behind IDBConnection (private API) for schema
  * introspection; to be replaced by information_schema queries or a public API.
@@ -108,7 +112,7 @@ final class DbDumper {
 						break;
 					}
 					$state['result'][$table] = ['blobs' => array_merge(...$state['segments']), 'rows' => $state['rows'],
-						'bytes' => $state['bytes'], 'sha256' => hash_final($hash)];
+						'bytes' => $state['bytes'], 'sha256' => hash_final($hash), 'schema' => self::structure($schema)];
 					$state = array_merge($state, ['i' => $state['i'] + 1, 'after' => null, 'segments' => [], 'rows' => 0, 'bytes' => 0, 'hash' => null]);
 					if (microtime(true) >= $deadline && $state['i'] < count($state['tables'])) {
 						break;
@@ -137,18 +141,33 @@ final class DbDumper {
 	 * Replace the contents of every table in the dump. Tables that exist but are not in the dump
 	 * are left alone (e.g. a newer app); the caller decides whether that is acceptable.
 	 *
-	 * @param array{tables: array<string, array{blobs: list<string>, rows: int}>} $manifest
+	 * @param array{tables: array<string, array{blobs: list<string>, rows: int, sha256: string, schema?: array}>, sequences?: array} $manifest
 	 */
-	public function restore(Repository $repo, array $manifest, ?array &$report = null, ?callable $heartbeat = null): int {
-		$sm = $this->db->createSchemaManager();
-		$existing = array_flip($sm->listTableNames());
+	public function restore(Repository $repo, array $manifest, ?array &$report = null, ?callable $heartbeat = null, bool $skipMissing = false): int {
+		$report = ['skipped' => [], 'restored' => [], 'created' => [], 'missing' => []];
+		$missing = $this->missingTables($manifest);
+		$unknown = array_values(array_filter($missing, fn (string $t) => !isset($manifest['tables'][$t]['schema'])));
+		if ($unknown !== [] && !$skipMissing) {
+			throw new \RuntimeException(self::describeMissing($unknown, $this->prefix));
+		}
+		// Tables the backup knows the structure of are created first (DDL is not transactional
+		// everywhere); tables it does not know are left out, as asked.
+		foreach ($missing as $table) {
+			if (isset($manifest['tables'][$table]['schema'])) {
+				$this->db->createSchemaManager()->createTable(self::tableFromStructure($table, $manifest['tables'][$table]['schema']));
+				$report['created'][] = $table;
+			} else {
+				$report['missing'][] = $table;
+			}
+		}
 		$total = 0;
-		$report = ['skipped' => [], 'restored' => []];
+		$foreignKeys = [];
 		$this->db->beginTransaction();
 		try {
+			$foreignKeys = $this->suspendForeignKeys();
 			foreach ($manifest['tables'] as $table => $info) {
-				if (!isset($existing[$table])) {
-					throw new \RuntimeException("Table $table from the backup does not exist here (different app versions?)");
+				if (in_array($table, $report['missing'], true)) {
+					continue;
 				}
 				// Unchanged since the backup: leave it alone (much faster than delete + insert).
 				$live = $this->checksum($table);
@@ -193,14 +212,130 @@ final class DbDumper {
 				$report['restored'][] = $table;
 			}
 			$this->restoreSequences($manifest['sequences'] ?? []);
+			$this->resumeForeignKeys($foreignKeys);
 			$this->db->commit();
 		} catch (\Throwable $e) {
 			$this->db->rollBack();
+			// PostgreSQL rolls the dropped foreign keys back with the transaction; MySQL's session
+			// switch must be turned on again by hand.
+			if (in_array('SET FOREIGN_KEY_CHECKS = 1', $foreignKeys, true)) {
+				$this->db->executeStatement('SET FOREIGN_KEY_CHECKS = 1');
+			}
 			throw $e;
 		}
 		return $total;
 	}
 
+	/**
+	 * Tables in the dump that do not exist on this instance.
+	 *
+	 * @return list<string>
+	 */
+	public function missingTables(array $manifest): array {
+		$existing = array_flip($this->db->createSchemaManager()->listTableNames());
+		return array_values(array_filter(array_keys($manifest['tables']), fn (string $t) => !isset($existing[$t])));
+	}
+
+	/** One message for all missing tables of an older backup, grouped by the app they probably belong to. */
+	public static function describeMissing(array $tables, string $prefix): string {
+		$groups = [];
+		foreach ($tables as $t) {
+			$name = substr($t, strlen($prefix));
+			$groups[explode('_', $name, 2)[0]][] = $t;
+		}
+		$lines = [];
+		foreach ($groups as $app => $list) {
+			$lines[] = '- ' . $app . ': ' . implode(', ', $list);
+		}
+		return "These tables from the backup do not exist here, and this backup does not record their structure (made before 1.0.0-rc.2):\n"
+			. implode("\n", $lines)
+			. "\nInstall the apps they belong to (often apps that were removed or switched off long ago) and run the restore again, or leave these tables out with --skip-missing-tables.";
+	}
+
+	/**
+	 * Portable description of a table: enough to create it again, also on another database type.
+	 *
+	 * @return array{columns: list<array<string, mixed>>, pk: list<string>, indexes: list<array{name: string, columns: list<string>, unique: bool}>}
+	 */
+	private static function structure(\Doctrine\DBAL\Schema\Table $table): array {
+		$columns = [];
+		foreach ($table->getColumns() as $c) {
+			$columns[] = ['name' => $c->getName(), 'type' => \Doctrine\DBAL\Types\Type::lookupName($c->getType()),
+				'length' => $c->getLength(), 'precision' => $c->getPrecision(), 'scale' => $c->getScale(),
+				'unsigned' => $c->getUnsigned(), 'fixed' => $c->getFixed(), 'notnull' => $c->getNotnull(),
+				'default' => $c->getDefault(), 'autoincrement' => $c->getAutoincrement()];
+		}
+		$indexes = [];
+		foreach ($table->getIndexes() as $i) {
+			if (!$i->isPrimary()) {
+				$indexes[] = ['name' => $i->getName(), 'columns' => array_values($i->getColumns()), 'unique' => $i->isUnique()];
+			}
+		}
+		return ['columns' => $columns, 'pk' => array_values($table->getPrimaryKey()?->getColumns() ?? []), 'indexes' => $indexes];
+	}
+
+	private static function tableFromStructure(string $name, array $structure): \Doctrine\DBAL\Schema\Table {
+		$table = new \Doctrine\DBAL\Schema\Table($name);
+		foreach ($structure['columns'] as $c) {
+			$table->addColumn($c['name'], $c['type'], array_filter([
+				'length' => $c['length'], 'precision' => $c['precision'], 'scale' => $c['scale'],
+				'unsigned' => $c['unsigned'], 'fixed' => $c['fixed'], 'notnull' => $c['notnull'],
+				'default' => $c['default'], 'autoincrement' => $c['autoincrement'],
+			], fn ($v) => $v !== null));
+		}
+		if ($structure['pk'] !== []) {
+			$table->setPrimaryKey($structure['pk']);
+		}
+		foreach ($structure['indexes'] as $i) {
+			$i['unique'] ? $table->addUniqueIndex($i['columns'], $i['name']) : $table->addIndex($i['columns'], $i['name']);
+		}
+		return $table;
+	}
+
+	/**
+	 * Stop checking foreign keys while rows are loaded in name order. PostgreSQL has no session
+	 * switch for this without superuser rights, so its foreign keys of Nextcloud tables are dropped
+	 * here and added back (and so checked) by resumeForeignKeys(), inside the same transaction.
+	 *
+	 * @return list<string> statements that restore the foreign keys
+	 */
+	private function suspendForeignKeys(): array {
+		$platform = $this->db->getDatabasePlatform();
+		if ($platform instanceof \Doctrine\DBAL\Platforms\PostgreSQLPlatform) {
+			$rows = $this->db->fetchAllAssociative(
+				"SELECT conrelid::regclass::text AS tbl, conname, pg_get_constraintdef(oid) AS def
+				 FROM pg_constraint WHERE contype = 'f' AND connamespace = current_schema()::regnamespace");
+			$restore = [];
+			foreach ($rows as $r) {
+				if (!str_starts_with(trim($r['tbl'], '"'), $this->prefix)) {
+					continue;
+				}
+				$this->db->executeStatement(sprintf('ALTER TABLE %s DROP CONSTRAINT %s', $r['tbl'], $this->db->quoteIdentifier($r['conname'])));
+				$restore[] = sprintf('ALTER TABLE %s ADD CONSTRAINT %s %s', $r['tbl'], $this->db->quoteIdentifier($r['conname']), $r['def']);
+			}
+			return $restore;
+		}
+		if ($platform instanceof \Doctrine\DBAL\Platforms\AbstractMySQLPlatform) {
+			$this->db->executeStatement('SET FOREIGN_KEY_CHECKS = 0');
+			return ['SET FOREIGN_KEY_CHECKS = 1'];
+		}
+		if ($platform instanceof \Doctrine\DBAL\Platforms\SqlitePlatform) {
+			$this->db->executeStatement('PRAGMA defer_foreign_keys = ON'); // checked at commit
+		}
+		return [];
+	}
+
+	/** Add the foreign keys back; a row that breaks one fails the restore here, before it commits. */
+	private function resumeForeignKeys(array $statements): void {
+		foreach ($statements as $sql) {
+			try {
+				$this->db->executeStatement($sql);
+			} catch (\Throwable $e) {
+				throw new \RuntimeException('The restored data breaks a foreign key (' . $sql . '): ' . $e->getMessage(), 0, $e);
+			}
+		}
+	}
+
 	/** Same checksum as dump(): sha256 over the row lines in primary-key order. */
 	public function checksum(string $table): array {
 		$hash = hash_init('sha256');

lib/Db/DbIndexCache.php

@@ -64,6 +64,13 @@ final class DbIndexCache implements IIndexCache {
 		}
 	}
 
+	/** Drop the cached index of a repository that no location uses any more. */
+	public function forget(string $repositoryId): void {
+		$qb = $this->db->getQueryBuilder();
+		$qb->delete('ngb_index')->where($qb->expr()->eq('repository_id', $qb->createNamedParameter($repositoryId)));
+		$qb->executeStatement();
+	}
+
 	/** All or nothing: when it fails, the cache stays as it was. */
 	public function replaceAll(string $repositoryId, array $packs): void {
 		$this->transaction(function () use ($repositoryId, $packs): void {

lib/Migration/Version000006Date20261018000000.php

@@ -0,0 +1,39 @@
+<?php
+
+declare(strict_types=1);
+
+// SPDX-FileCopyrightText: 2026 Dillard Blom
+// SPDX-License-Identifier: AGPL-3.0-or-later
+
+namespace OCA\NgBackup\Migration;
+
+use Closure;
+use OCP\IDBConnection;
+use OCP\Migration\IOutput;
+use OCP\Migration\SimpleMigrationStep;
+
+/**
+ * Removing a location left the cached index of its repository behind (tens of thousands of rows
+ * for a large backup). Drop the cache of repositories no location uses any more.
+ */
+final class Version000006Date20261018000000 extends SimpleMigrationStep {
+	public function __construct(
+		private IDBConnection $db,
+	) {
+	}
+
+	public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options): void {
+		$used = $this->db->getQueryBuilder();
+		$used->select('repository_id')->from('ngb_targets')->where($used->expr()->isNotNull('repository_id'));
+		$keep = array_column($used->executeQuery()->fetchAll(), 'repository_id');
+		$delete = $this->db->getQueryBuilder();
+		$delete->delete('ngb_index');
+		if ($keep !== []) {
+			$delete->where($delete->expr()->notIn('repository_id', $delete->createNamedParameter($keep, \OCP\DB\QueryBuilder\IQueryBuilder::PARAM_STR_ARRAY)));
+		}
+		$removed = $delete->executeStatement();
+		if ($removed > 0) {
+			$output->info("Removed $removed cached index rows of locations that no longer exist");
+		}
+	}
+}

lib/Service/RestoreFullService.php

@@ -43,34 +43,49 @@ final class RestoreFullService {
 	) {
 	}
 
+	/** Apps that only exist in a particular setup and are never needed elsewhere. */
+	private const ENVIRONMENT_APPS = ['nextcloud-aio'];
+
 	/**
-	 * Whether this installation can receive $snapshotId: same Nextcloud major version, and every
-	 * app that was enabled at backup time already installed here (its tables must exist before
-	 * the database restore can write into them).
+	 * Whether this installation can receive $snapshotId, all problems at once:
+	 * - the same Nextcloud major version is required;
+	 * - apps that were enabled at backup time but are not here (they work again once installed);
+	 * - tables in the backup that do not exist here: a backup made from 1.0.0-rc.2 on records
+	 *   their structure and the restore creates them, an older one cannot (missingTables).
 	 *
-	 * @return array{ok:bool, currentMajor:string, snapshotMajor:string, missingApps:list<string>}
+	 * @return array{ok:bool, currentMajor:string, snapshotMajor:string, missingApps:list<string>, missingTables:list<string>}
 	 */
-	public function checkCompatibility(Target $target, string $snapshotId): array {
-		$meta = $this->targets->repository($target)->snapshotMeta($snapshotId);
+	public function checkCompatibility(Target $target, string $snapshotId, bool $skipMissingTables = false): array {
+		$repo = $this->targets->repository($target);
+		$meta = $repo->snapshotMeta($snapshotId);
 		$currentMajor = explode('.', $this->config->getSystemValueString('version'))[0];
 		$snapshotMajor = explode('.', (string)($meta['nextcloud'] ?? ''))[0];
 		$enabled = array_flip($this->appManager->getEnabledApps());
-		$missingApps = array_values(array_diff(array_keys($meta['apps'] ?? []), array_keys($enabled)));
+		$missingApps = array_values(array_diff(array_keys($meta['apps'] ?? []), array_keys($enabled), self::ENVIRONMENT_APPS));
+		$missingTables = [];
+		if (isset($meta['db'])) {
+			$manifest = json_decode($repo->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR);
+			$prefix = $this->config->getSystemValueString('dbtableprefix', 'oc_');
+			$dumper = new DbDumper($this->db->getInner(), $prefix);
+			$missingTables = array_values(array_filter($dumper->missingTables($manifest),
+				fn (string $t) => !isset($manifest['tables'][$t]['schema']) && !str_starts_with($t, $prefix . 'ngb_')));
+		}
 		return [
-			'ok' => $currentMajor === $snapshotMajor && $missingApps === [],
+			'ok' => $currentMajor === $snapshotMajor && ($missingTables === [] || $skipMissingTables),
 			'currentMajor' => $currentMajor,
 			'snapshotMajor' => $snapshotMajor,
 			'missingApps' => $missingApps,
+			'missingTables' => $missingTables,
 		];
 	}
 
-	/** @return array{dbReport:array, dataFiles:int, configKeys:int, dbConsistent:bool} */
-	public function restoreFull(Target $target, string $snapshotId): array {
-		$check = $this->checkCompatibility($target, $snapshotId);
+	/** @return array{dbReport:array, dataFiles:int, configKeys:int, dbConsistent:bool, missingApps:list<string>} */
+	public function restoreFull(Target $target, string $snapshotId, bool $skipMissingTables = false): array {
+		$check = $this->checkCompatibility($target, $snapshotId, $skipMissingTables);
 		if (!$check['ok']) {
 			throw new \RuntimeException($this->describeBlockers($check));
 		}
-		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId) {
+		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $skipMissingTables, $check) {
 			$beat = LeaseService::throttled($refresh);
 			$repo = $this->targets->repository($target);
 			$meta = $repo->snapshotMeta($snapshotId);
@@ -102,7 +117,7 @@ final class RestoreFullService {
 				$ownAppConfig = $this->preserveOwnAppConfig();
 				$dumper = new DbDumper($this->db->getInner(), $prefix);
 				try {
-					$dumper->restore($repo, $manifest, $dbReport, $beat);
+					$dumper->restore($repo, $manifest, $dbReport, $beat, $skipMissingTables);
 				} finally {
 					$this->restoreOwnAppConfig($ownAppConfig);
 				}
@@ -125,8 +140,8 @@ final class RestoreFullService {
 				);
 			}
 
-			return ['dbReport' => $dbReport ?? ['skipped' => [], 'restored' => []], 'dataFiles' => $dataFiles, 'configKeys' => count($backupConfig),
-				'dbConsistent' => (bool)($manifest['consistent'] ?? true)];
+			return ['dbReport' => $dbReport ?? ['skipped' => [], 'restored' => [], 'created' => [], 'missing' => []], 'dataFiles' => $dataFiles, 'configKeys' => count($backupConfig),
+				'dbConsistent' => (bool)($manifest['consistent'] ?? true), 'missingApps' => $check['missingApps']];
 		});
 	}
 
@@ -209,11 +224,11 @@ final class RestoreFullService {
 		if ($check['currentMajor'] !== $check['snapshotMajor']) {
 			$parts[] = "this installation is Nextcloud {$check['currentMajor']}, the backup is from {$check['snapshotMajor']} (same major version required)";
 		}
-		if ($check['missingApps'] !== []) {
-			$parts[] = 'install these apps first (their tables must exist before the database restore can use them): '
-				. implode(', ', array_map(fn (string $a) => "occ app:install $a", $check['missingApps']));
+		if ($check['missingTables'] !== []) {
+			$prefix = $this->config->getSystemValueString('dbtableprefix', 'oc_');
+			$parts[] = DbDumper::describeMissing($check['missingTables'], $prefix);
 		}
-		return implode('; ', $parts);
+		return implode("\n", $parts);
 	}
 
 	private function withSharedLock(Target $target, callable $fn): mixed {

lib/Service/TargetService.php

@@ -259,6 +259,11 @@ final class TargetService {
 	/** Remove the target from NG Backup. The backups on the target itself are not touched. */
 	public function remove(Target $target): void {
 		$this->mapper->delete($target);
+		// The cached index of its repository is only of use while a location points at it
+		$repositoryId = $target->getRepositoryId();
+		if ($repositoryId !== null && !array_filter($this->list(), fn (Target $t) => $t->getRepositoryId() === $repositoryId)) {
+			$this->indexCache->forget($repositoryId);
+		}
 	}
 
 	public function backend(Target $target): IBackend {

psalm-baseline.xml

@@ -153,10 +153,6 @@
     </PossiblyFalseArgument>
   </file>
   <file src="lib/Db/DbDumper.php">
-    <InvalidArrayOffset>
-      <code><![CDATA[$info['sha256']]]></code>
-      <code><![CDATA[$manifest['sequences']]]></code>
-    </InvalidArrayOffset>
     <MissingClosureReturnType>
       <code><![CDATA[fn ($c) => $end[array_search($c, $columns, true)]]]></code>
       <code><![CDATA[fn ($c) => $q($c)]]></code>
@@ -189,6 +185,10 @@
       <code><![CDATA[$this->db]]></code>
       <code><![CDATA[$this->db]]></code>
       <code><![CDATA[$this->db]]></code>
+      <code><![CDATA[$this->db]]></code>
+      <code><![CDATA[$this->db]]></code>
+      <code><![CDATA[$this->db]]></code>
+      <code><![CDATA[$this->db]]></code>
       <code><![CDATA[TransactionIsolationLevel]]></code>
       <code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
       <code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
@@ -196,6 +196,10 @@
       <code><![CDATA[\Doctrine\DBAL\Platforms\PostgreSQLPlatform]]></code>
       <code><![CDATA[\Doctrine\DBAL\Platforms\PostgreSQLPlatform]]></code>
       <code><![CDATA[\Doctrine\DBAL\Platforms\PostgreSQLPlatform]]></code>
+      <code><![CDATA[\Doctrine\DBAL\Platforms\PostgreSQLPlatform]]></code>
+      <code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
+      <code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
+      <code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
       <code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
       <code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
       <code><![CDATA[\Doctrine\DBAL\Schema\Table]]></code>
@@ -438,6 +442,7 @@
     </EmptyArrayAccess>
     <UndefinedInterfaceMethod>
       <code><![CDATA[getInner]]></code>
+      <code><![CDATA[getInner]]></code>
     </UndefinedInterfaceMethod>
     <UnresolvableInclude>
       <code><![CDATA[include $path]]></code>

tests/integration/full-restore-roundtrip.sh

@@ -2,10 +2,14 @@
 # Disaster recovery: occ backup:key:import + backup:target:add + backup:restore:full on a
 # SECOND, fresh Nextcloud instance that never saw the first one, sharing only the backup
 # location (a local directory, bind-mounted into both containers) -- see PLAN.md 3.7.
-# Usage: full-restore-roundtrip.sh <nextcloud-image-tag>
-#   e.g. full-restore-roundtrip.sh 35-apache
+# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql]
+#   e.g. full-restore-roundtrip.sh 35-apache pgsql
+# With pgsql the origin also has what broke real disaster recoveries: the Mail app with a synced
+# account (foreign keys between its tables) and Deck installed and then switched off (tables of
+# an app that is not enabled). The restore instance has neither; --install-apps adds Mail.
 set -euo pipefail
 IMAGE="nextcloud:${1:-35-apache}"
+DB="${2:-sqlite}"
 APP=$(cd "$(dirname "$0")/../.." && pwd)
 NET=ngb-frt-net
 SHARED=$(mktemp -d)
@@ -14,7 +18,7 @@ fail() { echo "FAIL: $*"; exit 1; }
 ok() { echo "ok   $*"; }
 
 cleanup() {
-	docker rm -fv ngb-frt-origin ngb-frt-restore >/dev/null 2>&1 || true
+	docker rm -fv ngb-frt-origin ngb-frt-restore ngb-frt-db-origin ngb-frt-db-restore ngb-frt-imap >/dev/null 2>&1 || true
 	docker network rm "$NET" >/dev/null 2>&1 || true
 	docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
 	rmdir "$SHARED" 2>/dev/null || true
@@ -24,14 +28,28 @@ trap cleanup EXIT
 
 run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
 
+# Database settings for an instance: SQLite inside the container, or its own PostgreSQL container
+db_env() {
+	if [ "$DB" = pgsql ]; then
+		docker run -d --name "ngb-frt-db-$1" --network "$NET" -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nextcloud \
+			-e POSTGRES_PASSWORD=nextcloud postgres:16-alpine >/dev/null
+		for i in $(seq 1 30); do docker exec "ngb-frt-db-$1" pg_isready -q 2>/dev/null && break; sleep 2; done
+		echo "-e POSTGRES_HOST=ngb-frt-db-$1 -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nextcloud -e POSTGRES_PASSWORD=nextcloud"
+	else
+		echo "-e SQLITE_DATABASE=nextcloud"
+	fi
+}
+
 docker network create "$NET" >/dev/null
-docker run -d --name ngb-frt-origin --network "$NET" -e SQLITE_DATABASE=nextcloud \
+ORIGIN_DB=$(db_env origin)
+# shellcheck disable=SC2086
+docker run -d --name ngb-frt-origin --network "$NET" $ORIGIN_DB \
 	-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin \
 	-v "$APP:/var/www/html/custom_apps/ng_backup:ro" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
 for i in $(seq 1 90); do run_on ngb-frt-origin status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
 docker exec ngb-frt-origin chown www-data:www-data /var/www/html/custom_apps
 run_on ngb-frt-origin app:enable ng_backup >/dev/null
-ok "origin instance up"
+ok "origin instance up ($DB)"
 
 docker exec ngb-frt-origin sh -c "printf 'full restore roundtrip passphrase 2026\n' > /tmp/pass"
 run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI >/dev/null || fail "key init"
@@ -45,6 +63,26 @@ NC_PASS=$(openssl rand -hex 16)
 docker exec -u www-data -e OC_PASS="$NC_PASS" ngb-frt-origin php occ user:add --password-from-env frtest >/dev/null
 docker exec ngb-frt-origin sh -c "mkdir -p /var/www/html/data/frtest/files/Docs && head -c 200000 /dev/urandom > /var/www/html/data/frtest/files/Docs/report.bin && echo 'hello disaster recovery' > /var/www/html/data/frtest/files/Docs/note.txt && chown -R www-data:www-data /var/www/html/data/frtest"
 run_on ngb-frt-origin files:scan frtest >/dev/null
+
+if [ "$DB" = pgsql ]; then
+	# Mail with a synced account: oc_mail_accounts points at oc_mail_mailboxes (foreign keys),
+	# while the restore loads tables in name order, accounts before mailboxes.
+	docker run -d --name ngb-frt-imap --network "$NET" -e MAILNAME=mail.domain.tld -e MAIL_ADDRESS=user@domain.tld \
+		-e MAIL_PASS=mypassword ghcr.io/christophwurst/docker-imap-devel:latest >/dev/null
+	run_on ngb-frt-origin app:install mail >/dev/null || fail "install mail on the origin"
+	run_on ngb-frt-origin config:system:set app.mail.verify-tls-peer --type=boolean --value=false >/dev/null
+	sleep 10
+	run_on ngb-frt-origin mail:account:create frtest Test user@domain.tld ngb-frt-imap 993 ssl user@domain.tld mypassword \
+		ngb-frt-imap 25 tls user@domain.tld mypassword >/dev/null || fail "mail account"
+	run_on ngb-frt-origin mail:account:sync -f 1 >/dev/null || fail "mail sync"
+	[ "$(docker exec ngb-frt-db-origin psql -U nextcloud -d nextcloud -Atc 'select count(*) from oc_mail_accounts where drafts_mailbox_id is not null')" = 1 ] \
+		|| fail "the mail account does not point at its mailboxes"
+	# Deck installed and then switched off: its tables stay in the database and in the backup
+	run_on ngb-frt-origin app:install deck >/dev/null || fail "install deck on the origin"
+	run_on ngb-frt-origin app:disable deck >/dev/null
+	ok "origin has a synced Mail account and the tables of a disabled app"
+fi
+
 RUN_OUT=$(run_on ngb-frt-origin backup:run offsite -l disaster-test)
 SNAP=$(printf '%s\n' "$RUN_OUT" | head -1 | awk -F'snapshot ' '{print $2}' | tr -d ',')
 [ -n "$SNAP" ] || fail "no snapshot id from backup:run"
@@ -52,7 +90,9 @@ ok "test data backed up, snapshot $SNAP"
 
 docker cp ngb-frt-origin:/tmp/kit.json /tmp/ngb-frt-kit.json
 docker exec ngb-frt-origin sh -c "cat /tmp/pass" > /tmp/ngb-frt-pass
-docker run -d --name ngb-frt-restore --network "$NET" -e SQLITE_DATABASE=nextcloud \
+RESTORE_DB=$(db_env restore)
+# shellcheck disable=SC2086
+docker run -d --name ngb-frt-restore --network "$NET" $RESTORE_DB \
 	-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin \
 	-v "$APP:/var/www/html/custom_apps/ng_backup:ro" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
 for i in $(seq 1 90); do run_on ngb-frt-restore status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
@@ -68,7 +108,7 @@ run_on ngb-frt-restore backup:target:add offsite local -a null::null -o datadir=
 run_on ngb-frt-restore backup:list offsite | grep -q "$SNAP" || fail "the origin's snapshot is not listed after connecting the location"
 ok "recovery kit imported, existing repository adopted, snapshot visible"
 
-OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" --no-interaction 2>&1) || {
+OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" --no-interaction --install-apps 2>&1) || {
 	printf '%s\n' "$OUT" | tail -40
 	docker exec ngb-frt-restore sh -c 'tail -c 6000 /var/www/html/data/nextcloud.log' || true
 	fail "restore:full"
@@ -84,7 +124,17 @@ docker exec ngb-frt-restore test -f /var/www/html/data/frtest/files/Docs/note.tx
 run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after restore"
 ok "database, files and ng_backup's own state are all intact and usable after the restore"
 
+if [ "$DB" = pgsql ]; then
+	Q() { docker exec ngb-frt-db-restore psql -U nextcloud -d nextcloud -Atc "$1"; }
+	[ "$(Q "select count(*) from pg_tables where tablename = 'oc_deck_boards'")" = 1 ] || fail "the tables of the disabled app were not created"
+	[ "$(Q "select count(*) from oc_mail_accounts where drafts_mailbox_id is not null")" = 1 ] || fail "the mail account was not restored"
+	[ "$(Q "select count(*) from pg_constraint where contype = 'f' and conrelid::regclass::text like 'oc_mail_%'")" -gt 0 ] \
+		|| fail "the foreign keys of Mail are missing after the restore"
+	printf '%s\n' "$OUT" | grep -q "Created .* tables" || fail "the restore did not report the tables it created"
+	ok "tables of a disabled app created, Mail restored with its foreign keys"
+fi
+
 run_on ngb-frt-restore backup:run offsite -l post-restore-check >/dev/null || fail "a new backup after the restore fails"
 ok "the restored instance can run a new backup of its own"
 
-echo "FULL RESTORE ROUNDTRIP PASS"
+echo "FULL RESTORE ROUNDTRIP PASS ($DB)"