PLAN: 1.0.2 released; a branch per version

Merge pull request #28 from dillardblom/release-1.0.2
Release 1.0.2
2026-11-06 11:29:57 +01:00 · 2026-11-06 11:29:14 +01:00 · 2026-11-06 11:03:14 +01:00 · 2026-11-06 10:42:16 +01:00 · 2026-11-06 10:27:41 +01:00 · 2026-11-05 08:35:46 +01:00
35 changed files with 1529 additions and 197 deletions
@@ -138,3 +138,7 @@ jobs:
        run: tests/integration/full-restore-roundtrip.sh 35-apache mysql to-sqlite
      - name: Full restore of a SQLite backup onto PostgreSQL
        run: tests/integration/full-restore-roundtrip.sh 35-apache sqlite to-pgsql
+      - name: Full restore with a damaged database dump (--salvage)
+        run: tests/integration/full-restore-roundtrip.sh 35-apache sqlite salvage
+      - name: Full restore with a damaged and a missing pack (--salvage)
+        run: tests/integration/full-restore-roundtrip.sh 35-apache sqlite salvage-files
@@ -11,6 +11,51 @@ Each heading starts with that `<version>`, once: the app store shows a release t
 whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
 shown under 1.0.0. The git tag of a release candidate goes after the date.

+## [1.0.2] - 2026-11-06
+
+### Added
+- A warning on the admin page and in `occ backup:status` when background jobs do not run on a
+  system cron, or the cron stopped, with a link to how to set one up. Backups, also the scheduled
+  ones, need it (#13).
+- Without a working system cron, a backup started with "Back up now" moves on while the NG Backup
+  page stays open (#13).
+- `--salvage` for `occ backup:restore` and `occ backup:restore:full`: restore whatever can be
+  read when a backup is damaged. Files that cannot be read or decrypted are left out and listed,
+  and the command exits with code 2. When the database cannot be restored, `backup:restore:full`
+  leaves the fresh database as it is and puts the data directory and config of the backup in
+  `ngb-salvage-<snapshot>` in the data directory, to put back by hand.
+- The trash of restore points has a tab of its own, "Bin", instead of being shown below the
+  backups.
+
+### Changed
+- A backup in its database phase shows how far it is, for example "running (db: 47/183 tables)",
+  and one that has not moved for 15 minutes says "waiting for background jobs" with when it last
+  took a step. Before, it only said "running (db)", which looked the same as being stuck (#13).
+- Full restores from S3 read the packs ahead, eight ranges at a time, as restores over SFTP do
+  since 1.0.0-rc.4. Over SMB the packs are read in 8 MiB reads instead of 1 MiB: over a line with
+  a 130 ms round trip, reading was four times faster in a test (18 instead of 4.5 MiB/s).
+- Over SMB the index files, which a new server reads in `backup:target:add`, take fewer round
+  trips: from Canada to a Hetzner Storage Box (130 ms), 0.66 instead of 1.05 s per file, so about
+  5 instead of 8 hours for the 27,000 index files of a 1 TB repository.
+- `occ backup:restore:full` that runs again after it stopped (`--force`) goes on where it was: data
+  files already in place with the size and modification time of the backup are left, and only
+  the rest is read from the location. Before, a stopped restore of 1 TB started over.
+
+### Fixed
+- A full restore over SMB could hang at 100% CPU when the connection to the server dropped: a
+  failed read was counted as data. It is now read again over a new connection. Found in a
+  disaster recovery test of 1 TB from Canada to a Hetzner Storage Box, after 392 GiB.
+- The read-ahead folder of a restore that was killed while it wrote is removed by the next run;
+  it kept one file of up to 8 MiB.
+- `occ backup:target:add` only estimates the time left for reading the index once enough of it
+  is in. The first file also carries the time to connect, which made it say "about 642 min left"
+  for something done within a minute.
+- A restore that stops because a file is gone from the location now points at `--salvage`, as it
+  already did for damaged data.
+
+### Known issues
+- Empty folders are not backed up or restored; to be fixed in 1.0.3 (#27).
+
 ## [1.0.1] - 2026-11-01

 Fixes for what was reported after 1.0.0 (#22, #24) and small improvements. Nothing changes in
@@ -392,9 +392,62 @@ in README "Known limitations":
  gids (SFTP, Storage Box) → rc4 uitbrengen en om tests vragen → zelf een volledige DR van een
  productie-instantie van ~250 GB via S3 en via SMB. Drie volledige DR-restores vóór 1.0.0.
  Problemen of meldingen van gebruikers: rc5.
- **1.0.1** (klaar voor release 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
+- **1.0.2** (plan akkoord 2026-11-04): punten 1, 2 en 5 hieronder, de Bin, nieuwe screenshots en
+  het polish-punt over de schatting van `target:add`. Besluiten 2026-11-04:
+  - `--salvage` haalt ook bij een kapotte database terug wat kan: bestanden zijn per gebruiker
+    gescheiden en het waardevolst. Kan de database niet terug (dump onleesbaar, of past niet op
+    deze installatie), dan blijft de verse database staan en gaan data en config van de backup
+    naar `<datadir>/ngb-salvage-<snapshot>/`; vanaf daar is het handwerk. Exitcode 2.
+  - De DR-test met een backup mét tabelstructuur, zonder `--skip-missing-tables` (punt 3), is
+    nog niet gedaan en wordt ingepland vóór de release van 1.0.2. Ingepland (2026-11-04): een
+    verse snapshot van nc01 (gemaakt met 1.0.1, dus met tabelstructuur) volgens de DR-gids
+    terugzetten op VM 980 met de 1.0.2-build, over **SMB** (Storage Box; besluit 2026-11-04:
+    SMB in plaats van SFTP, zodat de DR-test de nieuwe SMB-leescode op echte schaal en over de
+    echte lijn vanuit Canada test; vorige meting 2,6 MiB/s, verwachting ~1 dag voor 1 TB), zonder
+    `--skip-missing-tables`; daarna `--salvage` op een deel van de bestanden (moet hetzelfde
+    resultaat geven, exitcode 0). Uitvoeren met Dillard, die VM 980 klaarzet; deze week, vóór
+    de release. Screenshots akkoord (2026-11-04).
+  **DR-test 1.0.2 over SMB (VM 980, snapshot a86271ed van nc01, 983 GiB, gestart 2026-11-04):**
+  index lezen 8 uur (~1 bestand/s; daarna versneld naar 0,66 s/bestand, gemeten vanaf VM 980);
+  restore zonder opties (geen `--install-apps`, geen `--skip-missing-tables`) liep ~12 MiB/s
+  (SFTP eerder 2,8-7,6; SMB vóór 1.0.2 2,6). Na 392 GiB viel de SMB-verbinding weg en bleef de
+  restore op 100% CPU hangen: de smbclient-extensie geeft bij een mislukte read een string met
+  lengte -1. Opgelost (controle + nieuwe verbinding), plus hervatten: `--force` laat bestanden met
+  dezelfde grootte en mtime staan. Hervat 2026-11-05 07:21 UTC na `maintenance:mode --off`
+  volgens de gids; binnen 10 min voorbij de 392 GiB. SMB op sub5 na de test weer uitzetten.
+  **Resultaat (klaar 2026-11-06 02:11 UTC, nazorg 02:27):** hervatte run 18 u 50 min, exit 0, 150.755
+  bestanden; 7-13 MiB/s (langzamer vanaf ~18 uur Duitse tijd, oorzaak niet vastgesteld). Vergeleken
+  met nc01: 150.467 van 150.758 bestanden gelijk in grootte en mtime, 397 op hash zonder verschil;
+  gebruikers, agenda's, contacten, shares en Deck gelijk; bestanden, agenda en contacten via WebDAV
+  getest als gebruiker. `backup:restore --salvage` op 16.904 bestanden: exit 0, geen verschil. Niet
+  vastgesteld: de 50.080 "updated" bij `files:scan`. Zonder `--install-apps` 51 van de 81 apps aan.
+  VM's 980-982 en de sim-backups op OVH en sub6 opgeruimd.
+  **Naar 1.0.3 (besluit 2026-11-06):** lege mappen komen niet terug (3.982 op nc01, 2.076 daarvan in
+  `files`, verklaart vrijwel zeker de 2.370 "removed" bij `files:scan`). De backup legt alleen
+  bestanden vast (`walk()`), dus oudere snapshots hebben ze niet. Voor een DR acceptabel, wel
+  fixen; issue #27. Daarna kleine volledige simulaties; een DR van 1 TB is hiervoor niet nodig.
+  **Salvage vóór de release getest (besluit 2026-11-06):** nieuwe modus `salvage-files` in
+  `full-restore-roundtrip.sh`, ook in CI: één pack beschadigd en één weg. Eerst stopt de restore,
+  daarna `--salvage --force` volgens de gids. Vond één fout: bij een ontbrekend bestand wees de
+  melding niet naar `--salvage` (opgelost).
+  Stand 2026-11-04 (`polish`, gepusht): cron-waarschuwing + tick vanaf de pagina +
+  voortgang van de db-fase; `--salvage` voor `backup:restore` en `backup:restore:full`; S3
+  vooruitlezen (8 ranges tegelijk, met `SaveAs` naar de spool); SMB in leesacties van 8 MiB
+  (gemeten met netem, 130 ms RTT: 4,5 → 18 MiB/s per leesactie; een volledige restore over SMB
+  met vertraging is nog niet gemeten); Bin-tabblad; schatting `target:add`; nieuwe screenshots
+  (1–3 vervangen, 4-bin erbij; in `info.xml` bij de release naar tag v1.0.2 laten wijzen en
+  4-bin toevoegen). Getest: unit (208), psalm, integratie NC35 sqlite/pgsql, S3/SFTP/FTP/SMB,
+  volledige restore met `--salvage`. **1.0.2 uitgebracht 2026-11-06** (tag `v1.0.2`, PR #28).
+- **Branches (besluit 2026-11-06):** werk voor een versie op een branch met die naam (`1.0.3`);
+  `polish` alleen voor kleine cosmetische zaken, per geval bekijken of en wanneer die gemerged
+  worden. SMB staat uit op sub5 en sub6; sub6 is het testaccount, eerst aanzetten voor een SMB-test.
+- **1.0.1** (uitgebracht 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
  1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november,
-  niet aangekondigd en los van de release.
+  niet aangekondigd en los van de release. Voor 1.0.2 ook: nieuwe screenshots (het labelveld bij
+  "Back up now" en het tabblad Docs staan er nog niet op; besluit 2026-11-01), en de prullenbak
+  van herstelpunten naar een eigen tabblad "Bin" naast "Backup and restore", "Settings" en "Docs":
+  onder de lijst met herstelpunten vervuilt hij het scherm (vooral met wat het bewaarbeleid erin
+  zet; besluit 2026-11-01). Nieuwe screenshots daarna, zodat de Bin er ook op staat.
  1. **Naar 1.0.2.** (Naar rc4: herstelsnelheid over SFTP.) S3 leest al 16 tegelijk bij index en tree, maar een
     restore leest de packs nog één voor één: ook vooruitlezen via de S3-client.
  2. **Naar 1.0.2.** Achtergrondjobs zonder systeem-cron (issue #13; besluit 2026-10-19). De PR die dit
@@ -7,7 +7,7 @@ installations and hosted Nextcloud. (Choosing SMB as a storage backend is the on
 see "Storage backend experiences" below — that requirement comes from files_external, not from
 ng_backup, and applies to any Nextcloud app that writes to an SMB share.)

-**Status: 1.0.1, stable.** Disaster recovery has been tested on real installations (see "Tested
+**Status: 1.0.2, stable.** Disaster recovery has been tested on real installations (see "Tested
 disaster recoveries" below). See "Known limitations" below, `PLAN.md` for the design and
 `docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
 `docs/DISASTER-RECOVERY.md`.
@@ -42,7 +42,8 @@ Need help restoring a backup after a crash? Ensembia offers paid restore assista
  background job, so a backup only moves forward when that job runs. Choosing "Cron" under
  **Administration settings → Basic settings → Background jobs** is not enough on its own: it
  tells Nextcloud that the system will call it, so you also need a cron line that does. Without
-  it a backup stays at "running (db)", and so does one started with "Back up now".
+  it scheduled backups do not run; the admin page and `occ backup:status` warn about it, and a
+  backup started with "Back up now" only moves on while the NG Backup page stays open.
  `occ backup:run` in a terminal works without cron (it runs in the foreground).
  - Regular server: `crontab -u www-data -e` and add
    `*/5 * * * * php -f /var/www/nextcloud/cron.php` (your Nextcloud path).
@@ -167,6 +168,7 @@ Full restores onto a fresh server, following only [the guide](docs/DISASTER-RECO
 | Backup | Location | Database | Result |
 |---|---|---|---|
 | Production, about 1 TB, 150,000 files (Nextcloud All-in-One) | Hetzner Storage Box (SFTP), Germany → Canada | PostgreSQL → PostgreSQL | 150,622 files restored in about 25 h (13-14 MiB/s), sample identical |
+| The same, 1.0.2 | Hetzner Storage Box (SMB), Germany → Canada | PostgreSQL → PostgreSQL | 150,755 files restored in about 29 h (7-13 MiB/s), going on after a lost connection; sample identical, empty folders not restored ([#27](https://github.com/dillardblom/ng_backup/issues/27)) |
 | 138 GB simulation: 20 users, calendars, contacts, mail, shares, team folders with an ACL, versions, trash | OVH S3, Warsaw → Canada | MariaDB → PostgreSQL | all 15,772 files and every database count identical |
 | The same | OVH S3 | MariaDB → MariaDB | all 15,772 files and every database count identical |
 | The same | Hetzner Storage Box (SMB), Germany → Canada | MariaDB → SQLite | all 15,772 files and every database count identical (27.6 h, about 1.5 MB/s) |
@@ -263,14 +265,9 @@ What is planned, without dates. Ideas and votes are welcome in the
 [issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
 a 👍 on its issue helps decide what comes first.

-**1.0.2**
- A warning on the admin page and in `occ backup:status` when background jobs do not run on a
-  system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
-  while the NG Backup page is open, and the database phase of a backup shows its progress
-  ([#13](https://github.com/dillardblom/ng_backup/issues/13)).
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
- `--salvage` for a restore: restore what can be read, skip a file that cannot, and list those at
-  the end instead of stopping.
+**1.0.3**
+- Empty folders in the backup and the restore
+  ([#27](https://github.com/dillardblom/ng_backup/issues/27)).

 **1.1.0**
 - Restore without the recovery kit: the backup key from the location itself, opened with one of
@@ -296,6 +293,9 @@ a 👍 on its issue helps decide what comes first.

 On the backlog; know them before you rely on NG Backup:

+- **Empty folders do not come back.** A backup only records files, so a folder without files is
+  not restored. To be fixed in 1.0.3
+  ([#27](https://github.com/dillardblom/ng_backup/issues/27)).
 - **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
  `config.php`) is not supported: a backup refuses to start there. Folders mounted through the
  External storage app are not in the data directory and are **not included**, without a warning.
@@ -40,12 +40,12 @@ NG Backup needs Nextcloud's background jobs on system cron (a real cron line, no
 Read "Known limitations" in the README before you rely on it, and test a restore. Feedback is very
 welcome: use the "Share your experience" link on the admin page.

-If you like NG Backup, don't forget to give the app a star in the app store and the
-[repository](https://github.com/dillardblom/ng_backup) a star on GitHub!
+**If you like NG Backup, don't forget to give the app a star in the app store and the
+[repository](https://github.com/dillardblom/ng_backup) a star on GitHub!**

 Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
 you need help with restoring a backup after a crash.]]></description>
-    <version>1.0.1</version>
+    <version>1.0.2</version>
    <licence>agpl</licence>
    <author mail="dillard.blom@ensembia.com">Dillard Blom</author>
    <namespace>NgBackup</namespace>
@@ -56,9 +56,10 @@ you need help with restoring a backup after a crash.]]></description>
    <website>https://github.com/dillardblom/ng_backup</website>
    <bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
    <repository type="git">https://github.com/dillardblom/ng_backup</repository>
-    <screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.1/screenshots/1-backups.png</screenshot>
-    <screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.1/screenshots/2-restore.png</screenshot>
-    <screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.1/screenshots/3-settings.png</screenshot>
+    <screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/1-backups.png</screenshot>
+    <screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/2-restore.png</screenshot>
+    <screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/3-settings.png</screenshot>
+    <screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/4-bin.png</screenshot>
    <dependencies>
        <php min-version="8.1"/>
        <lib>sodium</lib>
@@ -24,6 +24,7 @@ return [
 		['name' => 'AdminApi#testTarget', 'url' => '/api/targets/{id}/test', 'verb' => 'POST'],
 		['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'],
 		['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'],
+		['name' => 'AdminApi#tick', 'url' => '/api/tick', 'verb' => 'POST'],
 		['name' => 'AdminApi#setSchedule', 'url' => '/api/schedule', 'verb' => 'PUT'],
 		['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'],
 		['name' => 'AdminApi#browse', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/browse', 'verb' => 'GET'],
@@ -103,12 +103,31 @@ mode on yourself before, it stays on: switch it off with `occ maintenance:mode -
 If it stops halfway, maintenance mode stays on (the instance may be partly restored). Fix the cause,
 then switch maintenance mode off and run the restore again with `--force`, in one line (the message
 prints it): `occ maintenance:mode --off && occ backup:restore:full <location> <snapshot> --force`.
-Leave out `--install-apps` this time, so that the instance is open only for the seconds until the
+It goes on where it stopped: data files that are already in place with the size and modification
+time of the backup are left, so only the rest comes from the location again (since 1.0.2; before,
+it started over). The database is restored again in full, which takes minutes. Leave out `--install-apps` this time, so that the instance is open only for the seconds until the
 restore has switched maintenance mode on again; apps the first run could not install are listed at
 the end of the restore, install those afterwards.
 If occ then says that Nextcloud or one of the apps requires an upgrade, the database is already
 the old server's: run `occ upgrade` first, then the same command with `--force`.

+**If the backup itself is damaged** (the restore stops with "Decryption failed", "Blob
+verification failed" or a file that is missing on the location), try an older restore point
+first. If none is better, `--salvage` restores whatever can be read:
+
+    occ backup:restore:full offsite <snapshot> --install-apps --salvage
+
+- Files that cannot be read are left out and listed at the end; everything else is restored.
+- If the database cannot be restored (its dump is damaged, or it does not fit this
+  installation), the database stays as the fresh installation made it, and the data directory
+  and config of the backup go to a folder of their own in the data directory,
+  `ngb-salvage-<snapshot>`. Each user's files are in `data/<user>/files` there: create the users
+  again and copy their files back by hand (then `occ files:scan --all`). Calendars, contacts and
+  shares are in the database and cannot be recovered this way.
+
+The command then ends with exit code 2 instead of 0. The same option exists for
+`occ backup:restore`, to restore single folders.
+
 **9. After the restore.** These worked in our tests:

    occ upgrade
@@ -121,6 +140,9 @@ the old server's: run `occ upgrade` first, then the same command with `--force`.
 - From Nextcloud All-in-One to a regular server: also check the cache settings (`memcache.*`,
  `redis`), which point at AIO's containers.
 - Log in, open a few files, a calendar and a contact.
+- Empty folders do not come back: a backup only records files (to be fixed in 1.0.3, see
+  [issue #27](https://github.com/dillardblom/ng_backup/issues/27)). `occ files:scan` then removes
+  them from the file cache, so it reports them as removed.
 - Start background jobs again (cron), as in step 2 in reverse.

 **10. Point your domain at the new server** and check that clients reconnect.
@@ -23,6 +23,11 @@ final class StorageBackend implements IBackend, IReadsAhead {
 	private bool $s3Reads;
 	/** S3 requests in flight at once for getMany() and getRanges(). */
 	public const S3_CONCURRENCY = 16;
+	/**
+	 * Ranges readAhead() fetches at once over S3, each straight into a file in the spool (a range
+	 * is at most ReadAheadPlan::MAX_RANGE: the spool holds at most this many times that).
+	 */
+	public const S3_READ_AHEAD = 8;
 	/** Most ranges, and bytes, getRanges() holds in memory at once. */
 	private const RANGES_LOT = 64;
 	private const RANGES_LOT_BYTES = 64 * 1048576;
@@ -128,7 +133,7 @@ final class StorageBackend implements IBackend, IReadsAhead {

 	/**
 	 * Over SFTP and S3 several at a time (SftpReadStream::readMany(), s3Fetch()); elsewhere one
-	 * after the other.
+	 * after the other, over SMB in fewer round trips (smbWhole()).
 	 */
 	public function getMany(array $paths): \Generator {
 		$size = $this->s3Reads ? 16 * self::S3_CONCURRENCY : 8 * SftpReadStream::BATCH;
@@ -144,7 +149,8 @@ final class StorageBackend implements IBackend, IReadsAhead {
 				$fast = $fast === null ? null : array_combine($chunk, array_map(fn (string $a): ?string => $fast[$a] ?? null, $abs));
 			}
 			foreach ($chunk as $path) {
-				$data = $fast[$path] ?? null;
+				// over SMB in fewer round trips, one after the other (smbWhole())
+				$data = $fast[$path] ?? $this->smbWhole($this->abs($path));
 				if ($data === null) {
 					// not read in the batch (or no batches here): the usual way, which also
 					// reports a missing file as such
@@ -211,15 +217,11 @@ final class StorageBackend implements IBackend, IReadsAhead {
 	 */
 	private function s3Fetch(array $requests): ?array {
 		try {
-			/** @psalm-suppress UndefinedMethod, UndefinedThisPropertyFetch */
-			[$client, $bucket, $sse] = (function (): array {
-				$sse = method_exists($this, 'getServerSideEncryptionParameters') ? $this->getServerSideEncryptionParameters()
-					: (method_exists($this, 'getSSECParameters') ? $this->getSSECParameters() : []);
-				return [$this->getConnection(), $this->getBucket(), $sse];
-			})->call($this->storage);
-			if (!$client instanceof \Aws\S3\S3Client || !is_string($bucket)) {
+			$s3 = $this->s3Client();
+			if ($s3 === null) {
 				return null;
 			}
+			[$client, $bucket, $sse] = $s3;
 			$results = [];
 			$keys = array_keys($requests); // the pool numbers results 0, 1, ...: mapped back below
 			$commands = (function () use ($requests, $client, $bucket, $sse): \Generator {
@@ -245,16 +247,111 @@ final class StorageBackend implements IBackend, IReadsAhead {
 		}
 	}

+	/**
+	 * files_external's S3 client, bucket and server-side encryption parameters; null when the
+	 * storage is not as expected.
+	 *
+	 * @return array{0: \Aws\S3\S3Client, 1: string, 2: array}|null
+	 * @psalm-suppress UndefinedClass the AWS SDK comes with Nextcloud, not with this app
+	 */
+	private function s3Client(): ?array {
+		/** @psalm-suppress UndefinedMethod, UndefinedThisPropertyFetch */
+		[$client, $bucket, $sse] = (function (): array {
+			$sse = method_exists($this, 'getServerSideEncryptionParameters') ? $this->getServerSideEncryptionParameters()
+				: (method_exists($this, 'getSSECParameters') ? $this->getSSECParameters() : []);
+			return [$this->getConnection(), $this->getBucket(), $sse];
+		})->call($this->storage);
+		return $client instanceof \Aws\S3\S3Client && is_string($bucket) ? [$client, $bucket, is_array($sse) ? $sse : []] : null;
+	}
+
+	/**
+	 * readAhead() over S3: S3_READ_AHEAD ranges at a time, fetched together straight into files in
+	 * the spool, then handed out in order. One request per range instead of one per blob (with a
+	 * new HTTPS connection each), and several at once: a restore is no longer bound by the round
+	 * trip. A range that could not be fetched is handed out as null (the caller reads it the
+	 * usual way, which reports why).
+	 *
+	 * @param list<array{0:string, 1:int, 2:int}> $ranges
+	 * @return \Generator<int, resource|null>|null
+	 * @psalm-suppress UndefinedClass the AWS SDK comes with Nextcloud, not with this app
+	 */
+	private function s3ReadAhead(array $ranges, string $spool): ?\Generator {
+		// before anything is fetched: an invalid path throws
+		$keys = array_map(fn (array $r): string => trim($this->abs($r[0]), '/'), $ranges);
+		try {
+			$s3 = $this->s3Client();
+		} catch (\Throwable) {
+			$s3 = null;
+		}
+		if ($s3 === null) {
+			return null;
+		}
+		[$client, $bucket, $sse] = $s3;
+		$this->idleCheck();
+		return (static function () use ($client, $bucket, $sse, $ranges, $keys, $spool): \Generator {
+			$prefix = rtrim($spool, '/') . '/s3-' . bin2hex(random_bytes(4)) . '-';
+			foreach (array_chunk(array_keys($ranges), self::S3_READ_AHEAD) as $lot) {
+				$files = [];
+				$commands = [];
+				foreach ($lot as $n => $i) {
+					$files[$i] = $prefix . $i;
+					$commands[$n] = $client->getCommand('GetObject', ['Bucket' => $bucket, 'Key' => $keys[$i],
+						'Range' => 'bytes=' . $ranges[$i][1] . '-' . ($ranges[$i][1] + $ranges[$i][2] - 1), 'SaveAs' => $files[$i]] + $sse);
+				}
+				$fetched = [];
+				try {
+					(new \Aws\CommandPool($client, $commands, [
+						'concurrency' => self::S3_READ_AHEAD,
+						'fulfilled' => function (mixed $result, int $n) use (&$fetched, $lot): void {
+							$fetched[$lot[$n]] = true;
+						},
+						'rejected' => static function (): void {
+						},
+					]))->promise()->wait();
+				} catch (\Throwable) {
+					// what was not fetched is read the usual way
+				}
+				try {
+					foreach ($lot as $i) {
+						clearstatcache(true, $files[$i]);
+						$fh = isset($fetched[$i]) && @filesize($files[$i]) === $ranges[$i][2] ? @fopen($files[$i], 'rb') : false;
+						@unlink($files[$i]); // the open stream keeps its data
+						yield $i => is_resource($fh) ? $fh : null;
+						if (is_resource($fh)) {
+							fclose($fh);
+						}
+					}
+				} finally {
+					// stopped halfway: the rest of this lot
+					foreach ($files as $file) {
+						@unlink($file);
+					}
+				}
+			}
+		})();
+	}
+
 	/**
 	 * Over SFTP: SftpRangeFetcher, on readAheadConnections new connections of its own (built with the
-	 * reconnect function), so the storage's connection stays free for other requests. Elsewhere
-	 * not (yet): null.
+	 * reconnect function), so the storage's connection stays free for other requests. Over S3:
+	 * S3_READ_AHEAD ranges at a time through the S3 client (s3ReadAhead()). Elsewhere not (yet):
+	 * null.
 	 */
 	public function canReadAhead(): bool {
-		return $this->sftpReads && $this->reconnect !== null;
+		return ($this->sftpReads && $this->reconnect !== null) || $this->s3Reads || $this->smbNative();
+	}
+
+	private function smbNative(): bool {
+		return $this->storage instanceof \OCA\Files_External\Lib\Storage\SMB && extension_loaded('smbclient');
 	}

 	public function readAhead(array $ranges, string $spool): ?\Generator {
+		if ($ranges !== [] && $this->s3Reads) {
+			return $this->s3ReadAhead($ranges, $spool);
+		}
+		if ($ranges !== [] && $this->smbNative()) {
+			return $this->smbReadAhead($ranges, $spool);
+		}
 		$reconnect = $this->reconnect;
 		if (!$this->sftpReads || $reconnect === null || $ranges === []) {
 			return null;
@@ -306,6 +403,18 @@ final class StorageBackend implements IBackend, IReadsAhead {
 	}

 	private function readRange(string $path, int $offset, int $length): string {
+		try {
+			$chunks = $this->smbChunks($this->abs($path), $offset, $length);
+			if ($chunks !== null) {
+				$data = implode('', iterator_to_array($chunks, false));
+				if (strlen($data) === $length) {
+					return $data;
+				}
+			}
+		} catch (\Throwable) {
+			// read the usual way below, which reports what is wrong, over a new connection
+			$this->smbReconnect();
+		}
 		$fh = $this->get($path);
 		try {
 			if (fseek($fh, $offset) !== 0) {
@@ -418,6 +527,218 @@ final class StorageBackend implements IBackend, IReadsAhead {
 	 *
 	 * @return list<string>|null
 	 */
+	/**
+	 * Bytes per read over SMB (smbChunks()). libsmbclient keeps a large read in flight as many
+	 * requests at once; icewind's stream reads 1 MiB at a time. Over a line with a 130 ms round
+	 * trip: 4.5 MiB/s in 1 MiB reads, 18 MiB/s in 8 MiB reads (measured 2026-11-04, Samba in a
+	 * container behind netem). Without latency it makes no difference.
+	 */
+	public const SMB_READ = 8 * 1048576;
+
+	/**
+	 * $length bytes of $abs from $offset over SMB, SMB_READ at a time straight through the native
+	 * extension's state of files_external's SMB storage. Null when the storage is not SMB through
+	 * the extension, or its internals are not as expected (the usual way reads it then). The
+	 * generator throws when a read fails.
+	 *
+	 * @return \Generator<int, string>|null
+	 * @psalm-suppress UndefinedClass icewind/smb comes with files_external, not with this app
+	 */
+	private function smbChunks(string $abs, int $offset, int $length): ?\Generator {
+		$handle = $this->smbHandle($abs);
+		if ($handle === null) {
+			return null;
+		}
+		[$state, $url] = $handle;
+		/** @psalm-suppress MixedMethodCall the state is icewind's NativeState */
+		return (static function () use ($state, $url, $offset, $length): \Generator {
+			$fh = $state->open($url, 'r');
+			try {
+				$state->lseek($fh, $offset, SEEK_SET, $url);
+				for ($got = 0; $got < $length;) {
+					$want = min(self::SMB_READ, $length - $got);
+					$chunk = self::smbRead($state, $fh, $want, $url);
+					if ($chunk === '') {
+						throw new BackendException('Short read in ' . $url);
+					}
+					$got += strlen($chunk);
+					yield $chunk;
+				}
+			} finally {
+				$state->close($fh, $url);
+			}
+		})();
+	}
+
+	/**
+	 * A whole (small) file over SMB in four round trips: open, read, read up to the end, close.
+	 * files_external's fopen() first checks that the file exists (a stat with its attributes) and
+	 * icewind's stream reads 1 MiB at a time: about 7 round trips, 1 s per index file from Canada
+	 * to the Storage Box (130 ms), so ~8 h for the 27,000 index files of a 1 TB repository. Null
+	 * when the storage is not SMB through the extension or anything fails: the usual way then
+	 * reads the file and reports a missing one as such.
+	 *
+	 * @psalm-suppress UndefinedClass icewind/smb comes with files_external, not with this app
+	 */
+	private function smbWhole(string $abs): ?string {
+		$handle = $this->smbHandle($abs);
+		if ($handle === null) {
+			return null;
+		}
+		[$state, $url] = $handle;
+		try {
+			/** @psalm-suppress MixedMethodCall the state is icewind's NativeState */
+			$fh = $state->open($url, 'r');
+		} catch (\Throwable) {
+			return null;
+		}
+		$data = '';
+		try {
+			do {
+				$chunk = self::smbRead($state, $fh, self::SMB_READ, $url);
+				$data .= $chunk;
+			} while ($chunk !== '');
+		} catch (\Throwable) {
+			$data = null;
+		}
+		try {
+			/** @psalm-suppress MixedMethodCall */
+			$state->close($fh, $url);
+		} catch (\Throwable) {
+		}
+		if ($data === null) {
+			$this->smbReconnect(); // a read failed: the usual way reads it over a new connection
+		}
+		return $data;
+	}
+
+	/**
+	 * One read of at most $bytes through the native state, checked. When libsmbclient fails a read
+	 * (seen when the connection to the server dropped during a long restore), the smbclient
+	 * extension does not return false but a string whose length is -1, because it stores
+	 * smbc_read()'s -1 in an unsigned length; icewind passes it on. Counting that as data made
+	 * the read loop go backwards forever at 100% CPU (DR test 2026-11-05, after 392 GiB).
+	 *
+	 * @param resource $fh
+	 * @psalm-suppress MixedMethodCall, MixedArgument the state is icewind's NativeState
+	 */
+	private static function smbRead(object $state, $fh, int $bytes, string $url): string {
+		$chunk = $state->read($fh, $bytes, $url);
+		$n = is_string($chunk) ? strlen($chunk) : -1;
+		if ($n < 0 || $n > $bytes) {
+			throw new BackendException('Read failed in ' . $url);
+		}
+		return $chunk;
+	}
+
+	/**
+	 * After a failed read: drop icewind's connection state, so the next read (the usual way, or
+	 * the next range) connects again instead of failing on a connection that is gone.
+	 *
+	 * @psalm-suppress UndefinedClass icewind/smb comes with files_external, not with this app
+	 */
+	private function smbReconnect(): void {
+		if (!$this->storage instanceof \OCA\Files_External\Lib\Storage\SMB) {
+			return;
+		}
+		try {
+			(function (): void {
+				/** @psalm-suppress UndefinedThisPropertyFetch */
+				$share = $this->share;
+				if ($share instanceof \Icewind\SMB\Native\NativeShare) {
+					(function (): void {
+						/** @psalm-suppress UndefinedThisPropertyAssignment */
+						$this->state = null;
+					})->call($share);
+				}
+			})->call($this->storage);
+		} catch (\Throwable) {
+		}
+	}
+
+	/**
+	 * icewind's NativeState and the smb:// URL of $abs behind files_external's SMB storage, reached
+	 * through bound closures; null when the storage is not SMB through the native extension or its
+	 * internals are not as expected.
+	 *
+	 * @return array{0: object, 1: string}|null
+	 * @psalm-suppress UndefinedClass icewind/smb comes with files_external, not with this app
+	 */
+	private function smbHandle(string $abs): ?array {
+		if (!$this->storage instanceof \OCA\Files_External\Lib\Storage\SMB || !extension_loaded('smbclient')) {
+			return null;
+		}
+		try {
+			/** @var array{0: object, 1: string}|null $handle */
+			$handle = (function (string $path): ?array {
+				/** @psalm-suppress UndefinedThisPropertyFetch */
+				$share = $this->share;
+				if (!$share instanceof \Icewind\SMB\Native\NativeShare) {
+					return null;
+				}
+				/** @psalm-suppress UndefinedMethod */
+				$sharePath = ltrim($this->buildPath($path), '/');
+				/** @psalm-suppress UndefinedMethod */
+				return (fn (string $p): array => [$this->getState(), $this->buildUrl($p)])->call($share, $sharePath);
+			})->call($this->storage, $abs);
+		} catch (\Throwable) {
+			return null;
+		}
+		return $handle;
+	}
+
+	/**
+	 * readAhead() over SMB: one range after the other, each in SMB_READ reads into a file in the
+	 * spool. Not several at once (PHP cannot wait on more than one libsmbclient call), but whole
+	 * ranges of up to ReadAheadPlan::MAX_RANGE in large reads instead of blob by blob in 1 MiB
+	 * reads. A range that cannot be read is handed out as null.
+	 *
+	 * @param list<array{0:string, 1:int, 2:int}> $ranges
+	 * @return \Generator<int, resource|null>
+	 */
+	private function smbReadAhead(array $ranges, string $spool): \Generator {
+		// before anything is read: an invalid path throws
+		$abs = array_map(fn (array $r): array => [$this->abs($r[0]), $r[1], $r[2]], $ranges);
+		return (function () use ($abs, $spool): \Generator {
+			$file = rtrim($spool, '/') . '/smb-' . bin2hex(random_bytes(4));
+			try {
+				foreach ($abs as $i => [$path, $offset, $length]) {
+					$fh = null;
+					$out = false;
+					try {
+						$chunks = $this->smbChunks($path, $offset, $length);
+						$out = $chunks === null ? false : fopen($file, 'w+b');
+						if ($chunks !== null && $out !== false) {
+							foreach ($chunks as $chunk) {
+								if (fwrite($out, $chunk) !== strlen($chunk)) {
+									throw new BackendException('Cannot write to the spool');
+								}
+							}
+							$fh = ftell($out) === $length && rewind($out) ? $out : null;
+							if ($fh === null) {
+								fclose($out);
+							}
+						}
+					} catch (\Throwable) {
+						if (is_resource($out)) {
+							fclose($out);
+						}
+						$fh = null; // read the usual way, which reports why, over a new connection
+						$this->smbReconnect();
+					} finally {
+						@unlink($file); // an open stream keeps its data
+					}
+					yield $i => $fh;
+					if (is_resource($fh)) {
+						fclose($fh);
+					}
+				}
+			} finally {
+				@unlink($file);
+			}
+		})();
+	}
+
 	private function smbNames(string $abs): ?array {
 		if (!$this->storage instanceof \OCA\Files_External\Lib\Storage\SMB) {
 			return null;
@@ -31,6 +31,7 @@ final class Restore extends Command {
 			->addArgument('path', InputArgument::REQUIRED, 'Path in the snapshot (see backup:browse)')
 			->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'new-folder, merge or replace (user files)', RestoreService::MODE_NEW_FOLDER)
 			->addOption('to-directory', 'd', InputOption::VALUE_REQUIRED, 'Write to this local directory instead (any path, raw files)')
+			->addOption('salvage', null, InputOption::VALUE_NONE, 'Leave out a file that cannot be read or decrypted, list it at the end and go on, instead of stopping')
 			->setHelp(<<<'HELP'
 User files (data/<user>/files/...) are restored through Nextcloud, so versions, trash and the
 file cache stay correct. Modes:
@@ -42,30 +43,59 @@ Examples:
  occ backup:restore offsite 537bf7e9... data/alice/files/Documents
  occ backup:restore offsite 537bf7e9... data/alice/files/Documents/plan.odt --mode=merge
  occ backup:restore offsite 537bf7e9... config --to-directory=/root/restored-config
+With --salvage, a file whose data on the location is damaged or missing is left out and listed
+at the end; everything else is restored. The command then exits with code 2.
 HELP);
 	}

 	protected function execute(InputInterface $input, OutputInterface $output): int {
+		$skipped = $input->getOption('salvage') ? [] : null;
 		try {
 			$target = $this->targets->get($input->getArgument('target'));
 			$snapshot = $input->getArgument('snapshot');
 			$path = trim((string)$input->getArgument('path'), '/'); // CLI convenience; the service itself only accepts canonical paths
 			if ($dir = $input->getOption('to-directory')) {
-				$n = $this->restore->restoreToDirectory($target, $snapshot, $path, $dir);
+				$n = $this->restore->restoreToDirectory($target, $snapshot, $path, $dir, false, null, $skipped);
 				$output->writeln("Restored <info>$n</info> files to $dir.");
-				return 0;
+				return self::reportSkipped($skipped, $output);
 			}
 			$r = $this->restore->restoreUserFiles($target, $snapshot, $path, (string)$input->getOption('mode'),
 				function (int $done, int $total, string $name) use ($output) {
 					if ($output->isVerbose() || $done === $total || $done % 100 === 0) {
 						$output->writeln("  $done/$total $name");
 					}
-				});
+				}, $skipped);
 		} catch (\Throwable $e) {
 			$output->writeln('<error>' . $e->getMessage() . '</error>');
+			if ($skipped === null && self::backupLooksDamaged($e)) {
+				$output->writeln('The data of a file on the location is damaged or missing. Run the same command with --salvage to restore everything else and list what could not be restored.');
+			}
 			return 1;
 		}
 		$output->writeln(sprintf('Restored <info>%d</info> files to %s%s.', $r['restored'], $r['target'], $r['trashed'] ? ", moved {$r['trashed']} extra files to the trash" : ''));
-		return 0;
+		return self::reportSkipped($skipped, $output);
+	}
+
+	/** Damaged data, or a file that is gone from the location: what --salvage is for. */
+	public static function backupLooksDamaged(\Throwable $e): bool {
+		return $e instanceof \OCA\NgBackup\Repository\RepositoryException
+			|| $e instanceof \OCA\NgBackup\Crypto\CryptoException
+			|| ($e instanceof \OCA\NgBackup\Backend\BackendException && str_starts_with($e->getMessage(), 'Not found'));
+	}
+
+	/**
+	 * The files --salvage left out; exit code 2 when there are any.
+	 *
+	 * @param array<string, string>|null $skipped
+	 */
+	public static function reportSkipped(?array $skipped, OutputInterface $output): int {
+		if ($skipped === null || $skipped === []) {
+			return 0;
+		}
+		$output->writeln(sprintf('<comment>%d file%s could not be restored (damaged or missing on the location):</comment>', count($skipped), count($skipped) === 1 ? '' : 's'));
+		foreach ($skipped as $path => $reason) {
+			$output->writeln("  $path: $reason");
+		}
+		return 2;
 	}
 }
@@ -36,6 +36,7 @@ final class RestoreFull extends Command {
 			->addOption('force', null, InputOption::VALUE_NONE, 'Also on an installation that already has more than one user, and without asking')
 			->addOption('install-apps', null, InputOption::VALUE_NONE, 'First install (from the app store) or enable the apps that were enabled at backup time and are missing here')
 			->addOption('skip-missing-tables', null, InputOption::VALUE_NONE, 'Older backups only: leave out tables that do not exist here and whose structure the backup does not record')
+			->addOption('salvage', null, InputOption::VALUE_NONE, 'Restore what can be restored: leave out files that cannot be read, and when the database cannot be restored, put the files in a folder of their own')
 			->setHelp(<<<'HELP'
 For a FRESH Nextcloud installation only, same major version as the backup, right after
 occ maintenance:install, occ backup:key:import and occ backup:target:add -- before anyone else
@@ -55,6 +56,12 @@ here. Other files under config/ (if any) are not restored.

 Afterwards: check trusted_domains and overwritehost/overwriteprotocol match this server, and
 occ maintenance:repair.
+When the backup is damaged: --salvage restores what can be restored. Files whose data cannot be
+read or decrypted are left out and listed. When the database cannot be restored (its dump is
+damaged, or it does not fit this installation), the database stays as occ maintenance:install
+made it, and the data directory and config of the backup go to ngb-salvage-<snapshot> in the
+data directory instead: from there, the files can be put back by hand. The command then exits
+with code 2.

 Example:
  occ backup:restore:full offsite 537bf7e9c1a4...
@@ -82,29 +89,41 @@ HELP);
 			$target = $this->targets->get($input->getArgument('target'));
 			$snapshot = (string)$input->getArgument('snapshot');
 			$skipMissingTables = (bool)$input->getOption('skip-missing-tables');
+			$salvage = (bool)$input->getOption('salvage');
 			if ($input->getOption('install-apps')) {
-				$check = $this->restoreFull->checkCompatibility($target, $snapshot, $skipMissingTables);
+				$check = $this->restoreFull->checkCompatibility($target, $snapshot, $skipMissingTables, $salvage);
 				foreach ($check['missingApps'] as $app) {
 					$this->installApp($app, $output);
 				}
 			}
 			$output->writeln('Nextcloud is in maintenance mode while the restore runs; it is switched off when the restore completes.');
-			$result = $this->restoreFull->restoreFull($target, $snapshot, $skipMissingTables);
+			$result = $this->restoreFull->restoreFull($target, $snapshot, $skipMissingTables, $salvage);
 		} catch (\Throwable $e) {
 			$output->writeln('<error>' . $e->getMessage() . '</error>');
 			if (\OCP\Server::get(\OCP\IConfig::class)->getSystemValueBool('maintenance', false)) {
 				$output->writeln('The restore had started: the instance may be partly restored, and maintenance mode stays on so that nobody uses it like that.');
 				$output->writeln('Fix the cause, then switch maintenance mode off and run the same command again with --force, in one line:');
 				$again = 'occ backup:restore:full ' . escapeshellarg((string)$input->getArgument('target')) . ' ' . escapeshellarg((string)$input->getArgument('snapshot'))
-					. ($input->getOption('skip-missing-tables') ? ' --skip-missing-tables' : '') . ' --force --no-interaction';
+					. ($input->getOption('skip-missing-tables') ? ' --skip-missing-tables' : '') . ($input->getOption('salvage') ? ' --salvage' : '') . ' --force --no-interaction';
 				$output->writeln('  occ maintenance:mode --off && ' . $again);
 				$output->writeln('(Without --install-apps, so the instance is open only for the seconds until the restore starts again. Apps the first run could not install are listed at the end: install those afterwards.)');
 			} else {
 				$output->writeln('If this happened after the restore had started, the instance may be partly restored: fix the cause and run the same command again with --force.');
 			}
 			$output->writeln('If occ then says that Nextcloud needs an upgrade (the database is already the old server\'s), run occ upgrade first.');
+			if (!$input->getOption('salvage') && Restore::backupLooksDamaged($e)) {
+				$output->writeln('The backup itself looks damaged here. --salvage restores everything that can be read and lists the rest.');
+			}
 			return 1;
 		}
+		if ($result['dbError'] !== null) {
+			$output->writeln('<error>Salvage: ' . $result['dbError'] . '</error>');
+			$output->writeln(sprintf('The database stays as occ maintenance:install made it. The data directory and config of the backup (<info>%d</info> data files) are in:', $result['dataFiles']));
+			$output->writeln('  ' . $result['salvageDir']);
+			$output->writeln('Each user\'s files are in data/<user>/files there. Put back what is needed by hand, for example into the files of new users (then occ files:scan).');
+			Restore::reportSkipped($result['skipped'], $output);
+			return 2;
+		}
 		$output->writeln(sprintf(
 			'Database restored (<info>%d</info> tables, <info>%d</info> unchanged), config merged (<info>%d</info> keys), <info>%d</info> data files restored.',
 			count($result['dbReport']['restored']),
@@ -135,7 +154,12 @@ HELP);
 			$output->writeln('<comment>Maintenance mode was on before the restore and stays on: switch it off with occ maintenance:mode --off when the instance is ready.</comment>');
 		}
 		$output->writeln('Next: check trusted_domains and overwritehost/overwriteprotocol match this server, then occ maintenance:repair.');
-		return 0;
+		$code = 0;
+		if ($result['configError'] !== null) {
+			$output->writeln('<comment>Salvage: config.php of the backup was not merged (' . $result['configError'] . '). Set at least \'instanceid\', \'passwordsalt\' and \'secret\' to the old server\'s values if you have them, or data encrypted with them (server-side encryption, stored passwords) cannot be read.</comment>');
+			$code = 2;
+		}
+		return max($code, Restore::reportSkipped($result['skipped'], $output));
 	}

 	/** Enable an app that is here but switched off, or install it from the app store. */
@@ -9,6 +9,7 @@ namespace OCA\NgBackup\Command;

 use OCA\NgBackup\Db\RunMapper;
 use OCA\NgBackup\Format;
+use OCA\NgBackup\Service\CronHealth;
 use OCA\NgBackup\Service\KeyService;
 use OCA\NgBackup\Service\Schedule;
 use OCA\NgBackup\Service\TargetService;
@@ -29,6 +30,7 @@ final class Status extends Command {
 		private RunMapper $runs,
 		private VerifyService $verify,
 		private Schedule $schedule,
+		private CronHealth $cron,
 	) {
 		parent::__construct();
 	}
@@ -40,6 +42,10 @@ final class Status extends Command {

 	protected function execute(InputInterface $input, OutputInterface $output): int {
 		$zone = $this->schedule->get()['timezone'];
+		$cronProblem = CronHealth::describe($this->cron->check());
+		if ($cronProblem !== null) {
+			$output->writeln('<error>' . $cronProblem . '</error>');
+		}
 		if (!$this->keys->isInitialized()) {
 			$output->writeln('Key: <error>not created</error> (occ backup:key:init)');
 		} else {
@@ -57,7 +63,7 @@ final class Status extends Command {
 		$shown = $this->runs->findShown($all ? null : time() - self::RECENT_DAYS * 86400);
 		foreach ($shown as $r) {
 			$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
-			$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() . ($r->getStatus() === 'running' ? ' (' . $r->getPhase() . self::percent($r) . ')' : ''),
+			$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus(),
 				Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
 				$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
 				isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
@@ -88,8 +94,24 @@ final class Status extends Command {
 		return sprintf('%.1f TiB', $b);
 	}

+	/** " 47/183 tables" in the database phase, " 68%" or " ~68%" in the files phase; empty without a figure */
 	private static function percent(\OCA\NgBackup\Db\Run $r): string {
 		$p = \OCA\NgBackup\Job\BackupRun::progressOf($r);
-		return $p === null ? '' : sprintf(' %s%d%%', $p['approximate'] ? '~' : '', (int)floor($p['fraction'] * 100.0));
+		if ($p === null) {
+			return '';
+		}
+		if (isset($p['tables'], $p['totalTables'])) {
+			return sprintf(': %d/%d tables', $p['tables'], $p['totalTables']);
+		}
+		return sprintf(' %s%d%%', $p['approximate'] ? '~' : '', (int)floor($p['fraction'] * 100.0));
+	}
+
+	/** "running (db: 47/183 tables)", or "waiting for background jobs (last step 12 min ago)" */
+	private static function runningStatus(\OCA\NgBackup\Db\Run $r): string {
+		$waiting = $r->getKind() === \OCA\NgBackup\Service\BackupService::KIND_FULL ? CronHealth::waitingFor($r) : null;
+		if ($waiting !== null) {
+			return 'waiting for background jobs (' . $r->getPhase() . self::percent($r) . ', last step ' . CronHealth::ago($waiting) . ')';
+		}
+		return 'running (' . $r->getPhase() . self::percent($r) . ')';
 	}
 }
@@ -88,19 +88,21 @@ HELP);
 	 * @return callable(int, int): void
 	 */
 	private static function progress(OutputInterface $output): callable {
-		$start = microtime(true);
 		$shown = 0.0;
+		$first = null; // [time, files done] when the first files were in
 		$inPlace = $output->isDecorated();
-		return static function (int $done, int $total) use ($output, $start, &$shown, $inPlace): void {
+		return static function (int $done, int $total) use ($output, &$shown, &$first, $inPlace): void {
 			$now = microtime(true);
+			if ($first === null && $done > 0) {
+				$first = [$now, $done];
+			}
 			if ($done < $total && $now - $shown < ($inPlace ? 1 : 30)) {
 				return;
 			}
 			$shown = $now;
 			$line = sprintf('Reading the index: %s/%s files (%d%%)', number_format($done), number_format($total), intdiv(100 * $done, max(1, $total)));
-			$elapsed = $now - $start;
-			if ($done < $total && $done > 0 && $elapsed >= 5) {
-				$left = (int)ceil($elapsed / (float)$done * (float)($total - $done) / 60.0);
+			$left = $first === null ? null : self::minutesLeft($now - $first[0], $done - $first[1], $total - $done, $total);
+			if ($left !== null) {
 				$line .= $left <= 1 ? ', less than a minute left' : sprintf(', about %d min left', $left);
 			}
 			if ($inPlace) {
@@ -110,4 +112,17 @@ HELP);
 			}
 		};
 	}
+
+	/**
+	 * Minutes left for reading the index, from the rate since the first files were in: the first
+	 * file also carries the time to connect, which made an S3 location with 4,568 index files
+	 * say "about 642 min left" for something done within a minute. No estimate until 5% of the
+	 * files, or 50 files, came in after that, and at least 5 seconds passed.
+	 */
+	public static function minutesLeft(float $elapsed, int $done, int $left, int $total): ?int {
+		if ($left <= 0 || $done <= 0 || $elapsed < 5.0 || ($done < 50 && $done * 20 < $total)) {
+			return null;
+		}
+		return (int)ceil($elapsed / (float)$done * (float)$left / 60.0);
+	}
 }
@@ -48,6 +48,7 @@ final class AdminApiController extends Controller {
 		private \OCA\NgBackup\Service\KeySlotService $slots,
 		private \OCA\NgBackup\Service\PruneService $prune,
 		private \OCA\NgBackup\Service\Schedule $schedule,
+		private \OCA\NgBackup\Service\CronHealth $cron,
 	) {
 		parent::__construct(Application::APP_ID, $request);
 	}
@@ -75,9 +76,36 @@ final class AdminApiController extends Controller {
 			'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
 			'schedule' => $this->schedule->get()['time'],
 			'scheduleTimezone' => $this->schedule->get()['timezone'],
+			'cron' => $this->cron->check(),
 		]);
 	}

+	/** Seconds of backup work one tick() does: well within a normal web request. */
+	public const TICK_SECONDS = 10.0;
+
+	/**
+	 * Move running backups forward from the open admin page, when background jobs do not run on a
+	 * working system cron (#13). The page calls this while a backup runs, so "Back up now" works
+	 * without cron as long as the page stays open; scheduled backups still need cron. With a
+	 * working system cron this does nothing: the background job does the work.
+	 */
+	public function tick(): JSONResponse {
+		if ($this->cron->check()['ok']) {
+			return new JSONResponse(['stepped' => 0]);
+		}
+		$deadline = microtime(true) + self::TICK_SECONDS;
+		$stepped = 0;
+		foreach ($this->runs->findRunning(null, BackupService::KIND_FULL) as $run) {
+			if (microtime(true) >= $deadline) {
+				break;
+			}
+			// step() returns at once when the background job (or another tab) holds the run.
+			$this->backups->step($run, $deadline);
+			$stepped++;
+		}
+		return new JSONResponse(['stepped' => $stepped]);
+	}
+
 	#[PasswordConfirmationRequired]
 	public function initKey(string $passphrase, int $deleteDelayDays = \OCA\NgBackup\Service\KeyService::DEFAULT_DELETE_DELAY, string $label = 'Slot 1'): JSONResponse {
 		try {
@@ -313,6 +341,7 @@ final class AdminApiController extends Controller {
 		return ['id' => $r->getId(), 'targetId' => $r->getTargetId(), 'kind' => $r->getKind(), 'status' => $r->getStatus(),
 			'phase' => $r->getPhase(), 'snapshotId' => $r->getSnapshotId(), 'snapshot' => $snapshot, 'startedAt' => $r->getStartedAt(),
 			'updatedAt' => $r->getUpdatedAt(), 'finishedAt' => $r->getFinishedAt(), 'error' => $r->getError(),
-			'stats' => json_decode($r->getStats() ?? 'null', true), 'progress' => \OCA\NgBackup\Job\BackupRun::progressOf($r)];
+			'stats' => json_decode($r->getStats() ?? 'null', true), 'progress' => \OCA\NgBackup\Job\BackupRun::progressOf($r),
+			'waitingFor' => $r->getKind() === BackupService::KIND_FULL ? \OCA\NgBackup\Service\CronHealth::waitingFor($r) : null];
 	}
 }
@@ -176,12 +176,12 @@ final class BackupRun {
 	}

 	/**
-	 * progress() of a run, while it is in its files phase.
+	 * progress() of a run in its files phase; in its database phase, the tables dumped so far.
 	 *
-	 * @return array{fraction: float, approximate: bool}|null
+	 * @return array{fraction: float, approximate: bool, tables?: int, totalTables?: int}|null
 	 */
 	public static function progressOf(\OCA\NgBackup\Db\Run $run): ?array {
-		if ($run->getStatus() !== \OCA\NgBackup\Db\Run::RUNNING || $run->getPhase() !== 'files') {
+		if ($run->getStatus() !== \OCA\NgBackup\Db\Run::RUNNING || !in_array($run->getPhase(), ['db', 'files'], true)) {
 			return null;
 		}
 		try {
@@ -189,6 +189,14 @@ final class BackupRun {
 		} catch (\JsonException) {
 			return null;
 		}
+		if ($run->getPhase() === 'db') {
+			// Tables dumped so far, as of the last step (none known before the first step ends)
+			$tables = count($state['dump']['tables'] ?? []);
+			return $tables > 0
+				? ['fraction' => min(1.0, ($state['dump']['i'] ?? 0) / $tables), 'approximate' => false,
+					'tables' => (int)($state['dump']['i'] ?? 0), 'totalTables' => $tables]
+				: null;
+		}
 		return is_array($state['files'] ?? null) ? self::progress($state['files']) : null;
 	}

@@ -221,14 +221,21 @@ final class Repository {
 	 *             "$target/<full logical path>" (disaster recovery: restoring e.g. "data/" or
 	 *             "config/" directly into the real data/config directory, not a subdirectory
 	 *             named after the prefix). Ignored when $prefix is '' (nothing to strip).
+	 * @param array<string, string>|null $skipped salvage: an array to collect the files that could
+	 *             not be read or decrypted (path => reason), which are then left out instead of
+	 *             stopping the restore; null (default) stops at the first such file
+	 * @param bool $keepUnchanged leave a file that is already in place with the size and
+	 *             modification time of the snapshot, as a backup takes such a file as unchanged:
+	 *             a restore that was stopped and runs again goes on where it was, without reading
+	 *             what it already restored from the location again
 	 */
-	public function restore(string $snapshotId, string $target, string $prefix = '', bool $stripPrefix = false, ?callable $heartbeat = null): int {
+	public function restore(string $snapshotId, string $target, string $prefix = '', bool $stripPrefix = false, ?callable $heartbeat = null, ?array &$skipped = null, bool $keepUnchanged = false): int {
 		if ($prefix !== '' && !self::isSafePath(rtrim($prefix, '/'))) {
 			throw new RepositoryException('Invalid path prefix');
 		}
-		$plan = $this->readAhead($snapshotId, $prefix, $target);
+		$plan = $this->readAhead($snapshotId, $prefix, $target, $stripPrefix, $keepUnchanged);
 		try {
-			return $this->restoreEntries($snapshotId, $target, $prefix, $stripPrefix, $heartbeat, $plan);
+			return $this->restoreEntries($snapshotId, $target, $prefix, $stripPrefix, $heartbeat, $plan, $skipped, $keepUnchanged);
 		} finally {
 			$plan?->close();
 			if ($plan !== null) {
@@ -242,13 +249,23 @@ final class Repository {
 	 * pass over the snapshot lists the ranges of packs it needs, in order. The spool is a
 	 * folder in the target, which has room for the restore anyway.
 	 */
-	private function readAhead(string $snapshotId, string $prefix, string $target): ?ReadAheadPlan {
+	private function readAhead(string $snapshotId, string $prefix, string $target, bool $stripPrefix, bool $keepUnchanged): ?ReadAheadPlan {
 		if (!$this->backend instanceof IReadsAhead || !$this->backend->canReadAhead()) {
 			return null;
 		}
-		// what a restore that was stopped left behind: empty, its files were unlinked when opened
+		// what a restore that was stopped left behind: its files are unlinked once opened, but one
+		// that was being written when the process was killed stays (8 MiB, DR test 2026-11-05)
 		$stale = glob(rtrim($target, '/') . '/.ngb-restore-spool-*', GLOB_ONLYDIR);
 		foreach ($stale === false ? [] : $stale as $dir) {
+			if (is_link($dir)) {
+				continue;
+			}
+			$files = glob($dir . '/*');
+			foreach ($files === false ? [] : $files as $file) {
+				if (is_file($file) && !is_link($file)) {
+					@unlink($file);
+				}
+			}
 			@rmdir($dir);
 		}
 		$plan = new ReadAheadPlan(rtrim($target, '/') . '/.ngb-restore-spool-' . bin2hex(random_bytes(4)));
@@ -257,6 +274,9 @@ final class Repository {
 				if (!self::underPrefix($entry['p'], $prefix)) {
 					continue;
 				}
+				if ($keepUnchanged && self::inPlace(self::destination($target, $entry['p'], $prefix, $stripPrefix), $entry)) {
+					continue;
+				}
 				foreach ($entry['b'] as $id) {
 					try {
 						[$pack, $offset, $length] = $this->index->get($id);
@@ -285,18 +305,20 @@ final class Repository {
 		return $plan;
 	}

-	private function restoreEntries(string $snapshotId, string $target, string $prefix, bool $stripPrefix, ?callable $heartbeat, ?ReadAheadPlan $plan): int {
+	/** @param array<string, string>|null $skipped see restore() */
+	private function restoreEntries(string $snapshotId, string $target, string $prefix, bool $stripPrefix, ?callable $heartbeat, ?ReadAheadPlan $plan, ?array &$skipped, bool $keepUnchanged): int {
 		$count = 0;
 		$cleaned = [];
 		foreach ($this->streamEntries($snapshotId) as $entry) {
 			if (!self::underPrefix($entry['p'], $prefix)) {
 				continue;
 			}
-			$relPath = $entry['p'];
-			if ($stripPrefix && $prefix !== '') {
-				$relPath = $entry['p'] === rtrim($prefix, '/') ? basename($entry['p']) : substr($entry['p'], strlen($prefix));
+			$dest = self::destination($target, $entry['p'], $prefix, $stripPrefix);
+			if ($keepUnchanged && self::inPlace($dest, $entry)) {
+				// below the target as written by an earlier run of this restore (checked then)
+				$count++;
+				continue;
 			}
-			$dest = rtrim($target, '/') . '/' . $relPath;
 			if (!is_dir(dirname($dest))) {
 				mkdir(dirname($dest), 0700, true);
 			}
@@ -317,16 +339,30 @@ final class Repository {
 			if ($out === false) {
 				throw new RepositoryException('Cannot create a temporary file for ' . $entry['p']);
 			}
-			foreach ($entry['b'] as $id) {
-				fwrite($out, $this->restoreBlob($id, $plan));
-				if ($heartbeat !== null) {
-					$heartbeat();
+			try {
+				foreach ($entry['b'] as $id) {
+					fwrite($out, $this->restoreBlob($id, $plan));
+					if ($heartbeat !== null) {
+						$heartbeat();
+					}
 				}
-			}
-			fclose($out);
-			if (filesize($tmp) !== $entry['s']) {
-				unlink($tmp);
-				throw new RepositoryException('Size mismatch restoring ' . $entry['p']);
+				fclose($out);
+				clearstatcache(true, $tmp);
+				if (filesize($tmp) !== $entry['s']) {
+					throw new RepositoryException('Size mismatch restoring ' . $entry['p']);
+				}
+			} catch (RepositoryException|BackendException|\OCA\NgBackup\Crypto\CryptoException $e) {
+				// What the location gave back for this file is missing or damaged. Writing to the
+				// target (below) is not part of this: a full disk must still stop the restore.
+				if (is_resource($out)) {
+					fclose($out);
+				}
+				@unlink($tmp);
+				if ($skipped === null) {
+					throw $e;
+				}
+				$skipped[$entry['p']] = $e->getMessage();
+				continue;
 			}
 			if (!rename($tmp, $dest)) {
 				@unlink($tmp);
@@ -340,6 +376,26 @@ final class Repository {
 		return $count;
 	}

+	/** Where restore() writes the snapshot path $path. */
+	private static function destination(string $target, string $path, string $prefix, bool $stripPrefix): string {
+		$relPath = $path;
+		if ($stripPrefix && $prefix !== '') {
+			$relPath = $path === rtrim($prefix, '/') ? basename($path) : substr($path, strlen($prefix));
+		}
+		return rtrim($target, '/') . '/' . $relPath;
+	}
+
+	/**
+	 * A regular file (no link) at $dest with the size and modification time of the snapshot
+	 * entry: what a backup takes as unchanged, and what a finished restore of it leaves.
+	 *
+	 * @param array{s: int, m: int, ...} $entry
+	 */
+	private static function inPlace(string $dest, array $entry): bool {
+		clearstatcache(true, $dest);
+		return !is_link($dest) && is_file($dest) && filesize($dest) === $entry['s'] && filemtime($dest) === $entry['m'];
+	}
+
 	/** $prefix is "" (everything) or ends with "/"; the path without that "/" is a file to restore too. */
 	private static function underPrefix(string $path, string $prefix): bool {
 		return $prefix === '' || str_starts_with($path, $prefix) || $path === rtrim($prefix, '/');
@@ -0,0 +1,97 @@
+<?php
+
+declare(strict_types=1);
+
+// SPDX-FileCopyrightText: 2026 Dillard Blom
+// SPDX-License-Identifier: AGPL-3.0-or-later
+
+namespace OCA\NgBackup\Service;
+
+use OCA\NgBackup\Db\Run;
+use OCP\Exceptions\AppConfigTypeConflictException;
+use OCP\IAppConfig;
+use OCP\IURLGenerator;
+
+/**
+ * Whether Nextcloud's background jobs run on a system cron, which backups need: a backup moves
+ * forward only when a background job run takes its next step (#13). Without system cron, AJAX
+ * cron runs a job only now and then while someone has Nextcloud open, and webcron only as often as
+ * the outside service calls it.
+ */
+final class CronHealth {
+	/** A system cron that has not run for this long has stopped (it runs every 5 minutes). */
+	public const STOPPED_AFTER = 3600;
+
+	/**
+	 * A running backup that took no step for this long is waiting for background jobs. A step with
+	 * system cron takes at most a few minutes (BackupJob) and the job runs every minute.
+	 */
+	public const WAITING_AFTER = 900;
+
+	public function __construct(
+		private IAppConfig $appConfig,
+		private IURLGenerator $urls,
+	) {
+	}
+
+	/**
+	 * mode: ajax, webcron or cron; lastRun: the last background job run (0: never); ok: a system
+	 * cron that ran recently; stopped: system cron is set but has not run for STOPPED_AFTER.
+	 *
+	 * @return array{mode: string, lastRun: int, ok: bool, stopped: bool, docs: string}
+	 */
+	public function check(?int $now = null): array {
+		$now ??= time();
+		$mode = $this->coreString('backgroundjobs_mode', 'ajax');
+		$lastRun = (int)$this->coreString('lastcron', '0');
+		$stopped = $mode === 'cron' && $now - $lastRun > self::STOPPED_AFTER;
+		return [
+			'mode' => $mode,
+			'lastRun' => $lastRun,
+			'ok' => $mode === 'cron' && !$stopped,
+			'stopped' => $stopped,
+			'docs' => $this->urls->linkToDocs('admin-background-jobs'),
+		];
+	}
+
+	/**
+	 * Seconds since a running backup took its last step, when that is long enough to say it waits
+	 * for background jobs; null while it moves (or when it is not running).
+	 */
+	public static function waitingFor(Run $run, ?int $now = null): ?int {
+		$idle = ($now ?? time()) - $run->getUpdatedAt();
+		return $run->getStatus() === Run::RUNNING && $idle >= self::WAITING_AFTER ? $idle : null;
+	}
+
+	/** One line for occ and the log; null when system cron runs. */
+	public static function describe(array $health, ?int $now = null): ?string {
+		if ($health['ok']) {
+			return null;
+		}
+		$last = $health['lastRun'] > 0 ? 'last ran ' . self::ago(($now ?? time()) - $health['lastRun']) : 'never ran';
+		$what = $health['stopped']
+			? 'Background jobs are set to system cron, but cron has not run for over an hour (' . $last . ').'
+			: 'Background jobs do not run on a system cron: they are set to "' . ($health['mode'] === 'webcron' ? 'Webcron' : 'AJAX') . '" and ' . $last . '.';
+		return $what . ' Backups, also the scheduled ones, need a system cron that runs every 5 minutes: ' . $health['docs'];
+	}
+
+	/** "12 min ago" style, for durations from a minute to days */
+	public static function ago(int $seconds): string {
+		if ($seconds < 3600) {
+			return max(1, intdiv($seconds, 60)) . ' min ago';
+		}
+		if ($seconds < 172800) {
+			return intdiv($seconds, 3600) . ' h ago';
+		}
+		return intdiv($seconds, 86400) . ' days ago';
+	}
+
+	/** Core stores these with different value types across versions: read whatever is there. */
+	private function coreString(string $key, string $default): string {
+		try {
+			return $this->appConfig->getValueString('core', $key, $default);
+		} catch (AppConfigTypeConflictException) {
+			return (string)$this->appConfig->getValueInt('core', $key, (int)$default);
+		}
+	}
+}
@@ -58,9 +58,12 @@ final class RestoreFullService {
 	 * - tables in the backup that do not exist here: a backup made from 1.0.0-rc.2 on records
 	 *   their structure and the restore creates them, an older one cannot (missingTables).
 	 *
-	 * @return array{ok:bool, currentMajor:string, snapshotMajor:string, missingApps:list<string>, missingTables:list<string>}
+	 * With $salvage, a database dump that cannot be read does not block: dbUnreadable says why, and
+	 * the restore then salvages the files instead (see restoreFull()).
+	 *
+	 * @return array{ok:bool, currentMajor:string, snapshotMajor:string, missingApps:list<string>, missingTables:list<string>, dbUnreadable?:string}
 	 */
-	public function checkCompatibility(Target $target, string $snapshotId, bool $skipMissingTables = false): array {
+	public function checkCompatibility(Target $target, string $snapshotId, bool $skipMissingTables = false, bool $salvage = false): array {
 		$repo = $this->targets->repository($target);
 		$meta = $repo->snapshotMeta($snapshotId);
 		$currentMajor = explode('.', $this->config->getSystemValueString('version'))[0];
@@ -69,8 +72,19 @@ final class RestoreFullService {
 		$missingApps = array_values(array_diff(array_keys($meta['apps'] ?? []), array_keys($enabled), self::ENVIRONMENT_APPS));
 		$missingTables = [];
 		$prefixMismatch = null;
+		$dbUnreadable = null;
+		$manifest = null;
 		if (isset($meta['db'])) {
-			$manifest = json_decode($repo->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR);
+			try {
+				$manifest = json_decode($repo->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR);
+			} catch (\Throwable $e) {
+				if (!$salvage) {
+					throw $e;
+				}
+				$dbUnreadable = 'the database dump cannot be read: ' . $e->getMessage();
+			}
+		}
+		if (is_array($manifest)) {
 			$prefix = $this->config->getSystemValueString('dbtableprefix', 'oc_');
 			$sourcePrefix = DbDumper::dumpPrefix($manifest);
 			if ($sourcePrefix !== null && $sourcePrefix !== $prefix) {
@@ -87,11 +101,21 @@ final class RestoreFullService {
 			'missingApps' => $missingApps,
 			'missingTables' => $prefixMismatch === null ? $missingTables : [],
 			'prefixMismatch' => $prefixMismatch,
-		];
+		] + ($dbUnreadable !== null ? ['dbUnreadable' => $dbUnreadable] : []);
 	}

-	/** @return array{dbReport:array, dataFiles:int, configKeys:int, dbConsistent:bool, missingApps:list<string>, maintenanceLeftOn:bool} */
-	public function restoreFull(Target $target, string $snapshotId, bool $skipMissingTables = false): array {
+	/**
+	 * With $salvage, restore whatever can be restored instead of stopping:
+	 * - a file that cannot be read or decrypted is left out and listed (skipped);
+	 * - when the database cannot be restored here (its dump is unreadable or damaged, or it does
+	 *   not fit this installation), the fresh database stays as it is, and the data directory and
+	 *   config of the backup go to a folder of their own (salvageDir) instead: the files are what
+	 *   matters most, and putting Nextcloud back together from there is work by hand;
+	 * - a config.php that cannot be read is not merged (configError).
+	 *
+	 * @return array{dbReport:array, dataFiles:int, configKeys:int, dbConsistent:bool, missingApps:list<string>, maintenanceLeftOn:bool, skipped:array<string, string>, dbError:?string, salvageDir:?string, configError:?string}
+	 */
+	public function restoreFull(Target $target, string $snapshotId, bool $skipMissingTables = false, bool $salvage = false): array {
 		// Maintenance mode for the whole restore: background jobs (cron.php stops in maintenance
 		// mode), page visits (AJAX cron) and users must not write into tables or the data
 		// directory while they are being replaced; a row written in between would collide with a
@@ -114,17 +138,21 @@ final class RestoreFullService {
 			}
 		};
 		try {
-			$check = $this->checkCompatibility($target, $snapshotId, $skipMissingTables);
+			$check = $this->checkCompatibility($target, $snapshotId, $skipMissingTables, $salvage);
 		} catch (\Throwable $e) {
 			$reopen();
 			throw $e;
 		}
 		if (!$check['ok']) {
-			$reopen();
-			throw new \RuntimeException($this->describeBlockers($check));
+			if (!$salvage) {
+				$reopen();
+				throw new \RuntimeException($this->describeBlockers($check));
+			}
+			// The database cannot go into this installation; the files can.
+			$check['dbUnreadable'] = 'the database cannot be restored here: ' . $this->describeBlockers($check);
 		}
 		$appConfig->setValueInt(\OCA\NgBackup\AppInfo\Application::APP_ID, self::INTERRUPTED_KEY, time());
-		$result = $this->restoreInMaintenance($target, $snapshotId, $skipMissingTables, $check);
+		$result = $this->restoreInMaintenance($target, $snapshotId, $skipMissingTables, $check, $salvage);
 		$appConfig->deleteKey(\OCA\NgBackup\AppInfo\Application::APP_ID, self::INTERRUPTED_KEY);
 		if (!$wasInMaintenance) {
 			$this->config->setSystemValue('maintenance', false);
@@ -132,55 +160,56 @@ final class RestoreFullService {
 		return $result + ['maintenanceLeftOn' => $wasInMaintenance];
 	}

-	/** @return array{dbReport:array, dataFiles:int, configKeys:int, dbConsistent:bool, missingApps:list<string>} */
-	private function restoreInMaintenance(Target $target, string $snapshotId, bool $skipMissingTables, array $check): array {
-		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $skipMissingTables, $check) {
+	/** @return array{dbReport:array, dataFiles:int, configKeys:int, dbConsistent:bool, missingApps:list<string>, skipped:array<string, string>, dbError:?string, salvageDir:?string, configError:?string} */
+	private function restoreInMaintenance(Target $target, string $snapshotId, bool $skipMissingTables, array $check, bool $salvage): array {
+		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $skipMissingTables, $check, $salvage) {
 			$beat = LeaseService::throttled($refresh);
 			$repo = $this->targets->repository($target);
 			$meta = $repo->snapshotMeta($snapshotId);

 			$dbReport = null;
 			$manifest = [];
-			if (isset($meta['db'])) {
-				$manifest = json_decode($repo->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR);
-				$prefix = $this->config->getSystemValueString('dbtableprefix', 'oc_');
-				// ng_backup's own tables (and its own rows in the shared appconfig table) must
-				// survive this restore untouched: they describe the backup/key/target that is
-				// actively performing it right now, on THIS installation, not the one that was
-				// backed up. Overwriting them mid-restore would pull the rug from under it.
-				foreach (array_keys($manifest['tables']) as $table) {
-					/** @var string $table */
-					if (str_starts_with($table, $prefix . 'ngb_')) {
-						unset($manifest['tables'][$table]);
-					}
-				}
-				// Sequences are keyed separately from tables; excluding a table does not exclude
-				// its sequence, which would otherwise rewind a retained ngb_* table's sequence
-				// below its current ids (PostgreSQL only -- see DbDumper::restoreSequences()).
-				foreach (array_keys($manifest['sequences'] ?? []) as $seq) {
-					/** @var string $seq */
-					if (str_starts_with((string)($manifest['sequences'][$seq]['table'] ?? ''), $prefix . 'ngb_')) {
-						unset($manifest['sequences'][$seq]);
-					}
-				}
-				$ownAppConfig = $this->preserveOwnAppConfig();
-				$dumper = new DbDumper($this->db->getInner(), $prefix);
+			$dbError = $check['dbUnreadable'] ?? null;
+			$skipped = $salvage ? [] : null;
+			$dataDir = rtrim($this->config->getSystemValueString('datadirectory'), '/');
+			if ($dbError === null) {
 				try {
-					$dumper->restore($repo, $manifest, $dbReport, $beat, $skipMissingTables);
-				} finally {
-					$this->restoreOwnAppConfig($ownAppConfig);
+					$this->restoreDatabase($repo, $meta, $manifest, $dbReport, $beat, $skipMissingTables);
+				} catch (\Throwable $e) {
+					if (!$salvage) {
+						throw $e;
+					}
+					// DbDumper::restore() rolled its transaction back: the database is the fresh one.
+					$dbError = 'the database could not be restored: ' . $e->getMessage();
+					$dbReport = null;
 				}
 			}
+			if ($dbError !== null) {
+				$salvageDir = $dataDir . '/ngb-salvage-' . substr($snapshotId, 0, 12);
+				$dataFiles = $this->restore->restoreToDirectory($target, $snapshotId, 'data', $salvageDir . '/data', stripPrefix: true, heartbeat: $beat, skipped: $skipped, keepUnchanged: true);
+				$this->restore->restoreToDirectory($target, $snapshotId, 'config', $salvageDir . '/config', stripPrefix: true, heartbeat: $beat, skipped: $skipped);
+				return ['dbReport' => ['skipped' => [], 'restored' => [], 'created' => [], 'missing' => []], 'dataFiles' => $dataFiles, 'configKeys' => 0,
+					'dbConsistent' => true, 'missingApps' => $check['missingApps'], 'skipped' => $skipped ?? [], 'dbError' => $dbError,
+					'salvageDir' => $salvageDir, 'configError' => null];
+			}

 			// Read (but do not yet apply) the backed-up config.php before touching the data
 			// directory: once applied, 'secret' may differ from what ng_backup's own Target
 			// options were encrypted with (see restoreFull()'s docblock), and every further
 			// repository access through $this->targets would fail to decrypt them.
-			/** @var array<string, mixed> $backupConfig */
-			$backupConfig = $this->fetchBackupConfig($target, $snapshotId);
+			$configError = null;
+			try {
+				/** @var array<string, mixed> $backupConfig */
+				$backupConfig = $this->fetchBackupConfig($target, $snapshotId, $salvage);
+			} catch (\Throwable $e) {
+				if (!$salvage) {
+					throw $e;
+				}
+				$configError = $e->getMessage();
+				$backupConfig = [];
+			}

-			$dataDir = rtrim($this->config->getSystemValueString('datadirectory'), '/');
-			$dataFiles = $this->restore->restoreToDirectory($target, $snapshotId, 'data', $dataDir, stripPrefix: true, heartbeat: $beat);
+			$dataFiles = $this->restore->restoreToDirectory($target, $snapshotId, 'data', $dataDir, stripPrefix: true, heartbeat: $beat, skipped: $skipped, keepUnchanged: true);

 			// Last step: nothing after this may need $this->targets->repository($target) again.
 			if ($backupConfig !== []) {
@@ -190,15 +219,63 @@ final class RestoreFullService {
 			}

 			return ['dbReport' => $dbReport ?? ['skipped' => [], 'restored' => [], 'created' => [], 'missing' => []], 'dataFiles' => $dataFiles, 'configKeys' => count($backupConfig),
-				'dbConsistent' => (bool)($manifest['consistent'] ?? true), 'missingApps' => $check['missingApps']];
+				'dbConsistent' => (bool)($manifest['consistent'] ?? true), 'missingApps' => $check['missingApps'],
+				'skipped' => $skipped ?? [], 'dbError' => null, 'salvageDir' => null, 'configError' => $configError];
 		});
 	}

-	/** @return array<string, mixed> config.php's $CONFIG from the backup, minus the connection keys */
-	private function fetchBackupConfig(Target $target, string $snapshotId): array {
+	/**
+	 * Replace the database with the snapshot's dump (one transaction, see DbDumper::restore()).
+	 *
+	 * @param array<string, mixed> $meta the snapshot's meta record
+	 * @param array $manifest set to the dump's manifest
+	 */
+	private function restoreDatabase(\OCA\NgBackup\Repository\Repository $repo, array $meta, array &$manifest, ?array &$dbReport, callable $beat, bool $skipMissingTables): void {
+		if (!isset($meta['db'])) {
+			return;
+		}
+		$manifest = json_decode($repo->getObject($meta['db']), true, 512, JSON_THROW_ON_ERROR);
+		$prefix = $this->config->getSystemValueString('dbtableprefix', 'oc_');
+		// ng_backup's own tables (and its own rows in the shared appconfig table) must
+		// survive this restore untouched: they describe the backup/key/target that is
+		// actively performing it right now, on THIS installation, not the one that was
+		// backed up. Overwriting them mid-restore would pull the rug from under it.
+		foreach (array_keys($manifest['tables']) as $table) {
+			/** @var string $table */
+			if (str_starts_with($table, $prefix . 'ngb_')) {
+				unset($manifest['tables'][$table]);
+			}
+		}
+		// Sequences are keyed separately from tables; excluding a table does not exclude
+		// its sequence, which would otherwise rewind a retained ngb_* table's sequence
+		// below its current ids (PostgreSQL only -- see DbDumper::restoreSequences()).
+		foreach (array_keys($manifest['sequences'] ?? []) as $seq) {
+			/** @var string $seq */
+			if (str_starts_with((string)($manifest['sequences'][$seq]['table'] ?? ''), $prefix . 'ngb_')) {
+				unset($manifest['sequences'][$seq]);
+			}
+		}
+		$ownAppConfig = $this->preserveOwnAppConfig();
+		$dumper = new DbDumper($this->db->getInner(), $prefix);
+		try {
+			$dumper->restore($repo, $manifest, $dbReport, $beat, $skipMissingTables);
+		} finally {
+			$this->restoreOwnAppConfig($ownAppConfig);
+		}
+	}
+
+	/**
+	 * @return array<string, mixed> config.php's $CONFIG from the backup, minus the connection keys
+	 * @throws \RuntimeException with $salvage, when config.php itself cannot be read
+	 */
+	private function fetchBackupConfig(Target $target, string $snapshotId, bool $salvage = false): array {
 		$tmp = sys_get_temp_dir() . '/ngb-restore-config-' . bin2hex(random_bytes(6));
 		try {
-			$this->restore->restoreToDirectory($target, $snapshotId, 'config', $tmp, stripPrefix: true);
+			$skipped = $salvage ? [] : null;
+			$this->restore->restoreToDirectory($target, $snapshotId, 'config', $tmp, stripPrefix: true, skipped: $skipped);
+			if (isset($skipped['config/config.php'])) {
+				throw new \RuntimeException('config.php cannot be read: ' . $skipped['config/config.php']);
+			}
 			$path = $tmp . '/config.php';
 			if (!is_file($path)) {
 				return []; // nothing to merge; the live config.php stays as occ maintenance:install made it
@@ -84,8 +84,8 @@ final class RestoreService {
 	 *
 	 * @return array{restored:int, trashed:int, target:string}
 	 */
-	public function restoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode = self::MODE_NEW_FOLDER, ?callable $progress = null): array {
-		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $logicalPath, $mode, $progress) {
+	public function restoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode = self::MODE_NEW_FOLDER, ?callable $progress = null, ?array &$skipped = null): array {
+		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $logicalPath, $mode, $progress, &$skipped) {
 			$last = time();
 			// Keep the lease alive while files are being written.
 			$beat = function (int $done, int $total, string $name) use ($refresh, $progress, &$last): void {
@@ -97,11 +97,12 @@ final class RestoreService {
 					$progress($done, $total, $name);
 				}
 			};
-			return $this->doRestoreUserFiles($target, $snapshotId, $logicalPath, $mode, $beat);
+			return $this->doRestoreUserFiles($target, $snapshotId, $logicalPath, $mode, $beat, $skipped);
 		});
 	}

-	private function doRestoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode, ?callable $progress): array {
+	/** @param array<string, string>|null $skipped see Repository::restore() */
+	private function doRestoreUserFiles(Target $target, string $snapshotId, string $logicalPath, string $mode, ?callable $progress, ?array &$skipped): array {
 		if (!Repository::isSafePath($logicalPath) || !preg_match('#^data/([^/]+)/files(?:/(.*))?$#', $logicalPath, $m)) {
 			throw new \InvalidArgumentException('Expected a path like data/<user>/files/<folder or file>');
 		}
@@ -139,7 +140,16 @@ final class RestoreService {
 		$this->cleaned = [];
 		$restored = 0;
 		foreach ($names as $relPath => $entry) {
-			$this->writeFile($repo, $dest, (string)$relPath, $entry);
+			try {
+				$this->writeFile($repo, $dest, (string)$relPath, $entry);
+			} catch (\OCA\NgBackup\Repository\RepositoryException|\OCA\NgBackup\Backend\BackendException|\OCA\NgBackup\Crypto\CryptoException $e) {
+				// Damaged or missing on the location (writeFile removed its temporary)
+				if ($skipped === null) {
+					throw $e;
+				}
+				$skipped[$entry['p']] = $e->getMessage();
+				continue;
+			}
 			$restored++;
 			if ($progress !== null) {
 				$progress($restored, count($names), (string)$relPath);
@@ -159,15 +169,18 @@ final class RestoreService {
 	 * @param bool $stripPrefix write into $directory itself instead of $directory/$prefix (see
 	 *             Repository::restore()); used to restore "data/" or "config/" straight into the
 	 *             real data/config directory for occ backup:restore:full.
+	 * @param bool $keepUnchanged leave files already in place with the snapshot's size and
+	 *             modification time (see Repository::restore()): a full restore that runs again
+	 *             goes on where it stopped
 	 */
-	public function restoreToDirectory(Target $target, string $snapshotId, string $prefix, string $directory, bool $stripPrefix = false, ?callable $heartbeat = null): int {
+	public function restoreToDirectory(Target $target, string $snapshotId, string $prefix, string $directory, bool $stripPrefix = false, ?callable $heartbeat = null, ?array &$skipped = null, bool $keepUnchanged = false): int {
 		if ($prefix !== '' && !Repository::isSafePath($prefix)) {
 			throw new \InvalidArgumentException('Invalid path: ' . $prefix);
 		}
 		if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) {
 			throw new \RuntimeException("Cannot create $directory");
 		}
-		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $prefix, $directory, $stripPrefix, $heartbeat): int {
+		return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $prefix, $directory, $stripPrefix, $heartbeat, &$skipped, $keepUnchanged): int {
 			$own = LeaseService::throttled($refresh);
 			$beat = static function () use ($own, $heartbeat): void {
 				$own();
@@ -175,7 +188,7 @@ final class RestoreService {
 					$heartbeat();
 				}
 			};
-			return $this->targets->repository($target)->restore($snapshotId, $directory, trim($prefix, '/') === '' ? '' : trim($prefix, '/') . '/', $stripPrefix, $beat);
+			return $this->targets->repository($target)->restore($snapshotId, $directory, trim($prefix, '/') === '' ? '' : trim($prefix, '/') . '/', $stripPrefix, $beat, $skipped, $keepUnchanged);
 		}, 3600);
 	}

@@ -203,7 +216,7 @@ final class RestoreService {
 				throw new \RuntimeException('Write failed for ' . $relPath);
 			}
 			if ($written !== $entry['s']) {
-				throw new \RuntimeException("Size mismatch restoring $relPath");
+				throw new \OCA\NgBackup\Repository\RepositoryException("Size mismatch restoring $relPath");
 			}
 			if ($folder->nodeExists($name)) {
 				$existing = $folder->get($name);
@@ -12,11 +12,22 @@
 			</p>
 			<NcLoadingIcon v-if="!status" :size="32" />
 			<NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard>
+			<NcNoteCard v-if="status && !status.cron.ok" type="warning">
+				<p>
+					{{ status.cron.stopped
+						? t('ng_backup', 'Background jobs are set to system cron, but cron has not run since {time}.', { time: lastCron })
+						: t('ng_backup', 'Background jobs do not run on a system cron (they are set to "{mode}").', { mode: status.cron.mode === 'webcron' ? 'Webcron' : 'AJAX' }) }}
+					{{ t('ng_backup', 'Backups need a system cron that runs every 5 minutes, also the scheduled ones.') }}
+					<a :href="status.cron.docs" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'How to set up cron ↗') }}</a>
+				</p>
+				<p>{{ t('ng_backup', 'Until then, a backup started with "Back up now" only moves on while this page is open.') }}</p>
+			</NcNoteCard>
 		</NcSettingsSection>

 		<template v-if="status">
 			<div class="ngb-tabs">
 				<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
+				<NcButton v-if="status.key.everConfirmed && status.targets.length" :variant="tab === 'bin' ? 'primary' : 'secondary'" @click="tab = 'bin'">{{ t('ng_backup', 'Bin') }}</NcButton>
 				<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
 				<NcButton :variant="tab === 'docs' ? 'primary' : 'secondary'" @click="tab = 'docs'">{{ t('ng_backup', 'Docs') }}</NcButton>
 			</div>
@@ -29,6 +40,7 @@
 					</li>
 				</ul>
 			</NcSettingsSection>
+			<Bin v-else-if="tab === 'bin' && status.key.everConfirmed && status.targets.length" :status="status" @changed="refresh" />
 			<template v-else-if="tab === 'settings'">
 				<KeySetup :status="status" @changed="refresh" />
 				<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
@@ -46,7 +58,7 @@
 </template>

 <script setup>
-import { onBeforeUnmount, onMounted, ref } from 'vue'
+import { computed, onBeforeUnmount, onMounted, ref } from 'vue'
 import { t } from '@nextcloud/l10n'
 import NcButton from '@nextcloud/vue/components/NcButton'
 import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
@@ -54,6 +66,7 @@ import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
 import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
 import api from './api.js'
 import Backups from './components/Backups.vue'
+import Bin from './components/Bin.vue'
 import KeySetup from './components/KeySetup.vue'
 import Locations from './components/Locations.vue'
 import RestoreBrowser from './components/RestoreBrowser.vue'
@@ -105,13 +118,27 @@ async function refresh() {
 	} catch (e) {
 		error.value = e.message
 	}
-	// Poll while something runs, so progress shows up without reloading.
+	// Poll while something runs, so progress shows up without reloading. Without a working
+	// system cron, each poll also lets a running backup take a short step (see tick()).
 	clearTimeout(timer)
 	if (status.value?.runs.some(r => r.status === 'running')) {
-		timer = setTimeout(refresh, 4000)
+		timer = setTimeout(tickAndRefresh, 4000)
 	}
 }

+async function tickAndRefresh() {
+	if (status.value && !status.value.cron.ok && status.value.runs.some(r => r.status === 'running' && r.kind === 'full')) {
+		try {
+			await api.tick()
+		} catch (e) {
+			// The next poll tries again; the error of a failed step shows on its run.
+		}
+	}
+	await refresh()
+}
+
+const lastCron = computed(() => status.value?.cron.lastRun ? new Date(status.value.cron.lastRun * 1000).toLocaleString() : '')
+
 onMounted(refresh)
 onBeforeUnmount(() => clearTimeout(timer))
 </script>
@@ -33,6 +33,7 @@ export default {
 	testTarget: (id) => call('post', `/targets/${id}/test`),
 	setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
 	startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
+	tick: () => call('post', '/tick'),
 	setSchedule: (time, timezone) => call('put', '/schedule', { time, timezone }),
 	snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
 	browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
@@ -18,34 +18,6 @@
 		</div>
 		<NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard>

-		<template v-for="target in status.targets" :key="'trash' + target.id">
-			<div v-if="trash[target.id]?.length" class="ngb-trash">
-				<h3>{{ t('ng_backup', 'Trash of {name}', { name: target.name }) }}</h3>
-				<p class="ngb-muted">{{ t('ng_backup', 'Removed restore points can be brought back until they are deleted permanently.') }}</p>
-				<ul>
-					<li v-for="item in trash[target.id]" :key="item.id">
-						{{ item.time ? formatDateTime(item.time) : item.id }}{{ item.label ? ' – ' + item.label : '' }}
-						<span class="ngb-muted">{{ t('ng_backup', 'removed by {by}, deleted after {date}', { by: item.by, date: formatDate(item.deleteAfter) }) }}</span>
-						<button v-if="item.by === RETENTION"
-							class="ngb-info"
-							:aria-expanded="retentionInfo ? 'true' : 'false'"
-							:aria-label="t('ng_backup', 'What does this mean?')"
-							:title="t('ng_backup', 'What does this mean?')"
-							@click="retentionInfo = !retentionInfo">
-							ⓘ
-						</button>
-						<NcButton variant="tertiary" @click="untrash(target, item)">{{ t('ng_backup', 'Restore') }}</NcButton>
-					</li>
-				</ul>
-				<NcNoteCard v-if="retentionInfo && trash[target.id].some(item => item.by === RETENTION)" type="info">
-					{{ n('ng_backup',
-						'The retention policy keeps the newest restore points, and for older days, weeks and months only the last restore point of each. It moves the others to this trash, where they stay for %n day before they are deleted permanently. Until then you can bring one back with "Restore". Nothing was lost by accident.',
-						'The retention policy keeps the newest restore points, and for older days, weeks and months only the last restore point of each. It moves the others to this trash, where they stay for %n days before they are deleted permanently. Until then you can bring one back with "Restore". Nothing was lost by accident.',
-						status.key.deleteDelayDays) }}
-				</NcNoteCard>
-			</div>
-		</template>
-
 		<div v-if="status.runs.length" class="ngb-table-wrap">
 			<table class="ngb-table">
 				<thead>
@@ -66,7 +38,7 @@
 						<td>{{ targetName(run.targetId) }}</td>
 						<td>{{ run.kind }}</td>
 						<td>{{ run.snapshot ? formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}</td>
-						<td :class="'ngb-' + run.status">{{ run.status }}{{ run.status === 'running' ? ' (' + run.phase + percent(run) + ')' : '' }}{{ run.error ? ': ' + run.error : '' }}</td>
+						<td :class="'ngb-' + run.status">{{ runStatus(run) }}{{ run.error ? ': ' + run.error : '' }}</td>
 						<td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
 						<td>{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td>
 						<td>{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}</td>
@@ -95,9 +67,7 @@ import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
 import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
 import NcTextField from '@nextcloud/vue/components/NcTextField'
 import api from '../api.js'
-import { formatDateTime, formatDate, formatSize, toZone } from '../format.js'
-import { confirmPassword } from '@nextcloud/password-confirmation'
-import { reactive } from 'vue'
+import { formatDateTime, formatSize, toZone } from '../format.js'

 const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
 const emit = defineEmits(['changed', 'show-all-runs'])
@@ -110,35 +80,29 @@ const error = ref('')
 watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })

 const label = ref('')
-// "by" of a restore point the weekly cleanup removed (PruneService)
-const RETENTION = 'retention policy'
-const retentionInfo = ref(false)
-const trash = reactive({})
-async function loadTrash() {
-	for (const target of props.status.targets) {
-		try {
-			trash[target.id] = await api.trash(target.id)
-		} catch (e) {
-			trash[target.id] = []
-		}
-	}
-}
-watch(() => props.status.targets.map(tg => tg.id).join(','), loadTrash, { immediate: true })

-async function untrash(target, item) {
-	error.value = ''
-	try {
-		await confirmPassword()
-		await api.untrash(target.id, item.id)
-		await loadTrash()
-		emit('changed')
-	} catch (e) {
-		error.value = e?.message || ''
+// ": 47/183 tables" in the database phase; " 68%", or " ~68%" when the total is estimated from
+// Nextcloud's file cache (first backup)
+const percent = (run) => {
+	if (!run.progress) {
+		return ''
 	}
+	if (run.progress.totalTables) {
+		return ': ' + t('ng_backup', '{done}/{total} tables', { done: run.progress.tables, total: run.progress.totalTables })
+	}
+	return ' ' + (run.progress.approximate ? '~' : '') + Math.floor(run.progress.fraction * 100) + '%'
+}
+// "running (db: 47/183 tables)", or "waiting for background jobs (…, last step 12 min ago)"
+const runStatus = (run) => {
+	if (run.status !== 'running') {
+		return run.status
+	}
+	const detail = run.phase + percent(run)
+	if (run.waitingFor) {
+		return t('ng_backup', 'waiting for background jobs ({detail}, last step {minutes} min ago)', { detail, minutes: Math.floor(run.waitingFor / 60) })
+	}
+	return run.status + ' (' + detail + ')'
 }
-
-// " 68%", or " ~68%" when the total is estimated from Nextcloud's file cache (first backup)
-const percent = (run) => run.progress ? ' ' + (run.progress.approximate ? '~' : '') + Math.floor(run.progress.fraction * 100) + '%' : ''
 const isRunning = (targetId) => props.status.runs.some(r => r.targetId === targetId && r.kind === 'full' && r.status === 'running')
 const targetName = (id) => props.status.targets.find(tg => tg.id === id)?.name ?? '#' + id

@@ -168,13 +132,10 @@ async function saveSchedule() {
 .ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
 .ngb-time { max-width: 260px; }
 .ngb-label { max-width: 360px; }
-.ngb-info { background: none; border: none; padding: 0 4px; min-height: auto; cursor: pointer; color: var(--color-primary-element); }
 .ngb-table-wrap { overflow-x: auto; }
 .ngb-table { width: 100%; border-collapse: collapse; }
 .ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); }
 .ngb-failed { color: var(--color-error-text); }
 .ngb-muted { color: var(--color-text-maxcontrast); }
-.ngb-trash { margin: 12px 0; }
-.ngb-trash h3 { font-weight: bold; }
 .ngb-done { color: var(--color-success-text); }
 </style>
@@ -0,0 +1,93 @@
+<!--
+  - SPDX-FileCopyrightText: 2026 Dillard Blom
+  - SPDX-License-Identifier: AGPL-3.0-or-later
+-->
+<template>
+	<NcSettingsSection :name="t('ng_backup', 'Bin')"
+		:description="t('ng_backup', 'Restore points that were removed, most of them by the retention policy. They can be brought back until they are deleted permanently.')">
+		<NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard>
+		<NcLoadingIcon v-if="loading" :size="32" />
+		<NcNoteCard v-else-if="empty" type="info">{{ t('ng_backup', 'The bin is empty.') }}</NcNoteCard>
+
+		<template v-for="target in status.targets" :key="'trash' + target.id">
+			<div v-if="trash[target.id]?.length" class="ngb-trash">
+				<h3>{{ status.targets.length > 1 ? target.name : t('ng_backup', 'Removed restore points') }}</h3>
+				<ul>
+					<li v-for="item in trash[target.id]" :key="item.id">
+						{{ item.time ? formatDateTime(item.time) : item.id }}{{ item.label ? ' – ' + item.label : '' }}
+						<span class="ngb-muted">{{ t('ng_backup', 'removed by {by}, deleted after {date}', { by: item.by, date: formatDate(item.deleteAfter) }) }}</span>
+						<button v-if="item.by === RETENTION"
+							class="ngb-info"
+							:aria-expanded="retentionInfo ? 'true' : 'false'"
+							:aria-label="t('ng_backup', 'What does this mean?')"
+							:title="t('ng_backup', 'What does this mean?')"
+							@click="retentionInfo = !retentionInfo">
+							ⓘ
+						</button>
+						<NcButton variant="tertiary" @click="untrash(target, item)">{{ t('ng_backup', 'Restore') }}</NcButton>
+					</li>
+				</ul>
+				<NcNoteCard v-if="retentionInfo && trash[target.id].some(item => item.by === RETENTION)" type="info">
+					{{ n('ng_backup',
+						'The retention policy keeps the newest restore points, and for older days, weeks and months only the last restore point of each. It moves the others to this bin, where they stay for %n day before they are deleted permanently. Until then you can bring one back with "Restore". Nothing was lost by accident.',
+						'The retention policy keeps the newest restore points, and for older days, weeks and months only the last restore point of each. It moves the others to this bin, where they stay for %n days before they are deleted permanently. Until then you can bring one back with "Restore". Nothing was lost by accident.',
+						status.key.deleteDelayDays) }}
+				</NcNoteCard>
+			</div>
+		</template>
+	</NcSettingsSection>
+</template>
+
+<script setup>
+import { computed, reactive, ref, watch } from 'vue'
+import { n, t } from '@nextcloud/l10n'
+import { confirmPassword } from '@nextcloud/password-confirmation'
+import NcButton from '@nextcloud/vue/components/NcButton'
+import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
+import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
+import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
+import api from '../api.js'
+import { formatDate, formatDateTime } from '../format.js'
+
+const props = defineProps({ status: { type: Object, required: true } })
+const emit = defineEmits(['changed'])
+
+// "by" of a restore point the weekly cleanup removed (PruneService)
+const RETENTION = 'retention policy'
+const retentionInfo = ref(false)
+const trash = reactive({})
+const loading = ref(true)
+const error = ref('')
+const empty = computed(() => props.status.targets.every(target => !trash[target.id]?.length))
+
+async function loadTrash() {
+	for (const target of props.status.targets) {
+		try {
+			trash[target.id] = await api.trash(target.id)
+		} catch (e) {
+			trash[target.id] = []
+		}
+	}
+	loading.value = false
+}
+watch(() => props.status.targets.map(tg => tg.id).join(','), loadTrash, { immediate: true })
+
+async function untrash(target, item) {
+	error.value = ''
+	try {
+		await confirmPassword()
+		await api.untrash(target.id, item.id)
+		await loadTrash()
+		emit('changed')
+	} catch (e) {
+		error.value = e?.message || ''
+	}
+}
+</script>
+
+<style scoped>
+.ngb-info { background: none; border: none; padding: 0 4px; min-height: auto; cursor: pointer; color: var(--color-primary-element); }
+.ngb-muted { color: var(--color-text-maxcontrast); }
+.ngb-trash { margin: 12px 0; }
+.ngb-trash h3 { font-weight: bold; }
+</style>
@@ -12,7 +12,7 @@
 			<NcPasswordField v-model="pass2" :label="t('ng_backup', 'Repeat passphrase')" autocomplete="new-password"
 				:error="pass2 !== '' && pass1 !== pass2" :helper-text="pass2 !== '' && pass1 !== pass2 ? t('ng_backup', 'The passphrases do not match') : ''" />
 			<NcTextField v-model="delay" type="number" min="1" max="365"
-				:label="t('ng_backup', 'Days a removed backup stays in the trash before it is really deleted')"
+				:label="t('ng_backup', 'Days a removed backup stays in the Bin before it is really deleted')"
 				:helper-text="t('ng_backup', 'Protects against a malicious or mistaken cleanup. Cannot be changed later.')" />
 			<NcButton type="submit" variant="primary" :disabled="busy || pass1.length < 16 || pass1 !== pass2 || !(delay >= 1 && delay <= 365)">
 				{{ t('ng_backup', 'Create backup key') }}
@@ -22,7 +22,7 @@
 		<template v-else>
 			<p>
 				{{ t('ng_backup', 'Key fingerprint: {fingerprint}', { fingerprint: status.key.fingerprint }) }} ·
-				{{ t('ng_backup', 'Removed backups are kept in the trash for {days} days', { days: status.key.deleteDelayDays }) }}
+				{{ t('ng_backup', 'Removed backups are kept in the Bin for {days} days', { days: status.key.deleteDelayDays }) }}
 			</p>

 			<!-- Passphrase slots -->
@@ -0,0 +1,30 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OCA\NgBackup\Tests\Unit\Command;
+
+use OCA\NgBackup\Command\TargetAdd;
+use PHPUnit\Framework\TestCase;
+
+class TargetAddEstimateTest extends TestCase {
+	public function testNoEstimateFromTheFirstFewFiles(): void {
+		// 4,568 index files, 10 in after 6 s: too few to say anything
+		$this->assertNull(TargetAdd::minutesLeft(6.0, 10, 4558, 4568));
+	}
+
+	public function testEstimateOnceEnoughFilesCameIn(): void {
+		// 300 files in 10 s: 4,268 left at 30 a second is about 2.4 minutes
+		$this->assertSame(3, TargetAdd::minutesLeft(10.0, 300, 4268, 4568));
+	}
+
+	public function testFivePercentOfASmallIndexIsEnough(): void {
+		// 6 of 100 files in 6 s: 94 left at one a second is about 1.6 minutes
+		$this->assertSame(2, TargetAdd::minutesLeft(6.0, 6, 94, 100));
+	}
+
+	public function testNoEstimateInTheFirstSecondsOrWhenDone(): void {
+		$this->assertNull(TargetAdd::minutesLeft(2.0, 500, 4000, 4500));
+		$this->assertNull(TargetAdd::minutesLeft(60.0, 4568, 0, 4568));
+	}
+}
@@ -24,4 +24,25 @@ class BackupProgressTest extends TestCase {
 		$this->assertNull(BackupRun::progress(['phase' => 'files', 'estimate' => ['bytes' => 0, 'approximate' => true], 'stats' => ['bytes' => 1]]));
 		$this->assertNull(BackupRun::progress(['phase' => 'commit', 'estimate' => ['bytes' => 10, 'approximate' => false], 'stats' => ['bytes' => 1]]));
 	}
+
+	private static function runIn(string $phase, array $state, string $status = \OCA\NgBackup\Db\Run::RUNNING): \OCA\NgBackup\Db\Run {
+		$run = new \OCA\NgBackup\Db\Run();
+		$run->setStatus($status);
+		$run->setPhase($phase);
+		$run->setState(json_encode($state, JSON_THROW_ON_ERROR));
+		return $run;
+	}
+
+	public function testDatabasePhaseCountsTheTablesDumped(): void {
+		$run = self::runIn('db', ['dump' => ['tables' => array_fill(0, 183, 'oc_t'), 'i' => 47]]);
+		$this->assertSame(['fraction' => 47 / 183, 'approximate' => false, 'tables' => 47, 'totalTables' => 183], BackupRun::progressOf($run));
+	}
+
+	public function testDatabasePhaseHasNoFigureBeforeItsFirstStep(): void {
+		$this->assertNull(BackupRun::progressOf(self::runIn('db', ['files' => []])));
+	}
+
+	public function testNoProgressForAFinishedRun(): void {
+		$this->assertNull(BackupRun::progressOf(self::runIn('db', ['dump' => ['tables' => ['oc_a'], 'i' => 0]], \OCA\NgBackup\Db\Run::DONE)));
+	}
 }
@@ -75,11 +75,14 @@ class ReadAheadTest extends TestCase {
 		$this->assertSame(2, $backend->snapshotReads, 'the restore itself read the snapshot again');
 	}

-	public function testTheEmptySpoolOfAStoppedRestoreIsRemoved(): void {
+	public function testTheSpoolOfAStoppedRestoreIsRemoved(): void {
 		$target = $this->tempDir();
 		mkdir("$target/.ngb-restore-spool-0badc0de", 0700); // a restore that was killed
+		mkdir("$target/.ngb-restore-spool-8badf00d", 0700); // ... while it wrote a range
+		file_put_contents("$target/.ngb-restore-spool-8badf00d/smb-01234567", str_repeat('x', 4096));

 		$this->assertRestored($this->restore(new ReadAheadBackend(new LocalBackend($this->root)), '', $target));
+		$this->assertSame([], glob("$target/.ngb-restore-spool-*"));
 	}

 	public function testARangeThatCouldNotBeReadAheadIsReadTheUsualWay(): void {
@@ -101,6 +101,34 @@ class RepositoryTest extends TestCase {
 		$this->assertFileExists($stripped . '/note-1.txt');
 	}

+	public function testARestoreThatRunsAgainKeepsWhatIsInPlaceAndRestoresTheRest(): void {
+		$location = $this->tempDir();
+		$backend = new LocalBackend($location);
+		$repo = Repository::init($backend, 'pw');
+		$r = $repo->backupDirectory($this->src);
+		$target = $this->tempDir();
+		$repo->restore($r['snapshot'], $target);
+
+		// as a stopped restore leaves it: one file not there yet, one with other content
+		unlink($target . '/docs/d1/note-1.txt');
+		file_put_contents($target . '/docs/d0/note-0.txt', 'something else');
+		$this->assertSame(34, $repo->restore($r['snapshot'], $target, keepUnchanged: true));
+		$this->assertSame(self::hashTree($this->src), self::hashTree($target));
+
+		// all in place: nothing is read from the location, which could not give it anyway now
+		self::removeTree($location . '/packs');
+		$this->assertSame(34, Repository::open($backend, 'pw')->restore($r['snapshot'], $target, keepUnchanged: true));
+		$this->expectException(\Throwable::class);
+		Repository::open($backend, 'pw')->restore($r['snapshot'], $target);
+	}
+
+	private static function removeTree(string $dir): void {
+		foreach (new \RecursiveIteratorIterator(new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS), \RecursiveIteratorIterator::CHILD_FIRST) as $f) {
+			$f->isDir() ? rmdir($f->getPathname()) : unlink($f->getPathname());
+		}
+		rmdir($dir);
+	}
+
 	public function testRestoreRemovesTemporariesOfAStoppedRestore(): void {
 		$repo = Repository::init($this->backend, 'pw');
 		$r = $repo->backupDirectory($this->src);
@@ -0,0 +1,88 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OCA\NgBackup\Tests\Unit\Repository;
+
+use OCA\NgBackup\Backend\LocalBackend;
+use OCA\NgBackup\Crypto\KeyRing;
+use OCA\NgBackup\Repository\Repository;
+use OCA\NgBackup\Tests\Unit\TempDirTrait;
+use PHPUnit\Framework\TestCase;
+
+/**
+ * --salvage: a file whose data is damaged on the location is left out and listed, the rest of the
+ * restore goes on. Without it, the restore stops at that file.
+ */
+class SalvageTest extends TestCase {
+	use TempDirTrait;
+
+	private string $root;
+	private KeyRing $keys;
+	private string $src;
+	private string $snapshot;
+
+	protected function setUp(): void {
+		$this->root = $this->tempDir();
+		$this->keys = KeyRing::generate();
+		$this->src = $this->tempDir();
+		mkdir("$this->src/docs");
+		// Incompressible, 2 MiB each: every file is a blob of its own in one pack
+		foreach (['a', 'b', 'c'] as $name) {
+			file_put_contents("$this->src/docs/$name.bin", random_bytes(2 * 1048576));
+		}
+		file_put_contents("$this->src/note.txt", "small\n");
+		$repo = Repository::initWithKey(new LocalBackend($this->root), $this->keys, $this->keys->wrap('pw'));
+		$this->snapshot = $repo->backupDirectory($this->src)['snapshot'];
+	}
+
+	/** Flip bytes in the middle of the (only) pack: inside one of the 2 MiB files. */
+	private function damagePack(): void {
+		$packs = glob($this->root . '/packs/*/*');
+		$this->assertCount(1, $packs);
+		$fh = fopen($packs[0], 'r+b');
+		$middle = intdiv(filesize($packs[0]), 2);
+		fseek($fh, $middle);
+		$bytes = fread($fh, 16);
+		fseek($fh, $middle);
+		fwrite($fh, ~$bytes);
+		fclose($fh);
+	}
+
+	public function testSalvageLeavesOutTheDamagedFileAndRestoresTheRest(): void {
+		$this->damagePack();
+		$target = $this->tempDir();
+		$skipped = [];
+
+		$restored = Repository::openWithKey(new LocalBackend($this->root), $this->keys)
+			->restore($this->snapshot, $target, '', false, null, $skipped);
+
+		$this->assertSame(3, $restored);
+		$this->assertCount(1, $skipped);
+		$damaged = array_key_first($skipped);
+		$this->assertMatchesRegularExpression('#^docs/[abc]\.bin$#', $damaged);
+		$this->assertFileDoesNotExist("$target/$damaged");
+		foreach (['docs/a.bin', 'docs/b.bin', 'docs/c.bin', 'note.txt'] as $file) {
+			if ($file !== $damaged) {
+				$this->assertFileEquals("$this->src/$file", "$target/$file");
+			}
+		}
+		$this->assertSame([], glob("$target/docs/*.ngb-restore-*"), 'no temporary left of the damaged file');
+	}
+
+	public function testWithoutSalvageTheRestoreStopsAtTheDamagedFile(): void {
+		$this->damagePack();
+
+		$this->expectException(\Exception::class);
+		Repository::openWithKey(new LocalBackend($this->root), $this->keys)->restore($this->snapshot, $this->tempDir());
+	}
+
+	public function testSalvageOfAnUndamagedSnapshotSkipsNothing(): void {
+		$target = $this->tempDir();
+		$skipped = [];
+
+		$this->assertSame(4, Repository::openWithKey(new LocalBackend($this->root), $this->keys)
+			->restore($this->snapshot, $target, '', false, null, $skipped));
+		$this->assertSame([], $skipped);
+	}
+}
@@ -0,0 +1,88 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OCA\NgBackup\Tests\Unit\Service;
+
+use OCA\NgBackup\Db\Run;
+use OCA\NgBackup\Service\CronHealth;
+use OCP\Exceptions\AppConfigTypeConflictException;
+use OCP\IAppConfig;
+use OCP\IURLGenerator;
+use PHPUnit\Framework\TestCase;
+
+class CronHealthTest extends TestCase {
+	private const NOW = 1_800_000_000;
+	private const DOCS = 'https://docs.example.com/background-jobs';
+
+	private function health(string $mode, ?int $lastCron, bool $lastCronIsInt = false): CronHealth {
+		$config = $this->createMock(IAppConfig::class);
+		$config->method('getValueString')->willReturnCallback(function (string $app, string $key, string $default) use ($mode, $lastCron, $lastCronIsInt): string {
+			if ($key === 'backgroundjobs_mode') {
+				return $mode;
+			}
+			if ($lastCronIsInt) {
+				throw new AppConfigTypeConflictException('stored as int');
+			}
+			return $lastCron === null ? $default : (string)$lastCron;
+		});
+		$config->method('getValueInt')->willReturn($lastCron ?? 0);
+		$urls = $this->createMock(IURLGenerator::class);
+		$urls->method('linkToDocs')->with('admin-background-jobs')->willReturn(self::DOCS);
+		return new CronHealth($config, $urls);
+	}
+
+	public function testSystemCronThatRanRecentlyIsFine(): void {
+		$check = $this->health('cron', self::NOW - 120)->check(self::NOW);
+		$this->assertTrue($check['ok']);
+		$this->assertFalse($check['stopped']);
+		$this->assertNull(CronHealth::describe($check, self::NOW));
+	}
+
+	public function testAjaxAndWebcronAreNotEnough(): void {
+		foreach (['ajax' => 'AJAX', 'webcron' => 'Webcron'] as $mode => $name) {
+			$check = $this->health($mode, self::NOW - 60)->check(self::NOW);
+			$this->assertFalse($check['ok'], $mode);
+			$this->assertFalse($check['stopped'], $mode);
+			$this->assertStringContainsString("they are set to \"$name\" and last ran 1 min ago.", (string)CronHealth::describe($check, self::NOW));
+			$this->assertStringContainsString(self::DOCS, (string)CronHealth::describe($check, self::NOW));
+		}
+	}
+
+	public function testSystemCronThatStoppedRunning(): void {
+		$check = $this->health('cron', self::NOW - 2 * 3600)->check(self::NOW);
+		$this->assertFalse($check['ok']);
+		$this->assertTrue($check['stopped']);
+		$this->assertStringContainsString('has not run for over an hour (last ran 2 h ago)', (string)CronHealth::describe($check, self::NOW));
+	}
+
+	public function testDefaultsToAjaxThatNeverRan(): void {
+		$check = $this->health('ajax', null)->check(self::NOW);
+		$this->assertSame(0, $check['lastRun']);
+		$this->assertStringContainsString('and never ran.', (string)CronHealth::describe($check, self::NOW));
+	}
+
+	public function testReadsALastRunStoredAsAnInteger(): void {
+		$check = $this->health('cron', self::NOW - 60, true)->check(self::NOW);
+		$this->assertSame(self::NOW - 60, $check['lastRun']);
+		$this->assertTrue($check['ok']);
+	}
+
+	public function testARunningBackupWaitsForBackgroundJobsWhenItDidNotMoveForAWhile(): void {
+		$run = new Run();
+		$run->setStatus(Run::RUNNING);
+		$run->setUpdatedAt(self::NOW - 12 * 60);
+		$this->assertNull(CronHealth::waitingFor($run, self::NOW));
+		$run->setUpdatedAt(self::NOW - CronHealth::WAITING_AFTER);
+		$this->assertSame(CronHealth::WAITING_AFTER, CronHealth::waitingFor($run, self::NOW));
+		$run->setStatus(Run::DONE);
+		$this->assertNull(CronHealth::waitingFor($run, self::NOW));
+	}
+
+	public function testAgo(): void {
+		$this->assertSame('1 min ago', CronHealth::ago(10));
+		$this->assertSame('12 min ago', CronHealth::ago(12 * 60));
+		$this->assertSame('3 h ago', CronHealth::ago(3 * 3600 + 5));
+		$this->assertSame('4 days ago', CronHealth::ago(4 * 86400));
+	}
+}
@@ -2,13 +2,18 @@
 # Disaster recovery: occ backup:key:import + backup:target:add + backup:restore:full on a
 # SECOND, fresh Nextcloud instance that never saw the first one, sharing only the backup
 # location (a local directory, bind-mounted into both containers) -- see PLAN.md 3.7.
-# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|to-pgsql|to-mysql|to-sqlite]
+# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|salvage|salvage-files|to-pgsql|to-mysql|to-sqlite]
 #   e.g. full-restore-roundtrip.sh 35-apache pgsql
+# With "salvage-files" one pack is damaged and another is gone from the location. The restore must
+# stop and point at --salvage; run again as the guide says, it restores the database and every file
+# that can be read, and lists the rest (exit 2). NGB_FRT_DATA_MB sets the amount of data (default 200).
 # With "sftp" the location is an SFTP server instead of a shared directory, and the origin has a
 # file spread over several packs: the restore reads them ahead on connections of its own.
 # With "to-<db>" the restore instance runs that database whatever the origin runs (a MySQL or
 # SQLite backup records no ID counters; after the restore new rows must not collide with restored
 # ones, whatever the database on either side).
+# With "salvage" the snapshot's database dump is damaged on the location: the restore must stop
+# without changing anything, and with --salvage put the files in a folder of their own.
 # With "legacy" the origin runs NG Backup 1.0.0-rc.1 (0.9.7), whose backups do not record the
 # structure of their tables: the restore must then leave the tables of the disabled app out
 # with --skip-missing-tables, as for a real backup made before 1.0.0-rc.2.
@@ -34,7 +39,7 @@ cleanup() {
 	docker network rm "$NET" >/dev/null 2>&1 || true
 	docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
 	rmdir "$SHARED" 2>/dev/null || true
-	rm -f /tmp/ngb-frt-kit.json /tmp/ngb-frt-pass
+	rm -f /tmp/ngb-frt-kit.json /tmp/ngb-frt-pass /tmp/ngb-frt-sums
 	[ -n "${LEGACY_DIR:-}" ] && rm -rf "$LEGACY_DIR"
 	return 0
 }
@@ -103,6 +108,17 @@ if [ "$MODE" = sftp ]; then
 	docker exec ngb-frt-origin sh -c "head -c 90000000 /dev/urandom > /var/www/html/data/frtest/files/Docs/large.bin && chown www-data:www-data /var/www/html/data/frtest/files/Docs/large.bin"
 	LARGE_SUM=$(docker exec ngb-frt-origin sha256sum /var/www/html/data/frtest/files/Docs/large.bin | cut -d' ' -f1)
 fi
+if [ "$MODE" = salvage-files ]; then
+	# Half in large files that fill packs of their own, half in files of 1 MiB; incompressible
+	DATA_MB=${NGB_FRT_DATA_MB:-200}
+	docker exec -e DATA_MB="$DATA_MB" ngb-frt-origin sh -c '
+		cd /var/www/html/data/frtest/files && mkdir -p Large Small/a Small/b
+		i=0; while [ $((i * 50)) -lt $((DATA_MB / 2)) ]; do head -c 52428800 /dev/urandom > Large/f$i.bin; i=$((i + 1)); done
+		i=0; while [ $i -lt $((DATA_MB / 2)) ]; do head -c 1048576 /dev/urandom > Small/$([ $((i % 2)) = 0 ] && echo a || echo b)/f$i.bin; i=$((i + 1)); done
+		chown -R www-data:www-data . && find . -type f -print0 | xargs -0 sha256sum > /tmp/frt-sums'
+	docker cp ngb-frt-origin:/tmp/frt-sums /tmp/ngb-frt-sums
+	ok "origin has $DATA_MB MiB of test data, $(wc -l < /tmp/ngb-frt-sums) files"
+fi
 run_on ngb-frt-origin files:scan frtest >/dev/null

 # SQL on an instance's own database (pgsql/mysql only)
@@ -135,6 +151,24 @@ RUN_OUT=$(run_on ngb-frt-origin backup:run offsite -l disaster-test)
 SNAP=$(printf '%s\n' "$RUN_OUT" | head -1 | awk -F'snapshot ' '{print $2}' | tr -d ',')
 [ -n "$SNAP" ] || fail "no snapshot id from backup:run"
 ok "test data backed up, snapshot $SNAP"
+if [ "$MODE" = salvage ]; then
+	# The database dump of this snapshot damaged on the location: a byte flipped in its manifest
+	docker exec -u www-data ngb-frt-origin php -r '$p = "/mnt/shared/repo/db/" . $argv[1]; $d = file_get_contents($p); $i = intdiv(strlen($d), 2); $d[$i] = ~$d[$i]; file_put_contents($p, $d);' "$SNAP" \
+		|| fail "damage the database dump"
+	ok "database dump of the snapshot damaged on the location"
+fi
+if [ "$MODE" = salvage-files ]; then
+	# The two largest packs hold user files for sure: bytes flipped in one, the other gone
+	docker exec -u www-data ngb-frt-origin php -r '
+		$packs = glob("/mnt/shared/repo/packs/*/*");
+		usort($packs, fn ($a, $b) => filesize($b) <=> filesize($a));
+		count($packs) >= 3 or exit(1);
+		$h = fopen($packs[0], "r+b");
+		for ($o = 4096; $o < filesize($packs[0]); $o += 1048576) { fseek($h, $o); $b = fread($h, 16); fseek($h, $o); fwrite($h, ~$b); }
+		fclose($h);
+		unlink($packs[1]);' || fail "damage the packs"
+	ok "one pack damaged and one pack gone on the location"
+fi

 docker cp ngb-frt-origin:/tmp/kit.json /tmp/ngb-frt-kit.json
 docker exec ngb-frt-origin sh -c "cat /tmp/pass" > /tmp/ngb-frt-pass
@@ -164,6 +198,82 @@ printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
 }
 ok "recovery kit imported, existing repository adopted, snapshot visible"

+if [ "$MODE" = salvage ]; then
+	set +e
+	OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" --no-interaction 2>&1); CODE=$?
+	set -e
+	[ "$CODE" = 1 ] || { printf '%s\n' "$OUT" | tail -20; fail "a restore with a damaged database dump did not stop (exit $CODE)"; }
+	printf '%s\n' "$OUT" | grep -- "--salvage" >/dev/null || { printf '%s\n' "$OUT" | tail -20; fail "the stopped restore does not point at --salvage"; }
+	run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode stayed on after a restore that changed nothing"
+	ok "damaged database dump: the restore stops before changing anything and points at --salvage"
+	set +e
+	OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" --no-interaction --salvage 2>&1); CODE=$?
+	set -e
+	[ "$CODE" = 2 ] || { printf '%s\n' "$OUT" | tail -30; fail "restore:full --salvage exited with $CODE, not 2"; }
+	SALVAGE=/var/www/html/data/ngb-salvage-${SNAP:0:12}
+	printf '%s\n' "$OUT" | grep "$SALVAGE" >/dev/null || { printf '%s\n' "$OUT" | tail -30; fail "the output does not name the salvage folder"; }
+	[ "$(docker exec ngb-frt-restore cat "$SALVAGE/data/frtest/files/Docs/note.txt")" = "hello disaster recovery" ] || fail "the user's file is not in the salvage folder"
+	docker exec ngb-frt-restore test -f "$SALVAGE/config/config.php" || fail "config.php is not in the salvage folder"
+	run_on ngb-frt-restore user:list | grep frtest >/dev/null && fail "the fresh database was changed (the backed-up user is there)"
+	run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode is still on after the salvage"
+	run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after the salvage"
+	ok "with --salvage: the fresh database untouched, data and config of the backup in $SALVAGE (exit 2)"
+	echo "FULL RESTORE ROUNDTRIP PASS ($DB, salvage)"
+	exit 0
+fi
+
+if [ "$MODE" = salvage-files ]; then
+	set +e
+	OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" --no-interaction 2>&1); CODE=$?
+	set -e
+	[ "$CODE" = 1 ] || { printf '%s\n' "$OUT" | tail -20; fail "a restore with damaged data did not stop (exit $CODE)"; }
+	printf '%s\n' "$OUT" | grep -- "--salvage" >/dev/null || { printf '%s\n' "$OUT" | tail -20; fail "the stopped restore does not point at --salvage"; }
+	run_on ngb-frt-restore status 2>&1 | grep "maintenance: true" >/dev/null || fail "maintenance mode is off after a restore that stopped halfway"
+	ok "damaged data: the restore stops, keeps maintenance mode on and points at --salvage"
+
+	# As the guide and the message say: maintenance mode off and again, now with --salvage
+	run_on ngb-frt-restore maintenance:mode --off >/dev/null || fail "maintenance mode off"
+	set +e
+	OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" --no-interaction --salvage --force 2>&1); CODE=$?
+	set -e
+	[ "$CODE" = 2 ] || { printf '%s\n' "$OUT" | tail -30; fail "restore:full --salvage exited with $CODE, not 2"; }
+	printf '%s\n' "$OUT" | grep "could not be restored" >/dev/null || { printf '%s\n' "$OUT" | tail -30; fail "the damaged files are not listed"; }
+	printf '%s\n' "$OUT" | grep "ngb-salvage-" >/dev/null && { printf '%s\n' "$OUT" | tail -30; fail "the database was given up, only files are damaged"; }
+	docker cp /tmp/ngb-frt-sums ngb-frt-restore:/tmp/frt-sums
+	printf '%s\n' "$OUT" | docker exec -i ngb-frt-restore sh -c 'cat > /tmp/frt-out'
+	# Every file: listed and not there, or there and bit-identical
+	CHECK=$(docker exec ngb-frt-restore php -r '
+		$root = "/var/www/html/data/frtest/files/"; $listed = []; $reasons = [];
+		foreach (file("/tmp/frt-out", FILE_IGNORE_NEW_LINES) as $l) {
+			if (preg_match("#^  (?:data/)?frtest/files/(.+?): (.*)$#", $l, $m)) { $listed[$m[1]] = true; $reasons[] = $m[2]; }
+		}
+		$same = 0; $gone = 0; $bad = [];
+		foreach (file("/tmp/frt-sums", FILE_IGNORE_NEW_LINES) as $l) {
+			[$sum, $p] = preg_split("/\s+/", $l, 2); $p = substr($p, 2);
+			if (isset($listed[$p])) { file_exists($root . $p) ? $bad[] = "listed but there: $p" : $gone++; continue; }
+			if (!file_exists($root . $p)) { $bad[] = "not listed and not there: $p"; continue; }
+			hash_file("sha256", $root . $p) === $sum ? $same++ : $bad[] = "different: $p";
+		}
+		$temps = glob($root . "*/*.ngb-restore-*") ?: [];
+		foreach ($temps as $t) { $bad[] = "temporary left: $t"; }
+		echo "same=$same gone=$gone bad=", count($bad), "\n", implode("\n", array_slice($bad, 0, 10)), "\n";
+		echo "reasons: ", implode(" | ", array_unique(array_map(fn ($r) => substr(preg_replace("#[0-9a-f]{16,}#", "<id>", $r), 0, 70), $reasons))), "\n";')
+	printf '%s\n' "$CHECK"
+	printf '%s\n' "$CHECK" | grep -q " bad=0" || fail "files after the salvage are not as listed"
+	printf '%s\n' "$CHECK" | grep -qE "same=[1-9]" || fail "nothing was restored"
+	printf '%s\n' "$CHECK" | grep -qE "gone=[1-9]" || fail "no user file was hit by the damage"
+	ok "with --salvage: every file listed and absent, or there and identical"
+	run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode is still on after the salvage"
+	run_on ngb-frt-restore user:list | grep frtest >/dev/null || fail "the backed-up user is missing: the database was not restored"
+	[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt 2>/dev/null)" = "hello disaster recovery" ] \
+		|| printf '%s\n' "$OUT" | grep "Docs/note.txt" >/dev/null || fail "note.txt is neither restored nor listed"
+	run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after the salvage"
+	run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after the salvage"
+	ok "the database and config are the old server's, the instance works"
+	echo "FULL RESTORE ROUNDTRIP PASS ($DB, salvage-files)"
+	exit 0
+fi
+
 RESTORE_OPTS=(--no-interaction --install-apps)
 if [ "$LEGACY" = legacy ] && [ "$DB" != sqlite ]; then
 	# Without --skip-missing-tables an older backup must be refused, naming the app's tables
@@ -25,7 +25,8 @@ ok "location added"

 # Test data: a user with files.
 $X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
-$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin"
+# large.bin (40 MB) fills more than one pack: a restore to a directory reads ahead over several ranges
+$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin; head -c 40000000 /dev/urandom > $DATA/$U/files/large.bin"
 occ files:scan "$U" >/dev/null
 SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")

@@ -56,10 +57,47 @@ occ backup:restore ci "$SNAP" "data/$U/files/Docs/f2.bin" -d /tmp/ngb-rt-one | g
 $X sh -c "cmp -s $ONE/f2.bin $DATA/$U/files/Docs/f2.bin" || fail "the file restored to a directory differs"
 $X sh -c "ls -A $ONE | grep -q ngb-restore" && fail "the temporary of a stopped restore was left"
 ok "restore of one file to a directory; the temporary of a stopped restore removed"
+
+# All of the user's files to a directory: over S3 and SFTP the packs are read ahead
+occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-dir >/dev/null || fail "restore of a folder to a directory"
+SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
+[ "$SUM_BEFORE" = "$SUM_DIR" ] || fail "the files restored to a directory differ"
+$X sh -c "ls -A /tmp/ngb-rt-dir | grep -q ngb-restore-spool" && fail "the read-ahead spool was left in the directory"
+$X sh -c "rm -rf /tmp/ngb-rt-dir"
+ok "restore of all the user's files to a directory: identical, no spool left"
 # not grep -q: it stops reading at the first match, and occ, still writing, then fails (pipefail)
 occ backup:status | grep "started (" >/dev/null || fail "backup:status does not name the time zone"
 ok "backup:status names the time zone of its times"

+# Without a system cron (#13): backup:status warns, and the open admin page moves a backup started
+# with "Back up now" forward through /api/tick. Nothing runs cron in this container, so only the
+# ticks below can finish this backup.
+occ background:ajax >/dev/null
+occ backup:status | grep "do not run on a system cron" >/dev/null || fail "backup:status gives no warning without system cron"
+ADM=ngbadmin-$RANDOM; ADM_PW="Tick-admin-pw-$RANDOM-2026"
+$X sh -c "OC_PASS='$ADM_PW' php occ user:add --password-from-env --group=admin $ADM >/dev/null"
+api() { $X curl -sf -u "$ADM:$ADM_PW" -H 'OCS-APIRequest: true' -H 'Content-Type: application/json' "$@"; }
+API=http://localhost/index.php/apps/ng_backup/api
+STATUS=$(api "$API/status") || fail "status API"
+echo "$STATUS" | php -r '$s = json_decode(stream_get_contents(STDIN), true); exit(!$s["cron"]["ok"] && $s["cron"]["mode"] === "ajax" && $s["cron"]["docs"] !== "" ? 0 : 1);' || fail "status API does not report the missing cron"
+TID=$(echo "$STATUS" | php -r 'echo json_decode(stream_get_contents(STDIN), true)["targets"][0]["id"];')
+RUN=$(api -X POST -d "{\"targetId\": $TID, \"label\": \"page\"}" "$API/backups" | php -r 'echo json_decode(stream_get_contents(STDIN), true)["id"];') || fail "start a backup through the API"
+for i in $(seq 1 60); do
+  api -X POST "$API/tick" >/dev/null || fail "tick"
+  RS=$(api "$API/status" | php -r '$s = json_decode(stream_get_contents(STDIN), true); foreach ($s["runs"] as $r) { if ($r["id"] == '"$RUN"') { echo $r["status"]; } }')
+  [ "$RS" = running ] || break
+done
+[ "$RS" = done ] || fail "the backup started through the API ended as '$RS' after $i ticks"
+occ backup:list ci | grep " page " >/dev/null || fail "the snapshot of the ticked backup is not listed"
+ok "without system cron: warning in backup:status and the API, backup finished by $i ticks of the open page"
+occ background:cron >/dev/null
+$X php -f /var/www/html/cron.php >/dev/null 2>&1 || true
+occ backup:status | grep "system cron" >/dev/null && fail "backup:status still warns after cron ran"
+sleep 8 # APCu app config cache of the web server
+api "$API/status" | php -r 'exit(json_decode(stream_get_contents(STDIN), true)["cron"]["ok"] ? 0 : 1);' || fail "the status API still reports a missing cron after cron ran"
+ok "with a system cron that ran: no warning in backup:status or the API"
+occ user:delete "$ADM" >/dev/null
+
 # Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
 occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify"
 occ backup:verify ci "$SNAP" --deep | grep "^OK$" >/dev/null || fail "deep verify"
@@ -71,13 +109,39 @@ ok "database dump and restore"

 # Retention, trash and catalog.
 occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null
-occ backup:prune ci | grep "forgot 1" >/dev/null || fail "prune"
+occ backup:prune ci | grep "forgot 2" >/dev/null || fail "prune" # first and second; "page" (above) is the last
 occ backup:trash ci | grep " first " >/dev/null || fail "trash"
 TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
 occ backup:trash:restore ci "$TID" >/dev/null || fail "trash restore"
 [ "$(occ backup:list ci | grep -c ' ci ')" = "2" ] || fail "snapshot not back from the trash"
 ok "retention, trash and restore from trash"

+# Damaged data on the location (last: the location is not used after this). A byte flipped every
+# MiB of every pack hits big.bin (6 MB) for sure; a small file mostly survives. Without --salvage
+# the restore stops and points at --salvage; with it, the rest is restored and the damage listed.
+if [[ "$TARGET_ARGS" == local* ]]; then
+  # "second" went to the trash above: the backup of the page is still there
+  SNAP=$(occ backup:list ci | awk -F'|' '!f && / page /{gsub(/ /,"",$3); print $3; f=1}')
+  [ -n "$SNAP" ] || fail "the snapshot of the ticked backup is not listed"
+  $X php -r 'foreach (glob("/tmp/ngb-rt-target/repo/packs/*/*") as $p) { $h = fopen($p, "r+b"); for ($o = 4096; $o < filesize($p); $o += 1048576) { fseek($h, $o); $b = fread($h, 1); fseek($h, $o); fwrite($h, ~$b); } fclose($h); }'
+  set +e
+  OUT=$(occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-salvage 2>&1); CODE=$?
+  set -e
+  [ "$CODE" = 1 ] || { echo "$OUT"; fail "a restore of damaged data did not stop (exit $CODE)"; }
+  echo "$OUT" | grep -- "--salvage" >/dev/null || { echo "$OUT"; fail "the stopped restore does not point at --salvage"; }
+  set +e
+  OUT=$(occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-salvage --salvage 2>&1); CODE=$?
+  set -e
+  [ "$CODE" = 2 ] || { echo "$OUT"; fail "restore --salvage of damaged data exited with $CODE, not 2"; }
+  echo "$OUT" | grep "data/$U/files/big.bin: " >/dev/null || { echo "$OUT"; fail "the damaged big.bin is not listed"; }
+  N=$(echo "$OUT" | sed -n 's/^Restored \([0-9]*\) files.*/\1/p')
+  [ "${N:-0}" -ge 1 ] || { echo "$OUT"; fail "restore --salvage restored nothing"; }
+  $X sh -c "test ! -e /tmp/ngb-rt-salvage/data/$U/files/big.bin" || fail "the damaged big.bin was written anyway"
+  $X sh -c "find /tmp/ngb-rt-salvage -name '*.ngb-restore-*' | grep -q ." && fail "a temporary of a damaged file was left"
+  ok "damaged data: the restore stops and points at --salvage; with --salvage $N files restored and the damaged ones listed (exit 2)"
+  $X sh -c "rm -rf /tmp/ngb-rt-salvage"
+fi
+
 # No errors caused by loading files_external while it is disabled (its tables may not exist).
 if $X sh -c "grep -q 'external_mounts' $DATA/nextcloud.log 2>/dev/null"; then fail "files_external table errors in the log"; fi
 ok "no files_external errors in the log"