Full restore on SQLite: take the write lock first, so another writer is waited for instead of "database is locked"

SQLite in WAL mode starts a transaction as a reader; a write from another
process between our first read and our first write makes our first write
fail at once, whatever the busy timeout. The SQLite full-restore test now
runs a second writer alongside the restore.

CHANGELOG.md

@@ -28,6 +28,10 @@ instead carried by the app description, the git tag, the release, and this file.
   pasted key lost its line breaks; Nextcloud 34 still read it, Nextcloud 35 reads keys with a
   stricter library. The line breaks are now restored before the key is used, so a stored location
   works again without re-entering it.
+- A full restore onto SQLite could stop at once with "database is locked" when another process
+  wrote to the database just as the restore began (a background job that was already running).
+  The restore now takes the write lock first and waits for the other process instead. The
+  full-restore test on SQLite runs with a second process writing alongside.
 
 ## [1.0.0] - 2026-10-29
 

PLAN.md

@@ -462,8 +462,12 @@ in README "Known limitations":
       bij #22 en 2026-10-29 op `main` na de merge van 1.0.0): de restore stopt meteen na het
       aanzetten van de onderhoudsmodus. Niet altijd, dus een tweede proces dat op hetzelfde moment
       de database gebruikt (cron, een webverzoek of de opstart van de container). Kan ook bij een
-      echte restore naar SQLite gebeuren. Uitzoeken wie de lock vasthoudt; dan wachten en opnieuw
+      echte restore naar SQLite gebeuren. Oorzaak (2026-10-29): SQLite (WAL) begint een transactie
-      proberen (busy timeout) of eerst de andere schrijvers stoppen.
+      als lezer; schrijft een ander proces tussen ons eerste lezen en ons eerste schrijven, dan
+      faalt dat schrijven direct (geen wachttijd helpt). In CI was dat de installer, die
+      "installed" zet en daarna nog seconden doorschrijft. Nu neemt de restore eerst het
+      schrijfslot en wacht hij op de ander; de SQLite-test draait met een tweede schrijver
+      ernaast — **klaar** (2026-10-29).
 - **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
   de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
   `to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie

lib/Db/DbDumper.php

@@ -169,7 +169,7 @@ final class DbDumper {
 		$foreignKeys = [];
 		$this->db->beginTransaction();
 		try {
-			$foreignKeys = $this->suspendForeignKeys();
+			$foreignKeys = $this->suspendForeignKeys(); // first: on SQLite it also takes the write lock
 			foreach ($manifest['tables'] as $table => $info) {
 				if (in_array($table, $report['missing'], true)) {
 					continue;
@@ -412,6 +412,12 @@ final class DbDumper {
 			return $checks;
 		}
 		if ($platform instanceof \Doctrine\DBAL\Platforms\SqlitePlatform) {
+			// SQLite (WAL) starts a transaction as a reader. When another process (a cron job, the
+			// installer still finishing) writes between our first read and our first write, that
+			// write fails at once with "database is locked": no waiting helps a reader whose view
+			// is out of date. Take the write lock first, so we wait for the other writer instead
+			// and it waits for us after that.
+			$this->db->executeStatement('DELETE FROM ' . $this->db->quoteIdentifier($this->prefix . 'appconfig') . ' WHERE 0 = 1');
 			$this->db->executeStatement('PRAGMA defer_foreign_keys = ON'); // checked at commit
 		}
 		return [];

tests/integration/full-restore-roundtrip.sh

@@ -173,6 +173,20 @@ if [ "$LEGACY" = legacy ] && [ "$DB" != sqlite ]; then
 	ok "older backup refused without --skip-missing-tables, missing tables named"
 	RESTORE_OPTS+=(--skip-missing-tables)
 fi
+if [ "$DB" = sqlite ]; then
+	# Another process writing while the restore starts (a cron job that was already running, or
+	# the installer still finishing after it says "installed"): the restore must wait for it, not
+	# stop with "database is locked". Short write transactions on a table of its own, until the
+	# restore is done (/tmp/frt-side-stop; Nextcloud's own commands after it are not under test).
+	docker exec -d -u www-data ngb-frt-restore php -r '
+		$p = new PDO("sqlite:/var/www/html/data/nextcloud.db"); $p->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
+		$p->exec("CREATE TABLE IF NOT EXISTS frt_side_writer (t REAL)");
+		for ($end = microtime(true) + 300; microtime(true) < $end && !file_exists("/tmp/frt-side-stop");) {
+			$p->beginTransaction(); $p->exec("INSERT INTO frt_side_writer VALUES (" . microtime(true) . ")"); usleep(300000); $p->commit(); usleep(20000);
+		}
+		touch("/tmp/frt-side-done");'
+	sleep 1
+fi
 sftp_logins() { docker logs ngb-frt-sftp 2>&1 | grep -c "Accepted password" || true; }
 [ "$MODE" = sftp ] && LOGINS_BEFORE=$(sftp_logins)
 OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" "${RESTORE_OPTS[@]}" 2>&1) || {
@@ -184,6 +198,13 @@ printf '%s\n' "$OUT" | grep -q "maintenance mode while the restore runs" || fail
 run_on ngb-frt-restore status 2>&1 | grep -q "maintenance: false" || fail "maintenance mode is still on after the restore"
 run_on ngb-frt-restore config:app:get ng_backup restore_full_started >/dev/null 2>&1 && fail "the restore left its 'started' marker behind"
 ok "backup:restore:full completed, in maintenance mode and out of it again"
+if [ "$DB" = sqlite ]; then
+	docker exec ngb-frt-restore touch /tmp/frt-side-stop
+	for i in $(seq 1 20); do docker exec ngb-frt-restore test -f /tmp/frt-side-done && break; sleep 0.5; done
+	WRITES=$(docker exec -u www-data ngb-frt-restore php -r 'echo (new PDO("sqlite:/var/www/html/data/nextcloud.db"))->query("SELECT COUNT(*) FROM frt_side_writer")->fetchColumn();')
+	[ "${WRITES:-0}" -gt 3 ] || fail "the other writer did not run alongside the restore ($WRITES writes)"
+	ok "the restore waited for another process writing to SQLite ($WRITES writes alongside)"
+fi
 
 # Everything below must work in a FRESH occ process (a new PHP process per docker exec), since
 # earlier bugs here only showed up after config.php's secret was rewritten mid-restore.