Phase 0: logical DB dump/restore without external tools (spike C)

DbDumper reads every prefixed table through the Nextcloud connection in one REPEATABLE READ (read-only) transaction, with keyset pagination on the primary key, and streams each table as JSON lines into its own blob stream (BlobWriter), so unchanged tables are deduplicated. Restore replaces table contents in one transaction and resets PostgreSQL sequences. Spike C (NC34, PostgreSQL, standard image without pg_dump/pgsql ext): 300k-row table with bytea: dump 6.3s / 50 MiB peak, 94 MiB JSON -> 20 MiB stored; unchanged second dump stores nothing; restore makes every table checksum-identical and all 110 sequences continue above the maximum.
2026-10-01 18:26:27 +02:00
parent f30074c4ae
commit 74de572160
4 changed files with 443 additions and 0 deletions
@@ -0,0 +1,49 @@
+<?php
+
+declare(strict_types=1);
+
+// SPDX-FileCopyrightText: 2026 Dillard Blom
+// SPDX-License-Identifier: AGPL-3.0-or-later
+
+namespace OCA\NgBackup\Repository;
+
+/**
+ * Write side of a blob stream: data written here is cut into blobs as it arrives and handed
+ * to the repository right away. Nothing is buffered beyond one blob, so a database dump or any
+ * other generated stream never needs a temporary file.
+ */
+final class BlobWriter {
+	private string $buffer = '';
+	/** @var list<string> */
+	private array $blobs = [];
+	private int $bytes = 0;
+
+	/** @param \Closure(string): string $store stores one blob, returns its id */
+	public function __construct(
+		private \Closure $store,
+		private int $blobSize = Repository::BLOB_SIZE,
+	) {
+	}
+
+	public function write(string $data): void {
+		$this->buffer .= $data;
+		$this->bytes += strlen($data);
+		while (strlen($this->buffer) >= $this->blobSize) {
+			$this->blobs[] = ($this->store)(substr($this->buffer, 0, $this->blobSize));
+			$this->buffer = substr($this->buffer, $this->blobSize);
+		}
+	}
+
+	/** @return list<string> blob ids, in order */
+	public function finish(): array {
+		if ($this->buffer !== '') {
+			$this->blobs[] = ($this->store)($this->buffer);
+			$this->buffer = '';
+		}
+		return $this->blobs;
+	}
+
+	public function bytes(): int {
+		return $this->bytes;
+	}
+}