- DbDumper::dumpStep(): tables are dumped until the step's deadline in one
REPEATABLE READ transaction and resumed at the last primary key in the
next step; the running table checksum is carried over. A dump that fits
in one step is fully consistent, otherwise the snapshot records
dbConsistent=false. BackupService uses it, so the database phase no
longer needs one long request (hosted Nextcloud, web cron)
- restore inserts 100 rows per statement and leaves tables whose checksum
already matches the dump untouched
- spike C: 30-step dump gives the same checksums as the one-shot dump;
restore rewrote only the 3 damaged tables of 147; 300k-row table
restored in 14.6 s instead of 46 s
- detect files changing during reading or between resumable steps and
re-read them (size/mtime/ctime/inode), spike F overwrites mid-run
- drop pending blob ids when a pack or index upload fails
- dump and restore PostgreSQL sequence state exactly, bound parameters,
validated catalog names; spike C checks all sequences exactly
- keep invalid UTF-8 byte-exact (tagged base64)
- refuse large tables without a primary key
- StorageBackend: check write/flush/close results, validate base path
- user_migration import/export: root paths, intermediate folders,
partial writes, read errors
- spikes: never touch an existing account, own fixtures, 512M note
DbDumper reads every prefixed table through the Nextcloud connection in
one REPEATABLE READ (read-only) transaction, with keyset pagination on
the primary key, and streams each table as JSON lines into its own blob
stream (BlobWriter), so unchanged tables are deduplicated. Restore
replaces table contents in one transaction and resets PostgreSQL
sequences.
Spike C (NC34, PostgreSQL, standard image without pg_dump/pgsql ext):
300k-row table with bytea: dump 6.3s / 50 MiB peak, 94 MiB JSON -> 20 MiB
stored; unchanged second dump stores nothing; restore makes every table
checksum-identical and all 110 sequences continue above the maximum.
dillard