181 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dillard	a8355df12e	Full restore: tables the restore creates get this installation's charset and collation (utf8mb4 on MySQL/MariaDB); test with an emoji	2026-10-22 03:36:46 +02:00
Dillard Blom	f0537f0f2c	Merge pull request #18 from dillardblom/rc5 ... Release 1.0.0-rc.5 v1.0.0-rc.5	2026-10-22 02:28:05 +02:00
dillard	fd0f786f5f	Full restore: a run again after an interrupted restore keeps maintenance mode on when its check refuses; retry text about apps	2026-10-22 01:59:01 +02:00
dillard	2eabdf44d5	README and store description: background jobs must run on system cron, with how to set it up; rc5 date	2026-10-22 01:03:52 +02:00
dillard	61e2a28b67	Full restore: maintenance mode on before the checks (off again when they refuse); the printed retry command is complete	2026-10-22 00:53:08 +02:00
dillard	7ed53ad0eb	Full restore: after an error maintenance mode stays on, with the one line to switch it off and run again; maintenance that was on before stays on	2026-10-22 00:36:58 +02:00
dillard	bdcb4f0cae	Full restore runs in maintenance mode, switched off again at the end (also after an error)	2026-10-22 00:31:34 +02:00
dillard	2616b6e543	Init guard also counts snapshots in the trash and the catalog as backup data	2026-10-21 22:42:43 +02:00
dillard	9bc4e4a4ab	Never create a repository where backup data lies but no config could be read	2026-10-21 22:25:48 +02:00
dillard	827e46381e	Log a snapshot that cannot be adopted when a location is added; full-restore test prints what target:add and backup:list said when the snapshot is missing	2026-10-21 22:08:14 +02:00
dillard	8eb6c4c033	Index: check known index files spread over the list	2026-10-21 20:56:45 +02:00
dillard	d623416306	Index: an empty listing is taken only when known index files are gone too	2026-10-21 20:49:32 +02:00
dillard	95184dadc7	Index: an empty listing is checked (config present, listed once more) before it is taken, so a repository another server emptied is accepted and a hiccup recovers	2026-10-21 19:51:46 +02:00
dillard	9b6bd7d2ab	Release 1.0.0-rc.5 (app version 0.9.11)	2026-10-21 13:10:03 +02:00
dillard	2fcd38f478	Full-restore test: Mail and Deck checks only when the origin had them (not on a SQLite origin)	2026-10-21 13:09:54 +02:00
dillard	177361acae	Index load keeps the index when it refuses an empty listing; counters of unchanged tables are repaired too, never moved backwards	2026-10-21 09:08:50 +02:00
dillard	85ea0acc50	Full-restore test: restore onto another database (to-pgsql, to-mysql, to-sqlite), every crossing in CI	2026-10-20 23:10:25 +02:00
dillard	ac57e31c93	Full restore onto PostgreSQL: counters of restored tables go after their data when the backup records none (MySQL, SQLite); test MariaDB onto PostgreSQL	2026-10-20 23:04:53 +02:00
dillard	e461c48056	CI: back to Nextcloud 35 for the full-restore test (the failures were a slow app store, not the version)	2026-10-19 22:18:33 +02:00
dillard	d504067cd8	Index: an empty listing is an error in checks and cleanups too; refresh() fetches only index files it has not loaded; full-restore CI on Nextcloud 35.0.0	2026-10-19 22:13:26 +02:00
dillard	c37b26ea38	Full restore: an empty listing never empties the index; read a failed pack once more; recognise and replace SFTP connections a Storage Box dropped quietly; version 0.9.11	2026-10-19 21:53:05 +02:00
Dillard Blom	0accd5a3f2	Merge pull request #16 from dillardblom/rc4 ... Release 1.0.0-rc.4 v1.0.0-rc.4	2026-10-19 11:04:15 +02:00
dillard	583592f69b	Release 1.0.0-rc.4 (app version 0.9.10)	2026-10-19 10:34:01 +02:00
dillard	a384ab41bb	App store description: Ensembia line	2026-10-19 10:31:21 +02:00
dillard	9b85221e50	Read ahead: a plan that cannot be made leaves the restore to the usual reads; only a whole buffered SFTP packet counts as an answer waiting	2026-10-19 08:54:13 +02:00
dillard	c95f552cc6	Read ahead over SFTP: one connection with 32 requests in flight by default, more with restore_sftp_connections; a Storage Box gets slower with parallel readers	2026-10-19 07:46:12 +02:00
dillard	6121f684e9	Full restore: say to run occ upgrade before rerunning a restore that stopped after the database; remove empty spool folders of a stopped restore	2026-10-19 05:34:54 +02:00
dillard	43a71684f9	Read ahead over SFTP: take answers phpseclib has read already instead of waiting up to 10 s on the socket	2026-10-19 05:28:26 +02:00
dillard	88b719879d	Release candidate 4 (app version 0.9.10): full restore over SFTP reads ahead; SFTP leg in the full-restore test	2026-10-19 03:51:21 +02:00
dillard	1d7f7e6423	Read ahead: resolve paths before opening connections; a failing readAhead() leaves no spool folder	2026-10-19 01:52:58 +02:00
dillard	ccbbf01c54	Full restore over SFTP: read packs ahead on four connections at once ... A restore read one pack at a time over one SFTP connection, which over a long line is bound by the round trip and by what one TCP connection gets: about 2-3 MiB/s from a Hetzner Storage Box to Canada, with PHP mostly waiting. Repository::restore() now first lists the ranges of packs the restore needs, in order (ReadAheadPlan), and the location reads them ahead into unlinked files in a spool folder in the target (IReadsAhead). Over SFTP, SftpRangeFetcher does that on four connections of its own, 16 requests in flight each, taking the answers of whichever connection has one. A range that fails, a damaged copy, or a blob that moved since the plan is read the usual way, which reports what is wrong. Other locations restore as before. On a test line with 120 ms RTT and 0.2% loss, a 1.6 GiB restore went from 5.1 to 17.8 MiB/s, all files identical.	2026-10-19 01:40:49 +02:00
dillard	b789d17636	Screenshot URLs per release tag, so the app store fetches new images	2026-10-18 23:47:32 +02:00
dillard	8e5c8ccda4	Full restore: skip the ID counters of tables left out with --skip-missing-tables; test a backup made by rc.1; version 0.9.9	2026-10-18 23:44:00 +02:00
dillard	e0b79939f9	PLAN: 1.0.0 only reliability; 1.0.1 restore speed, cron warning, DR test with table structure	2026-10-18 23:19:18 +02:00
Dillard Blom	39b4fe359b	Merge pull request #15 from dillardblom/release-rc2 ... Release 1.0.0-rc.2 v1.0.0-rc.2	2026-10-18 23:16:32 +02:00
dillard	40e147f3a0	Release 1.0.0-rc.2 (app version 0.9.8)	2026-10-18 23:10:02 +02:00
Dillard Blom	7f6e5c15f8	Merge pull request #14 from dillardblom/rc2 ... 1.0.0-rc.2: disaster recovery fixes	2026-10-18 23:08:55 +02:00
dillard	11ac30635a	target:add: show progress and time left while reading the index of an existing repository	2026-10-18 22:14:45 +02:00
dillard	9e1378c6ab	PLAN: DR finding 16, restore during another server's cleanup	2026-10-18 21:57:17 +02:00
dillard	04e91359a0	Restore: follow blobs that another server's cleanup repacked during the restore	2026-10-18 21:56:58 +02:00
dillard	46eaceb928	DR guide step 9 from the test; PLAN: small DR test with rc2 passed	2026-10-18 08:51:00 +02:00
dillard	9eeaefd938	Refuse a full restore when the table prefix differs	2026-10-18 02:39:14 +02:00
dillard	4f6a2e5b70	Check MySQL foreign keys explicitly; keep index prefix lengths and foreign keys of created tables ... Switching FOREIGN_KEY_CHECKS back on does not check the rows loaded meanwhile, and DDL would commit halfway, so on MySQL every foreign key is checked with a query for rows that point at nothing, before the commit. The recorded table structure now keeps index options (the prefix lengths MySQL needs for long text columns) and foreign keys; tables the restore creates get their foreign keys after the commit. The full-restore test also runs on MariaDB.	2026-10-18 02:09:00 +02:00
dillard	7b735ea941	DR guide step 7 for rc2; changelog; version 0.9.8	2026-10-18 01:45:39 +02:00
dillard	d8ef50732f	Full restore creates missing tables and suspends foreign keys; --install-apps; clearer messages ... Backups now record each table's structure in the dump manifest. The full restore creates tables that do not exist on the new instance (left behind by apps that were removed or are switched off), lists all problems at once, and no longer stops on apps that are missing. Foreign keys are not checked while rows are loaded in name order, and checked again before the restore commits (PostgreSQL: dropped and added back in the transaction). --install-apps installs or enables the apps that were enabled at backup time; --skip-missing-tables leaves out tables of older backups whose structure is unknown. nextcloud-aio is never required. backup:target:add without a key points to backup:key:import, and says that connecting can take a while. Removing a location drops its cached index; a migration removes the cache of locations that are gone. The full-restore test now also runs on PostgreSQL with Mail (foreign keys) and a disabled app.	2026-10-18 01:45:01 +02:00
dillard	5af712acc3	README: BBR for backups over a long distance	2026-10-18 01:26:27 +02:00
dillard	01e16199f7	DR guide: use your own container name	2026-10-18 01:15:43 +02:00
dillard	0924da48ac	PLAN: upload speed over long distances; DR guide container name	2026-10-18 01:15:34 +02:00
dillard	88477a1a9f	PLAN: SFTP restore speed over long distances; foreign keys valid after the restore	2026-10-18 00:59:06 +02:00
dillard	f633de59b4	PLAN: foreign keys block the database restore (Mail)	2026-10-17 23:15:13 +02:00