dillard
3694184674
Version 1.0.1: changelog, README status and roadmap
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
2026-11-01 12:03:19 +01:00
dillard
a11e6c7254
1.0.1: label for "Back up now", retention explained in the trash, occ times in the schedule's zone, single-file restore to a directory, leftovers of a stopped restore removed
...
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
- Admin page: an optional label (at most 100 characters) for a backup started with "Back up now";
an (i) by "removed by retention policy" in the trash that explains what it means and until when
the restore point can be brought back. backup:trash says the same below its list.
- backup:status, backup:list, backup:trash and backup:browse show times in the zone of the daily
schedule and name it, instead of the server's UTC.
- backup:restore --to-directory of a file restored nothing: the path got a "/" appended.
- A restore removes "<name>.ngb-restore-<hex>" temporaries that a stopped restore left in the
folders it writes to.
- Roadmap: the cron warning (#13 ), reading ahead from S3/SMB and --salvage move to 1.0.2.
2026-10-30 19:32:53 +01:00
dillard
6f1a37f4f4
Changelog: release candidate headings carry their app version, so the app store no longer shows all of them under 1.0.0
2026-10-29 13:07:25 +01:00
dillard
5cde57bc1b
Changelog: 1.0.0 says what the app is instead of release bookkeeping; rc.6 drops its outdated outlook
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
2026-10-29 12:42:44 +01:00
dillard
3eec2e9554
Full restore on SQLite: take the write lock first, so another writer is waited for instead of "database is locked"
...
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
SQLite in WAL mode starts a transaction as a reader; a write from another
process between our first read and our first write makes our first write
fail at once, whatever the busy timeout. The SQLite full-restore test now
runs a second writer alongside the restore.
2026-10-29 03:06:05 +01:00
dillard
cdf0b70e25
Changelog: the admin page's single-line key field as the cause of #24 ; PLAN 1.0.1: #24 , database locked at the start of a full restore on SQLite
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
2026-10-29 02:40:03 +01:00
dillard
66615c2a88
Merge branch 'main' into polish
...
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
# Conflicts:
# CHANGELOG.md
2026-10-29 02:27:01 +01:00
dillard
a0c4abb60a
Version 1.0.0: same code as 1.0.0-rc.6; changelog, README status
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
2026-10-29 01:32:12 +01:00
dillard
4847ce293d
SFTP: restore the line breaks of an OpenSSH private key before use ( #24 )
...
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
Nextcloud 35 reads keys with phpseclib 3, which refuses an OpenSSH-format
key whose line breaks were lost ("Unable to read key"); phpseclib 2 on
Nextcloud 34 read it. The key is fixed when used, so stored locations work
again without re-entering it.
2026-10-28 01:41:58 +01:00
dillard
95e3dfcf6d
User restore by user id, backup:user:list ( #22 )
...
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
backup:user:restore <location> <uid> only said "is not a recorded user export": the argument
is the manifest path, which only backup:user:backup printed once. backup:user:list shows the
exports again; a bare user id works when that user has exactly one export, otherwise the error
lists them. The export is resolved before the replace confirmation.
Fixes #22
2026-10-27 00:10:59 +01:00
dillard
3a0f882a38
1.0.0-rc.6: changelog, SMB disaster recovery result in the README, 1.0.1 items (salvage mode, leftover restore files, single-file restore)
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
2026-10-24 00:40:51 +02:00
dillard
343389b98b
Changelog: the second read covers the database phase too
2026-10-22 19:48:22 +02:00
dillard
67cc350b8c
Daily backup time in the administrator's time zone ( #19 ); a time saved before is converted once so it shows as it ran
2026-10-22 16:38:54 +02:00
dillard
fd12c586b6
Roadmap: restore mode for hosted Nextcloud, read-ahead for S3 and SMB; changelog for rc6
2026-10-22 13:07:09 +02:00
dillard
2eabdf44d5
README and store description: background jobs must run on system cron, with how to set it up; rc5 date
2026-10-22 01:03:52 +02:00
dillard
7ed53ad0eb
Full restore: after an error maintenance mode stays on, with the one line to switch it off and run again; maintenance that was on before stays on
2026-10-22 00:36:58 +02:00
dillard
bdcb4f0cae
Full restore runs in maintenance mode, switched off again at the end (also after an error)
2026-10-22 00:31:34 +02:00
dillard
9bc4e4a4ab
Never create a repository where backup data lies but no config could be read
2026-10-21 22:25:48 +02:00
dillard
9b6bd7d2ab
Release 1.0.0-rc.5 (app version 0.9.11)
2026-10-21 13:10:03 +02:00
dillard
ac57e31c93
Full restore onto PostgreSQL: counters of restored tables go after their data when the backup records none (MySQL, SQLite); test MariaDB onto PostgreSQL
2026-10-20 23:04:53 +02:00
dillard
c37b26ea38
Full restore: an empty listing never empties the index; read a failed pack once more; recognise and replace SFTP connections a Storage Box dropped quietly; version 0.9.11
2026-10-19 21:53:05 +02:00
dillard
583592f69b
Release 1.0.0-rc.4 (app version 0.9.10)
2026-10-19 10:34:01 +02:00
dillard
c95f552cc6
Read ahead over SFTP: one connection with 32 requests in flight by default, more with restore_sftp_connections; a Storage Box gets slower with parallel readers
2026-10-19 07:46:12 +02:00
dillard
6121f684e9
Full restore: say to run occ upgrade before rerunning a restore that stopped after the database; remove empty spool folders of a stopped restore
2026-10-19 05:34:54 +02:00
dillard
88b719879d
Release candidate 4 (app version 0.9.10): full restore over SFTP reads ahead; SFTP leg in the full-restore test
2026-10-19 03:51:21 +02:00
dillard
b789d17636
Screenshot URLs per release tag, so the app store fetches new images
2026-10-18 23:47:32 +02:00
dillard
8e5c8ccda4
Full restore: skip the ID counters of tables left out with --skip-missing-tables; test a backup made by rc.1; version 0.9.9
2026-10-18 23:44:00 +02:00
dillard
40e147f3a0
Release 1.0.0-rc.2 (app version 0.9.8)
2026-10-18 23:10:02 +02:00
dillard
11ac30635a
target:add: show progress and time left while reading the index of an existing repository
2026-10-18 22:14:45 +02:00
dillard
04e91359a0
Restore: follow blobs that another server's cleanup repacked during the restore
2026-10-18 21:56:58 +02:00
dillard
9eeaefd938
Refuse a full restore when the table prefix differs
2026-10-18 02:39:14 +02:00
dillard
7b735ea941
DR guide step 7 for rc2; changelog; version 0.9.8
2026-10-18 01:45:39 +02:00
dillard
7f29d009f3
App store description for 1.0.0; developed and sponsored by Ensembia
2026-10-17 15:52:33 +02:00
dillard
76e8bb0245
Release 1.0.0-rc.1 (app version 0.9.7)
2026-10-17 13:07:41 +02:00
dillard
629b8dd315
Passphrases of at least 16 characters; explain that the location holds the key too
2026-10-16 00:07:39 +02:00
dillard
942e8ec003
Web interface: dates and times in the user's Nextcloud locale
2026-10-15 21:43:09 +02:00
dillard
144097d159
Run the backup job at every cron run; skip deflate on compressed data
2026-10-15 20:59:04 +02:00
dillard
5b7dd08e71
Ask for feedback on the admin page, with an experience report form
2026-10-15 10:32:21 +02:00
dillard
ca003c88ba
Fast S3 verify and snapshot commit; shorter runs list; durations with days
...
On S3, index files and the tree segments read back when a snapshot is
written are fetched 16 at a time through the storage's S3 client
(IBackend::getRanges(), getMany()). Blobs that follow each other in a pack
are read as one range on every location. The snapshot is written in a step
of its own and the run shows "commit" meanwhile. On a 1 TB production
repository the check went from about 4 hours to 3.5 minutes.
backup:status and the web interface list the runs of the last 7 days,
running ones and at least the newest 5; --all / "Show older runs" list
all. Durations of a day or more show the days.
2026-10-14 17:12:41 +02:00
dillard
c38cc47534
Release 0.9.6-beta.7
2026-10-14 13:14:46 +02:00
dillard
dfd3f38d8c
Backups to SMB no longer crawl; add occ backup:cancel
...
Each 30-second step opened the location again and read its list of
index files. Over SMB that cost a request per file, so on a large
repository opening took longer than the step: each step stored one
4 MiB pack and the list grew further. A first backup of nc01 to a
Storage Box over SMB managed 12 GiB in 35 hours.
occ backup:run and system cron now keep the repository open for the
whole run (reopened when another process took a step or the location
was edited), opening no longer counts towards the step's time, and
SMB folders are listed in one request.
occ backup:cancel stops a running backup: no new step starts and the
step in progress finishes first. A run of an earlier version, which
does not hold back its next step, is marked cancelled after two
minutes unless it is recording its snapshot.
2026-10-14 12:15:27 +02:00
dillard
92e693cfe6
Runs list: show the total size of each backup next to what it uploaded
2026-10-13 11:02:18 +02:00
dillard
025bbfbb62
Verify: list the pack folders only with more packs than folders
...
With few packs a request per pack is fewer requests than 256 listings. On
FTP every listing takes a data connection; the FTP server of the integration
tests has ten passive ports, and 256 listings in a row slowed it down until a
read failed ("Not found: snapshots/...").
2026-10-12 14:50:56 +02:00
dillard
8ed7455619
Release 0.9.5-beta.6
2026-10-12 14:27:39 +02:00
dillard
30a250bc11
Stale-backup notification: "No successful backup to … yet" instead of "since never"
2026-10-12 11:32:42 +02:00
dillard
9c06ab2841
SFTP: replace a dropped connection; write the index cache in one transaction
...
A Hetzner Storage Box drops an SFTP connection that carried a lot of data
and then stayed quiet for a few minutes; files_external keeps using the dead
client, on which every file looks missing. StorageBackend now checks the
connection after a minute without requests, and replaces a connection that
is dead after a failed request (once per request; put, move and delete are
not repeated). SftpReadStream closes a connection it left in an unknown state.
The verify cache refresh took more than 6 minutes on PostgreSQL for 27,547
packs, one autocommitted insert per pack: putMany()/replaceAll() write in one
transaction. After a verify the cache holds the index files as they are, so a
blob in two packs stays cached for both.
2026-10-12 01:58:54 +02:00
dillard
d659322e60
Merge branch 'fix/sftp-fresh-connection' into polish
2026-10-11 17:16:33 +02:00
dillard
df732f185a
Index: refresh the cache after a verify in one pass over the blobs, not one per pack
2026-10-11 14:31:57 +02:00
dillard
379b81db02
Verify: index files several at a time over SFTP, pack folders listed, progress stored and shown
2026-10-11 13:39:38 +02:00
dillard
828f3f142a
Merge branch 'fix/sftp-fresh-connection' into polish
2026-10-11 13:26:47 +02:00
dillard