e58b86fc48
Off, once a day or every N hours from a time, each in its own time zone. Existing locations keep the time that held for all of them. A backup that has to wait shows as queued, and its restore point has the time it really began.
591 lines
38 KiB
Markdown
591 lines
38 KiB
Markdown
# Changelog
|
||
|
||
All notable changes to NG Backup are documented here. The format follows
|
||
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning follows the stages in
|
||
`PLAN.md` section 9. The `<version>` in `appinfo/info.xml` is plain SemVer-numeric on purpose:
|
||
the app store accepts a `-beta` suffix, but Nextcloud servers then only offer the release on the
|
||
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
||
instead carried by the app description, the git tag, the release, and this file.
|
||
|
||
Each heading starts with that `<version>`, once: the app store shows a release the section(s)
|
||
whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
|
||
shown under 1.0.0. The git tag of a release candidate goes after the date.
|
||
|
||
## [Unreleased]
|
||
|
||
### Added
|
||
- Pin a restore point, on the restore page or with `occ backup:pin`: the retention policy leaves it
|
||
alone, for example the one from before an upgrade. Unpinning notifies all administrators.
|
||
- A note on a restore point that you can change afterwards, on the restore page or with
|
||
`occ backup:note`. Pins and notes are stored encrypted on the location, so a new server sees them.
|
||
An older version of NG Backup can still read the location, but drops them when it writes to it.
|
||
- A schedule per location: off, once a day, or every N hours from a given time, on the page under
|
||
Backups or with `occ backup:target:schedule`. After the update every location keeps the time that
|
||
held for all of them; a new location starts with the schedule of the first one.
|
||
|
||
### Changed
|
||
- Locations are backed up one after the other: a backup that has to wait shows as queued. A restore
|
||
point has the time its backup really began, not the time it was queued.
|
||
- Choosing a time no longer starts a backup right away when that time has already passed today.
|
||
- A location without a schedule no longer gets the notification that it has had no recent backup.
|
||
- A file or folder that is left out because of its name now gets a notification of its own, in the
|
||
log and in `occ backup:run`. It says the item is missing from the backup, gives example paths and
|
||
asks to rename it; before, it was reported as unreadable, with advice about permissions.
|
||
|
||
## [1.0.3] - 2026-11-06
|
||
|
||
### Fixed
|
||
- Empty folders are backed up and restored (#27), in `occ backup:restore`, `occ backup:restore:full`
|
||
and the restore page, where they are shown too. They come back from the first backup made with
|
||
this version; older restore points do not contain them. An older version of NG Backup can still
|
||
read new restore points, and then only leaves the empty folders out.
|
||
- A full disk during a restore no longer looks like a damaged backup. The restore now stops with a
|
||
message about the disk, also with `--salvage`, and does not point at `--salvage` any more. The same
|
||
goes for other trouble writing to the restore directory, such as permissions.
|
||
- A file or folder whose name a snapshot cannot hold (a backslash, a control character, not UTF-8) is
|
||
left out and reported as unreadable. Before, a backslash or control character could make the whole
|
||
restore point unreadable, and a name that is not UTF-8 made the backup fail.
|
||
- A restore that goes on after a stop also removes the temporary files in folders where every file
|
||
is already in place, and checks that those folders are inside the restore directory before it
|
||
leaves their files alone.
|
||
- After a restore that was killed and then ran again, a backup no longer has to wait up to two hours
|
||
for the locks of the killed run.
|
||
- A restore only removes read-ahead folders of its own (`.ngb-restore-spool-` with 8 hex digits).
|
||
- A restore keeps its lock on the location during a long row of files that are already in place.
|
||
- A database restore that fails no longer leaves behind tables it created, which could stop the
|
||
installation of that app later. When only NG Backup's own settings cannot be put back after the
|
||
database was restored, `--salvage` no longer says the database is still the fresh one.
|
||
- A connection that breaks while opening a pack on the location (seen with SMB) is retried once and
|
||
is handled by `--salvage`, instead of stopping the restore with the error of the storage.
|
||
- Without a system cron, several running backups take turns while the page is open.
|
||
- The Bin tab shows how far it is while it reads the removed restore points, which can take a while.
|
||
It also shows when it cannot read a location, instead of an empty bin, and refreshes when a run
|
||
ends.
|
||
|
||
## [1.0.2] - 2026-11-06
|
||
|
||
### Added
|
||
- A warning on the admin page and in `occ backup:status` when background jobs do not run on a
|
||
system cron, or the cron stopped, with a link to how to set one up. Backups, also the scheduled
|
||
ones, need it (#13).
|
||
- Without a working system cron, a backup started with "Back up now" moves on while the NG Backup
|
||
page stays open (#13).
|
||
- `--salvage` for `occ backup:restore` and `occ backup:restore:full`: restore whatever can be
|
||
read when a backup is damaged. Files that cannot be read or decrypted are left out and listed,
|
||
and the command exits with code 2. When the database cannot be restored, `backup:restore:full`
|
||
leaves the fresh database as it is and puts the data directory and config of the backup in
|
||
`ngb-salvage-<snapshot>` in the data directory, to put back by hand.
|
||
- The trash of restore points has a tab of its own, "Bin", instead of being shown below the
|
||
backups.
|
||
|
||
### Changed
|
||
- A backup in its database phase shows how far it is, for example "running (db: 47/183 tables)",
|
||
and one that has not moved for 15 minutes says "waiting for background jobs" with when it last
|
||
took a step. Before, it only said "running (db)", which looked the same as being stuck (#13).
|
||
- Full restores from S3 read the packs ahead, eight ranges at a time, as restores over SFTP do
|
||
since 1.0.0-rc.4. Over SMB the packs are read in 8 MiB reads instead of 1 MiB: over a line with
|
||
a 130 ms round trip, reading was four times faster in a test (18 instead of 4.5 MiB/s).
|
||
- Over SMB the index files, which a new server reads in `backup:target:add`, take fewer round
|
||
trips: from Canada to a Hetzner Storage Box (130 ms), 0.66 instead of 1.05 s per file, so about
|
||
5 instead of 8 hours for the 27,000 index files of a 1 TB repository.
|
||
- `occ backup:restore:full` that runs again after it stopped (`--force`) goes on where it was: data
|
||
files already in place with the size and modification time of the backup are left, and only
|
||
the rest is read from the location. Before, a stopped restore of 1 TB started over.
|
||
|
||
### Fixed
|
||
- A full restore over SMB could hang at 100% CPU when the connection to the server dropped: a
|
||
failed read was counted as data. It is now read again over a new connection. Found in a
|
||
disaster recovery test of 1 TB from Canada to a Hetzner Storage Box, after 392 GiB.
|
||
- The read-ahead folder of a restore that was killed while it wrote is removed by the next run;
|
||
it kept one file of up to 8 MiB.
|
||
- `occ backup:target:add` only estimates the time left for reading the index once enough of it
|
||
is in. The first file also carries the time to connect, which made it say "about 642 min left"
|
||
for something done within a minute.
|
||
- A restore that stops because a file is gone from the location now points at `--salvage`, as it
|
||
already did for damaged data.
|
||
|
||
### Known issues
|
||
- Empty folders are not backed up or restored; to be fixed in 1.0.3 (#27).
|
||
- A full disk during a restore looks like a damaged backup ("Size mismatch"), and with `--salvage`
|
||
the restore goes on and lists the remaining files. The backup itself is fine: make room and run
|
||
the restore again. To be fixed in 1.0.3.
|
||
|
||
## [1.0.1] - 2026-11-01
|
||
|
||
Fixes for what was reported after 1.0.0 (#22, #24) and small improvements. Nothing changes in
|
||
your backups or settings.
|
||
|
||
### Added
|
||
- `occ backup:user:list [location] [user]` shows the user exports on a location, with the manifest
|
||
path `backup:user:restore` expects. Before, that path was only printed once by
|
||
`backup:user:backup` (#22).
|
||
- A "Docs" tab on the admin page, with links to the how-to, the disaster recovery guide and the
|
||
roadmap. It also shows before the backup key is set up.
|
||
- A label for a backup started with "Back up now" (optional; `occ backup:run -l` already had one).
|
||
It shows with the restore point.
|
||
- An explanation in the trash of restore points of what "removed by retention policy" means: the
|
||
policy keeps the newest restore points and the last one of each recent day, week and month, and
|
||
moves the others to the trash, where they can be brought back until they are deleted
|
||
permanently. `occ backup:trash` says the same below its list.
|
||
|
||
### Changed
|
||
- `occ backup:status`, `backup:list`, `backup:trash` and `backup:browse` show times in the time
|
||
zone of the daily schedule and name it, as the admin page shows them in the administrator's.
|
||
Before, they showed the server's UTC, which read as a wrong start time.
|
||
- The admin page no longer says NG Backup is in beta.
|
||
- `occ backup:user:restore` also takes a user id instead of the manifest path, when that user has
|
||
exactly one export on the location. With more, and for a manifest path that is not on the
|
||
location, the error lists that user's exports; for a user without exports it names the
|
||
expected form (#22). Before, `backup:user:restore <location> <uid>` only said "is not a
|
||
recorded user export for this location".
|
||
|
||
### Fixed
|
||
- An SFTP location with an RSA key in OpenSSH format ("BEGIN OPENSSH PRIVATE KEY") stopped
|
||
working after an upgrade to Nextcloud 35 with "Unable to read key", and such a key could not be
|
||
added there at all (#24). The "Private key" field on the admin page is a single line, so a
|
||
pasted key lost its line breaks; Nextcloud 34 still read it, Nextcloud 35 reads keys with a
|
||
stricter library. The line breaks are now restored before the key is used, so a stored location
|
||
works again without re-entering it.
|
||
- A full restore onto SQLite could stop at once with "database is locked" when another process
|
||
wrote to the database just as the restore began (a background job that was already running).
|
||
The restore now takes the write lock first and waits for the other process instead. The
|
||
full-restore test on SQLite runs with a second process writing alongside.
|
||
- `occ backup:restore ... --to-directory` of a single file restored nothing ("Restored 0
|
||
files"); only a folder worked. A path to a file now restores that file.
|
||
- A restore that was stopped (a crash, a timeout, a restart) left its half-written temporary file
|
||
(`<name>.ngb-restore-<id>`) next to the file it was restoring, and a `files:scan` then showed it
|
||
in Files. A restore now removes such leftovers in each folder it writes to.
|
||
- The app store showed the notes of every release candidate under 1.0.0, because their headings
|
||
started with "1.0.0-rc". Each heading now starts with the app version of that release.
|
||
|
||
## [1.0.0] - 2026-10-29
|
||
|
||
First stable release. NG Backup makes encrypted, incremental backups of your whole Nextcloud
|
||
(files, database and configuration) to S3, SFTP, WebDAV, SMB and other storage, and restores them
|
||
from the web interface or with occ. Disaster recovery has been tested on real installations; see
|
||
"Tested disaster recoveries" in the README, and read "Known limitations" before you rely on it.
|
||
|
||
The code is the same as 1.0.0-rc.6 (app version 0.9.12): updating from it changes nothing in your
|
||
backups or settings.
|
||
|
||
## [0.9.12] - 2026-10-24 (1.0.0-rc.6)
|
||
|
||
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
||
simulations and a user report.
|
||
|
||
### Fixed
|
||
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
||
Berlin (#19). The time is now saved with the administrator's time zone and shown with it. A time
|
||
saved before is converted once to the zone of the server or an administrator, so it shows as it
|
||
effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer) and nothing moves.
|
||
- A full restore over SMB stopped after 17 hours with "Decryption failed": a read over a long
|
||
line handed out wrong bytes (the backup itself was fine, the same files restored identically
|
||
afterwards). A blob that does not decrypt as first read, of a file or of the database, is now
|
||
read once more on a request of its own, and only a second failure stops the restore.
|
||
- A full restore onto MySQL/MariaDB stopped with "Incorrect string value" when it created a table
|
||
(of an app that is not installed here) whose rows hold emoji or other 4-byte characters: the
|
||
table got 3-byte utf8. Tables the restore creates now get this installation's character set and
|
||
collation (utf8mb4). Found in a disaster recovery simulation onto MariaDB.
|
||
|
||
### Changed
|
||
- The admin page uses more of the screen (sections up to 1400 pixels instead of 900), so the
|
||
list of backup runs fits without scrolling sideways.
|
||
- README: a roadmap, the tested disaster recoveries, measurements for OVH S3, the Storage Box over
|
||
SFTP and SMB, how to build the SMB extension when PECL is down, and what works without shell
|
||
access. The guide lists shell access among what you need.
|
||
- The full-restore integration test checks a 4-byte character in a table the restore creates, and
|
||
retries app installs from the app store.
|
||
|
||
## [0.9.11] - 2026-10-22 (1.0.0-rc.5)
|
||
|
||
Fifth release candidate for 1.0.0; the app version is 0.9.11. A full restore of 1 TB over SFTP
|
||
stopped after 13 hours with "Blob not in index"; the backup itself was fine.
|
||
|
||
### Changed
|
||
- README and app store description: NG Backup needs background jobs on system cron, with how to
|
||
set that up (issues #13, #17).
|
||
|
||
### Fixed
|
||
- A full restore no longer stops with "Blob not in index" after a pack could not be read for a
|
||
moment. Reading the index again then got an empty listing (from a connection the location had
|
||
dropped) and emptied the index. Reading the index again now only adds, an empty listing of a
|
||
location that has an index is an error instead of an empty index, and a pack that could not be
|
||
read is read once more before the restore gives up.
|
||
- After a full restore of a MySQL/MariaDB or SQLite backup onto PostgreSQL, new files, users or
|
||
activities could not be added ("duplicate key", "File entry could not be inserted"): such
|
||
backups record no ID counters, and they stayed at 1. Every counter of a restored table now goes
|
||
after its data. Found in a disaster recovery simulation of 138 GB from MariaDB to PostgreSQL,
|
||
where all 15,772 files and every count were identical.
|
||
- A full restore now runs in maintenance mode: a background job, a page visit or a user writing
|
||
into the instance meanwhile could collide with restored rows ("duplicate key") or be lost. It
|
||
is switched off when the restore completes; after an error it stays on, and the message gives
|
||
the one line to switch it off and run the restore again. Maintenance mode that was on before
|
||
the restore stays on.
|
||
- Adding a location whose repository config could not be read, while backup data is there,
|
||
no longer creates a new repository over it: it stops with an explanation. A snapshot that
|
||
cannot be taken over when a location is added is now logged instead of left out silently.
|
||
- An SFTP connection that a Hetzner Storage Box dropped without phpseclib noticing is now
|
||
recognised and replaced: the check asks the server and counts its answer.
|
||
- The extra connection a full restore reads ahead on is replaced when it breaks (up to 20 times),
|
||
instead of reading the rest of the restore the slow way.
|
||
|
||
## [0.9.10] - 2026-10-19 (1.0.0-rc.4)
|
||
|
||
Fourth release candidate for 1.0.0; the app version is 0.9.10. 1.0.0-rc.3 (0.9.9) was not
|
||
released: its fixes are in here.
|
||
|
||
### Changed
|
||
- A full restore over SFTP reads the packs ahead on a connection of its own, 32 requests in
|
||
flight, into temporary files in the restore folder. From a Hetzner Storage Box to a server in
|
||
Canada (122 ms) packs now come in at about 14 MiB/s instead of 2.8. More connections can be set
|
||
with `restore_sftp_connections` (README). A part that cannot be read this way is read as
|
||
before. Restores from other locations work as before.
|
||
- App store description: the line about Ensembia.
|
||
|
||
### Fixed
|
||
- After a full restore stopped halfway, running it again with `--force` was refused when the
|
||
database was already restored: Nextcloud then first wants `occ upgrade`. The guide and the
|
||
error message now say so.
|
||
- A full restore of a backup made before 1.0.0-rc.2 with `--skip-missing-tables` failed on
|
||
PostgreSQL with "relation ... does not exist": the left-out tables were still looked up to set
|
||
their ID counters. It now skips them. A new test restores a backup made by 1.0.0-rc.1 on
|
||
PostgreSQL and MariaDB.
|
||
- The app store showed old screenshots: it keeps a copy per URL, and the URLs always pointed at
|
||
`main`. They now point at the release's tag.
|
||
|
||
## [0.9.8] - 2026-10-18 (1.0.0-rc.2)
|
||
|
||
Second release candidate for 1.0.0; the app version is 0.9.8. Fixes from a full disaster recovery
|
||
test of a production instance (about 1 TB, Nextcloud All-in-One onto a plain Nextcloud with
|
||
PostgreSQL).
|
||
|
||
### Added
|
||
- `docs/DISASTER-RECOVERY.md`: step-by-step guide for when your server is gone.
|
||
- `occ backup:restore:full --install-apps` installs (from the app store) or enables the apps the
|
||
old server had enabled, before the restore starts.
|
||
|
||
### Changed
|
||
- Backups record the structure of every database table. A full restore creates tables that do not
|
||
exist on the new instance, for example those left behind by an app that was removed long ago or
|
||
is switched off, instead of stopping at the first one.
|
||
- A full restore no longer refuses when apps are missing: it restores their data and lists the
|
||
apps to install afterwards. `nextcloud-aio` is never required, so a backup of Nextcloud
|
||
All-in-One can be restored onto a plain Nextcloud.
|
||
- A full restore onto an installation with a different table prefix (`dbtableprefix`) is refused
|
||
before it starts, with what to change, instead of restoring into tables nobody uses.
|
||
- All problems are listed at once, with the app each missing table probably belongs to. For
|
||
backups made before this version, `--skip-missing-tables` leaves out tables whose structure is
|
||
unknown.
|
||
- A fuller app store description, ready for 1.0.0, and a note on paid restore help by Ensembia.
|
||
- README: switching the server to TCP BBR made backups over a long distance about 3 times faster.
|
||
|
||
### Fixed
|
||
- A full restore failed on foreign keys (for example those of the Mail app), because tables are
|
||
restored in name order. Foreign keys are now checked after all rows are loaded, before the
|
||
restore commits.
|
||
- `occ backup:target:add` without a key pointed to `backup:key:init`, which makes a new key that
|
||
cannot read existing backups. It now points to `backup:key:import` for a restore, and says that
|
||
connecting to a location with backups can take a while. While it reads the index of a location
|
||
with backups, it shows how far it is and about how long it still takes.
|
||
- A restore stopped with "Not found: packs/..." when the location was cleaned up while it ran,
|
||
for example by the weekly cleanup of the original server during a recovery test on a second
|
||
server: blobs that were moved into new packs were still looked for in the old ones. A restore
|
||
now reloads the index and reads them from where they are now.
|
||
- Removing a location left its cached index in the database (tens of thousands of rows for a large
|
||
backup). It is removed with the location now, and once for locations that are already gone.
|
||
|
||
## [0.9.7] - 2026-10-17 (1.0.0-rc.1)
|
||
|
||
Release candidate for 1.0.0; the app version is 0.9.7. Only bug fixes until 1.0.0. Tested on a
|
||
production instance (about 1 TB, 150,000 files): full and incremental backups and a verify on
|
||
SFTP and SMB (Hetzner Storage Box) and S3 (OVH), a deep verify on SFTP, and partial restores from
|
||
SMB and S3 checked against checksums.
|
||
|
||
### Added
|
||
- The admin page asks for feedback, with a link to a short "Experience report" form on GitHub:
|
||
which location, setup and size you used, and how it went.
|
||
|
||
### Changed
|
||
- New passphrases must be at least 16 characters (was 12). The location stores the key
|
||
encrypted with each passphrase, so the passphrase is what really protects the backups.
|
||
Existing passphrases keep working. The README and the how-to now say so, and why the
|
||
recovery kit must still be kept off the server.
|
||
- Dates and times in the web interface follow the locale in the user's Nextcloud settings instead
|
||
of the browser's language, so a 24-hour locale shows 16:00 rather than 4:00 PM.
|
||
- The list of backup runs (web interface and `occ backup:status`) shows the runs of the last 7
|
||
days, those still running, and at least the 5 newest; a failed run stays visible for that week.
|
||
`occ backup:status --all` and "Show older runs" in the web interface list all of them.
|
||
- `occ backup:status` and `occ backup:verify` show durations of a day or more with the days
|
||
("1d 12:10:13" instead of "12:10:13").
|
||
|
||
### Fixed
|
||
- With system cron, a running backup moved forward only every other cron run: Nextcloud
|
||
skipped the background job when cron came a few seconds before its 5-minute interval was
|
||
over, so a backup was idle 6 of every 10 minutes. The job now runs at every cron run.
|
||
- Photos, video and other compressed data are no longer deflated only to throw the result away:
|
||
a quick test on three small slices of each block skips it. Compressing was the slowest part of
|
||
a backup (about 24 MiB/s).
|
||
- Checking a backup on an S3 location (`occ backup:verify`) took hours on a large repository:
|
||
the index files were downloaded one at a time, at about two a second. They are now downloaded
|
||
16 at a time: on a production instance (27,585 index files, 1 TB) the check went from about
|
||
4 hours to 3.5 minutes.
|
||
- Writing the snapshot at the end of a long backup to S3 took a quarter of an hour without any
|
||
progress shown: it read back the file list of every step one at a time. These reads are now
|
||
done 16 at a time on S3 (96 reads: 82 seconds before, 2 now), blobs that follow each other in
|
||
a pack are read at once on every location, the snapshot is written in a step of its own, and
|
||
the run shows "commit" while it does so.
|
||
|
||
## [0.9.6-beta.7] - 2026-10-14
|
||
|
||
Bugfix release for SMB locations, with `occ backup:cancel` to stop a backup that does not get
|
||
anywhere. Tested on a production instance: a first full backup of 151,021 files (807 GiB) to OVH
|
||
S3 completed, and a backup to a Hetzner Storage Box over SMB that had slowed to 12 GiB in 35
|
||
hours ran at about 5.6 MiB/s after the update, while the S3 backup was still reading.
|
||
|
||
### Added
|
||
- The list of backup runs (web interface and `occ backup:status`) shows the total size of each
|
||
backup next to what it uploaded: an incremental backup of 1 TB that uploads 800 MB now shows
|
||
both. `occ backup:status` shows sizes in GiB where that reads better than MiB.
|
||
- `occ backup:cancel <run>` stops a running backup; `occ backup:status` shows the run numbers.
|
||
No new step of the run starts, the step in progress finishes (usually within a minute), then
|
||
the process working on the run (`occ backup:run` or cron) stops. What it already stored is
|
||
removed by the next cleanup. A backup started by an earlier version is stopped as well: after
|
||
two minutes it is marked cancelled while it works, and its process stops at the end of the
|
||
step.
|
||
|
||
### Fixed
|
||
- Backups to SMB locations, and to other slow locations with many packs, slowed to a crawl: each
|
||
30-second step opened the location again, and reading the list of index files took longer
|
||
than the step itself. Each step then stored a single 4 MiB pack, which made the list longer
|
||
still. A first full backup of 0.8 TB to a Hetzner Storage Box over SMB managed 12 GiB in 35
|
||
hours. `occ backup:run` and system cron now open the location once per run, opening no longer
|
||
counts towards the step's time, and SMB folders are listed in one request instead of one per
|
||
file. A backup to an SMB location that is not getting anywhere keeps running in its old
|
||
process after the update: update first, then stop it with `occ backup:cancel <run>` and start
|
||
it again. The new backup reuses the packs the old one stored.
|
||
|
||
## [0.9.5-beta.6] - 2026-10-12
|
||
|
||
Bugfix release for SFTP locations and large repositories. Tested on a production instance with a
|
||
1 TB repository on a Hetzner Storage Box over SFTP: a check in 5.5 minutes, a deep check of all
|
||
316,529 blobs (815 GiB) in 11 hours, and an incremental backup in under 8 minutes.
|
||
|
||
### Added
|
||
- Progress of a running backup in the web interface and `occ backup:status`, e.g.
|
||
"running (files) 68%". The total is the size of the previous snapshot; for a first backup it
|
||
is estimated from Nextcloud's file cache (users' files, versions and trash) and shown as
|
||
"~68%".
|
||
- Edit a backup location (web interface and `occ backup:target:edit`): change the password, host,
|
||
folder, connection type or name without losing its history. Before this the only option was a
|
||
new location, which starts an empty repository and, with a retention lock at the provider,
|
||
keeps paying for the old one. The new settings are only saved when they lead to the same
|
||
repository; stored passwords are never sent to the browser, and an empty password field keeps
|
||
the stored one. Administrators are notified of the change.
|
||
- `occ backup:verify` reports how far it is every 30 seconds, and `occ backup:status` shows the
|
||
progress of a running check and how the last check of each location ended. On a large
|
||
repository a check takes hours, a deep one days.
|
||
|
||
### Changed
|
||
- Checking and opening a large repository over SFTP is much faster: the index files (one per
|
||
pack, 27,547 for 1 TB) are downloaded 32 at a time instead of one after the other, 114 instead
|
||
of 6 files per second to a Hetzner Storage Box (about 4 minutes instead of 77 for 1 TB). A check
|
||
of a repository with more than 256 packs also lists the pack folders (256 requests) instead of
|
||
asking for every pack.
|
||
- App store listing: the author's e-mail address is shown.
|
||
- App store listing: the homepage link points to the GitHub repository instead of back to the
|
||
app store page itself.
|
||
- The notification for a location without a successful backup read "No successful backup to …
|
||
since never" when there had never been one; it now reads "No successful backup to … yet".
|
||
|
||
### Fixed
|
||
- 0.9.4 could not open a repository at the root of an SFTP location (empty path): its faster
|
||
SFTP reads sent their first request before phpseclib had opened the SFTP channel, which only
|
||
happens on the first SFTP call after login. Every backup then failed with "No repository at
|
||
this location". The channel is now opened first.
|
||
- A check (`occ backup:verify`) did not renew its hold on the location while it downloaded the
|
||
index; on a large repository that takes longer than the hold lasts, so a weekly cleanup could
|
||
have started during the check.
|
||
- A long first backup could end as failed although its snapshot was complete: when its final
|
||
step took longer than the step's lease (150 s with `occ backup:run`), the background job took
|
||
the run over, finished it a second time and failed on the snapshot it had already recorded,
|
||
overwriting the result. The lease now lasts at least 15 minutes, a step writes the run only if
|
||
no other process wrote it since the step loaded it (in one database statement), and recording
|
||
a snapshot that is already recorded is no longer an error.
|
||
- `occ backup:verify` on a large repository failed with "SFTP connection failed while opening
|
||
snapshots/…" or "Not found: snapshots/…". A Hetzner Storage Box drops an SFTP connection that
|
||
carried a lot of data (some 40 MB) and then stayed quiet for a few minutes, and the check went
|
||
quiet after downloading the index: it refreshed the local index cache, first in 29 minutes (a
|
||
pass over all blobs for every pack: 27,547 packs, 400,000 blobs for 1 TB), then still in more
|
||
than 6 minutes (a database transaction per pack on PostgreSQL). Nextcloud's SFTP storage keeps
|
||
using a dropped connection, on which every file looks missing. Now:
|
||
- an SFTP connection that was not used for a minute is checked first, and one that turns out
|
||
to be dead after a failed request is replaced, once per request; uploads, moves and deletes
|
||
are only checked first, never repeated;
|
||
- the index cache is written in one transaction (27,547 packs in 11 seconds on a test server,
|
||
against about a minute row by row);
|
||
- after a check the cache holds the index files as they are: before, a blob that was in two
|
||
packs was only cached for one of them.
|
||
|
||
## [0.9.4-beta.5] - 2026-10-09
|
||
|
||
Bugfix release: faster restores.
|
||
|
||
### Fixed
|
||
- Restores and `occ backup:verify --deep` fetched every blob with its own request, so on a
|
||
remote location they ran far below the speed of the connection. The blobs of a pack are now
|
||
read from one open stream, in the order they were written. Restoring 86 MB in 202 files from a
|
||
Hetzner Storage Box over SMB went from 0.78 MB/s to 4.2 MB/s, the speed of reading the packs
|
||
as plain files over that connection.
|
||
- Reading from an SFTP location waited a full round trip for every 256 KiB, whatever the line
|
||
speed. Reads now keep up to 8 requests in flight. Restoring 1.97 GB in 500 files from a Hetzner
|
||
Storage Box (47 ms away) took 694 s before the fix above, 499 s with it and 142 s with both
|
||
(2.8 → 3.9 → 13.9 MB/s); a full 51 GB restore ran at 13.0 MB/s. When a server error
|
||
interrupts such a read, it goes on from the same position with Nextcloud's own SFTP stream;
|
||
after three failing files in a row, or on a Nextcloud whose SFTP library lacks what this needs,
|
||
only Nextcloud's own stream is used.
|
||
- README: a WebDAV location on a slow upload needs a higher `davstorage.request_timeout`.
|
||
|
||
## [0.9.3-beta.4] - 2026-10-08
|
||
|
||
Bugfix release.
|
||
|
||
### Fixed
|
||
- A backup failed with "cannot execute INSERT in a read-only transaction" on PostgreSQL as soon
|
||
as the database dump filled a whole pack (32 MiB): the pack's index cache row was written
|
||
inside the read-only dump transaction. On other databases that row was rolled back silently.
|
||
- Backups to a remote location slowed down as the repository grew: every backup step listed the
|
||
repository's index with a round trip per file. Over SFTP to a Hetzner Storage Box a 60 GiB
|
||
repository already lost a large part of each step to it; backing up continuously
|
||
(`occ backup:run`) went from 45 KiB/s to 5.8 MB/s with the fix.
|
||
|
||
## [0.9.2-beta.3] - 2026-10-08
|
||
|
||
Bugfix release.
|
||
|
||
### Added
|
||
- README: notes on using a Hetzner Storage Box over WebDAV and SMB.
|
||
|
||
### Fixed
|
||
- Adding a backup location in the web interface failed with "Request failed with status code 403"
|
||
when the login was more than 30 minutes old: the form did not ask for password confirmation.
|
||
- Settings page: the "Backup and restore" / "Settings" tabs line up with the sections below them.
|
||
- Backups: the runs table scrolls horizontally on narrow screens instead of running off the page.
|
||
- `occ backup:verify --deep`: a blob that could not be read (network or storage error) was reported
|
||
as "corrupted or tampered". Reads are now retried, and a read that keeps failing is reported
|
||
separately as unreadable, with the error (run the check again; if the same blobs stay unreadable,
|
||
their pack may be damaged or truncated). Only a blob that was read but fails decryption or its
|
||
hash check counts as corrupted. Found on a Hetzner Storage Box over WebDAV, where the blob
|
||
turned out to be intact.
|
||
|
||
## [0.9.1-beta.2] - 2026-10-07
|
||
|
||
Second beta, the first release in the Nextcloud app store (signed).
|
||
|
||
### Added
|
||
- App store listing: screenshots, website, repository and documentation link in `appinfo/info.xml`.
|
||
|
||
### Fixed
|
||
- Restore: the "new folder" option showed `"Restored "` without the date pattern.
|
||
- Restore: the restore-point dropdown cut off the time.
|
||
- Settings: the passphrase actions are on one line instead of stacked.
|
||
- `docs/HOWTO.md`: the SFTP example used a non-existent auth mechanism (`sftp::password`
|
||
instead of `password::password`).
|
||
|
||
## [0.9.0-beta.1] - 2026-10-07
|
||
|
||
First beta. Tested thoroughly and usable **alongside** your current backup, but **not yet a
|
||
drop-in replacement** for it. We test what we can ourselves (Nextcloud 32–35, SQLite, MariaDB and
|
||
PostgreSQL, S3, SFTP, FTP, SMB and NFS), but we need field experience from other setups: please
|
||
report what works and what doesn't, especially on large instances and on storage providers not
|
||
listed in the README.
|
||
|
||
### Added
|
||
- Disaster recovery: `occ backup:key:import` and `occ backup:restore:full` restore the database,
|
||
data directory and config onto a fresh installation of the same major version.
|
||
- Per-user backup and restore via user_migration: `occ backup:user:backup` and
|
||
`occ backup:user:restore` (replace the account after a safety export, or restore as
|
||
`<uid>-bak`), also onto a server where the user does not exist yet. The commands ask before
|
||
installing user_migration (or take `--install-user-migration`).
|
||
- Checksum audit: `occ backup:verify`, shallow or `--deep`, for snapshots and per-user exports.
|
||
- Retention for per-user exports: `occ backup:retention --user-last=N`.
|
||
- FTP and SMB covered by the integration tests; NFS through the `local` backend on a mounted share.
|
||
- `docs/HOWTO.md`: step by step from setup to a tested restore.
|
||
- Warnings when a backup finishes with unreadable files or folders, or with files that kept
|
||
changing while being read (log, `occ`, notification, snapshot).
|
||
|
||
### Changed
|
||
- `backup:restore:full` refuses on an installation with more than one user and asks first;
|
||
`backup:user:restore --mode=replace` asks before deleting an existing account (`--force` for
|
||
scripts in both cases).
|
||
- A failed scheduled backup is retried at most three times a day, an hour apart.
|
||
- The recovery kit carries the deletion delay; the kit file is created owner-only.
|
||
- Adding a location through the web API needs password confirmation.
|
||
|
||
### Fixed
|
||
- Exports of users whose id contains `.part-` could be deleted by cleanup on remote locations.
|
||
- Cleanup no longer frees the data of a per-user export whose manifest is missing.
|
||
- Crash safety of moving snapshots to and from the trash, of catalog writes by concurrent
|
||
backups and per-user exports, and of short uploads.
|
||
- A file that keeps changing no longer makes a backup restart forever.
|
||
- Long restores and exports keep their lock on the location alive.
|
||
- Floating-point database columns are dumped exactly.
|
||
|
||
### Known limitations
|
||
See "Known limitations" in the README: memory use on very large instances, never two servers
|
||
writing to one location, removing a passphrase slot does not revoke access, no rollback detection
|
||
before a disaster recovery, and a few smaller points. None of them block the road to 1.0.0; they
|
||
are on the backlog in `PLAN.md`.
|
||
|
||
## [0.1.0-alpha.1] - 2026-10-05
|
||
|
||
First public alpha. Feature-complete for phase 1: scheduled, encrypted, incremental backups, and
|
||
restore of files and folders. **Not for production use** — test on a copy of production (dev,
|
||
test, staging). Full disaster recovery (`backup:restore:full` on a fresh installation) is phase 2
|
||
and not implemented yet.
|
||
|
||
### Added
|
||
- Encrypted (XChaCha20-Poly1305), content-addressed, deduplicated, incremental backups, streamed
|
||
from source to destination without a local copy of the data.
|
||
- Backup locations via files_external's own backend/auth classes, without ever mounting the
|
||
location: S3 (and compatible services), SFTP, WebDAV, FTP, Swift, or a local path.
|
||
- A recovery kit with up to three passphrase slots, mandatory download-and-confirm before the
|
||
first backup runs, and an optional encrypted copy by email.
|
||
- Resumable, time-boxed background jobs (database dump, file walk, upload), so a run survives
|
||
short PHP time limits and a crash is resumed, never restarted, without leaving the instance in
|
||
maintenance mode.
|
||
- Database dump and restore without external tools, via Nextcloud's own `IDBConnection`/DBAL,
|
||
streamed through the same pipeline as file data.
|
||
- Restore of files and folders via the web interface and `occ`, into a new folder, merged, or
|
||
replacing the original (with the previous content going to the trash).
|
||
- Retention policy (last/daily/weekly/monthly), prune with pack repacking, and a snapshot trash
|
||
with a configurable deletion delay.
|
||
- Append-only locations and documented recommendations against ransomware and a compromised
|
||
server (object lock, delete-less credentials, off-site copies).
|
||
- Admin settings page (locations, schedule, retention, key management) and a JSON API.
|
||
- Notifications and an admin setup check for a failed or overdue backup.
|
||
- CI: unit tests on PHP 8.2–8.4, and an end-to-end backup→restore round trip on Nextcloud 32–35 ×
|
||
SQLite/MariaDB/PostgreSQL, and against real S3 and SFTP backends (not just a local path).
|
||
|
||
### Known limitations
|
||
- No full/disaster-recovery restore yet (phase 2).
|
||
- No per-user backup/restore via user_migration yet (phase 2).
|
||
- No FTP/SMB coverage in CI yet, and no verification/checksum-audit command yet (phase 2).
|
||
- See the "Storage backend experiences" section of the README for provider-specific quirks found
|
||
so far (e.g. Hetzner Storage Box sub-accounts).
|
||
|
||
[1.0.1]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.1
|
||
[1.0.0]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0
|
||
[0.9.12]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.6
|
||
[0.9.11]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.5
|
||
[0.9.10]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.4
|
||
[0.9.8]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.2
|
||
[0.9.7]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.1
|
||
[0.9.6-beta.7]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.6-beta.7
|
||
[0.9.5-beta.6]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.5-beta.6
|
||
[0.9.4-beta.5]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.4-beta.5
|
||
[0.9.3-beta.4]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.3-beta.4
|
||
[0.9.2-beta.3]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.2-beta.3
|
||
[0.9.1-beta.2]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.1-beta.2
|
||
[0.9.0-beta.1]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.0-beta.1
|
||
[0.1.0-alpha.1]: https://github.com/dillardblom/ng_backup/releases/tag/v0.1.0-alpha.1
|