On S3, index files and the tree segments read back when a snapshot is
written are fetched 16 at a time through the storage's S3 client
(IBackend::getRanges(), getMany()). Blobs that follow each other in a pack
are read as one range on every location. The snapshot is written in a step
of its own and the run shows "commit" meanwhile. On a 1 TB production
repository the check went from about 4 hours to 3.5 minutes.
backup:status and the web interface list the runs of the last 7 days,
running ones and at least the newest 5; --all / "Show older runs" list
all. Durations of a day or more show the days.
A run only showed "running (files)", which says nothing during a backup
of a terabyte. The files phase now records an expected size on its first
step: the previous snapshot's size, or for a first backup the users'
files, versions and trash from Nextcloud's file cache (approximate, shown
with a ~). The web interface and occ backup:status show the share of it
done so far, including a large file that is still being read; it stays
below 100% until the run finishes.
Show the date of the restore point a restore run used, both in the
backup history table and when confirming a restore. Split the admin
page into a "Backup and restore" tab (locations, backups, restore)
and a "Settings" tab (key, slots, recovery kit), so daily work no
longer sits behind the key setup at the top of the page.
Signed-off-by: Dillard Blom <dillard.blom@ensembia.com>
- removed snapshots go to a trash on the location and their data is only
freed after the deletion delay, which is chosen when the key is created
(default 7 days, 1-365) and cannot be changed afterwards; any admin can
bring them back (occ backup:trash, backup:trash:restore, web UI)
- up to three passphrase slots open the same master key (like LUKS), so
one person leaving or dying does not lock the company out; slot changes
are written to every location, announced to all admins and require a
new recovery kit confirmation, while backups keep running
- occ backup:key:slots, backup:key:slot:add|replace|remove; key init takes
--label and --delete-delay-days
- web UI: deletion delay and first slot at key creation, slot management,
trash per location with restore; all behind password confirmation
- README: key, slots (safe / CTO / head of IT), and protection against
ransomware: object lock in compliance mode (e.g. 90 days), credentials
without delete rights, provider snapshots, off-site copy
- AdminApiController (admin only, CSRF-protected): status, create key,
download recovery kit, confirm (statement accepted + exact phrase),
backends with their usable auth mechanisms, add/test/remove locations,
start backups, schedule, snapshots, browse, queue restores
- RestoreJob runs web-initiated restores in the background; backup and
restore runs are kept apart in the run table
- admin section 'NG Backup' with a Vue 3 / @nextcloud/vue 9 page: key and
recovery-kit flow (download, tick statement, 'Yes, I confirm'), location
form built from files_external's parameter definitions, schedule, run
list with polling, snapshot browser and restore with the three modes
- auth mechanisms that depend on a user session or on files_external's
own stored credentials are not offered
API tested with curl on NC34 (status, backends, schedule validation,
confirmation refusal, browse, queued restore and backup completed by
cron); non-admins get 403, anonymous 401.
dillard