- The admin page and backup:status warn when background jobs do not run on a
system cron, or the cron stopped, with a link to how to set one up.
- Without a working system cron, the open admin page moves a running backup
forward through /api/tick, a short step per poll.
- The database phase shows the tables dumped ("db: 47/183 tables"); a backup
that has not moved for 15 minutes says it is waiting for background jobs.
- The trash of restore points is on a tab of its own, Bin.
- New screenshots; changelog and plan for 1.0.2.
- Admin page: an optional label (at most 100 characters) for a backup started with "Back up now";
an (i) by "removed by retention policy" in the trash that explains what it means and until when
the restore point can be brought back. backup:trash says the same below its list.
- backup:status, backup:list, backup:trash and backup:browse show times in the zone of the daily
schedule and name it, instead of the server's UTC.
- backup:restore --to-directory of a file restored nothing: the path got a "/" appended.
- A restore removes "<name>.ngb-restore-<hex>" temporaries that a stopped restore left in the
folders it writes to.
- Roadmap: the cron warning (#13), reading ahead from S3/SMB and --salvage move to 1.0.2.
backup:user:restore <location> <uid> only said "is not a recorded user export": the argument
is the manifest path, which only backup:user:backup printed once. backup:user:list shows the
exports again; a bare user id works when that user has exactly one export, otherwise the error
lists them. The export is resolved before the replace confirmation.
Fixes#22
On S3, index files and the tree segments read back when a snapshot is
written are fetched 16 at a time through the storage's S3 client
(IBackend::getRanges(), getMany()). Blobs that follow each other in a pack
are read as one range on every location. The snapshot is written in a step
of its own and the run shows "commit" meanwhile. On a 1 TB production
repository the check went from about 4 hours to 3.5 minutes.
backup:status and the web interface list the runs of the last 7 days,
running ones and at least the newest 5; --all / "Show older runs" list
all. Durations of a day or more show the days.
Each 30-second step opened the location again and read its list of
index files. Over SMB that cost a request per file, so on a large
repository opening took longer than the step: each step stored one
4 MiB pack and the list grew further. A first backup of nc01 to a
Storage Box over SMB managed 12 GiB in 35 hours.
occ backup:run and system cron now keep the repository open for the
whole run (reopened when another process took a step or the location
was edited), opening no longer counts towards the step's time, and
SMB folders are listed in one request.
occ backup:cancel stops a running backup: no new step starts and the
step in progress finishes first. A run of an earlier version, which
does not hold back its next step, is marked cancelled after two
minutes unless it is recording its snapshot.
A changed password or host name could only be handled by adding a new
location, which starts an empty repository and, with a retention lock at
the provider, keeps the old one billed. Edit now changes a location in
place (web interface and occ backup:target:edit). The settings are only
saved when they lead to the same repository; a failed connection and a
missing repository give separate errors, and the folder is not created
while checking. Stored passwords are never sent to the browser: an empty
password field keeps the stored value.
An unreadable blob may still be a truncated pack, so don't rule out damage
in the verify message; README: Hetzner needs WebDAV/SMB and external
reachability enabled for the sub-account.
Tested both on sub-accounts: works end to end; WebDAV needs an existing
root, SMB needs port 445 and the smbclient extension; fixed cost per run
and slow deep verify over remote storage.
The open points from the pre-beta review are on the backlog in PLAN.md and
listed as known limitations in the README; none block 0.9.0-beta or 1.0.0.
The how-to warns against pointing a test server at a live location and
explains that removing a slot does not revoke access.
backup:user:backup and backup:user:restore now ask the administrator before
installing or enabling user_migration (or take --install-user-migration), so
nothing changes on the server without consent. The restore warns that
user_migration imports only its allowlisted app settings.
Also: a clear error for unknown locations instead of a raw database message;
prune reports the number of affected users, not the number of kept ones; and
--user-last rejects values that are not whole numbers. Adds docs/HOWTO.md and
updates the README and PLAN.md for the beta.
No ng_backup code changes needed: files_external's generic backend/auth adapter (the same
ExternalStorageFactory/StorageBackend already used for S3/SFTP) already covers FTP and SMB, and
NFS was never a files_external backend to begin with (PLAN.md 3.5) -- the "local" backend plus an
admin-managed OS-level mount already does the job. This commit is test coverage and documentation.
tests/integration/targets-matrix.sh now also runs the full round trip against:
- FTP (stilliard/pure-ftpd). A first attempt with delfer/alpine-ftp-server (vsftpd) failed with a
generic "create directory operation failed": files_external's FTP backend always builds
absolute paths, which that server's chroot rejects even via plain php ftp_mkdir() with an
absolute path (a relative path on the same server works fine) -- not a files_external or
ng_backup bug, just a server-specific quirk, confirmed by switching test servers.
- SMB (dperson/samba), via a small Nextcloud image built on the fly with the smbclient PECL
extension (libsmbclient-dev + pecl install), in its own temp directory as the build context
(never /tmp directly). The backend *lists* as available with just the smbclient CLI binary too
(files_external's shell-wrapped fallback), but that fallback's stream cannot seek, and ng_backup
reads blobs back by byte range while backing up -- confirmed live, it fails the very first
(small) backup with "Seek failed in packs/...". The native extension is genuinely required here,
not just preferred.
README: new "Storage backend experiences" entries for both findings (and a pointer from the
"no extra PHP extensions" intro line, since SMB is the one real exception), plus NFS guidance
(mount it yourself, point the local backend at the mount point).
PLAN.md: FTP/SMB marked done; SMB/NFS testing on the user's own OMV instance stays open (internal,
not something CI can cover).
Local review (Codex + CodeRabbit) found three issues, all fixed: the README's first-pass SMB claim
("CLI tool is not enough") was imprecise about *why* -- it lists fine without the extension, it
just doesn't work for ng_backup's read pattern; the cleanup trap could abort before removing the
network if an earlier docker rm failed, and didn't remove the built image/Dockerfile; and the
Docker build used all of /tmp as its context instead of a dedicated directory.
info.xml's version field is restricted by Nextcloud's own info.xsd to plain
numeric SemVer, no pre-release suffix, so the alpha/beta/rc stage from
PLAN.md's release table lives in the git tag and CHANGELOG instead; noted
that in PLAN.md so it isn't rediscovered later. Bumped the version, wrote
an honest description for an alpha (feature-complete for phase 1, not for
production), and added CHANGELOG.md.
README: new "Storage backend experiences" section, seeded with the Hetzner
Storage Box sub-account root-path quirk found while testing a real SFTP
target (root must be "home", not "/"), open for others to extend via PR.
- restore runs from the web are created under the location's exclusive
lock, like backup runs
- raw restore writes to an unpredictable temporary name opened with
exclusive create (never follows a planted symlink) and validates the
path prefix; the restore service rejects non-canonical paths instead
of trimming them (occ normalises user input before calling it)
- restore jobs report progress every minute, so a long restore is not
taken for a dead one
- README: Object Lock turns deletions into delete markers
- removed snapshots go to a trash on the location and their data is only
freed after the deletion delay, which is chosen when the key is created
(default 7 days, 1-365) and cannot be changed afterwards; any admin can
bring them back (occ backup:trash, backup:trash:restore, web UI)
- up to three passphrase slots open the same master key (like LUKS), so
one person leaving or dying does not lock the company out; slot changes
are written to every location, announced to all admins and require a
new recovery kit confirmation, while backups keep running
- occ backup:key:slots, backup:key:slot:add|replace|remove; key init takes
--label and --delete-delay-days
- web UI: deletion delay and first slot at key creation, slot management,
trash per location with restore; all behind password confirmation
- README: key, slots (safe / CTO / head of IT), and protection against
ransomware: object lock in compliance mode (e.g. 90 days), credentials
without delete rights, provider snapshots, off-site copy
dillard