Off, once a day or every N hours from a time, each in its own time zone. Existing locations
keep the time that held for all of them. A backup that has to wait shows as queued, and its
restore point has the time it really began.
Both live in the catalog on the location, so a new server sees them; the database holds
a local copy for the lists. The retention policy leaves a pinned restore point alone and
does not count it. Available on the restore page and as occ backup:pin and backup:note.
- The admin page and backup:status warn when background jobs do not run on a
system cron, or the cron stopped, with a link to how to set one up.
- Without a working system cron, the open admin page moves a running backup
forward through /api/tick, a short step per poll.
- The database phase shows the tables dumped ("db: 47/183 tables"); a backup
that has not moved for 15 minutes says it is waiting for background jobs.
- The trash of restore points is on a tab of its own, Bin.
- New screenshots; changelog and plan for 1.0.2.
- Admin page: an optional label (at most 100 characters) for a backup started with "Back up now";
an (i) by "removed by retention policy" in the trash that explains what it means and until when
the restore point can be brought back. backup:trash says the same below its list.
- backup:status, backup:list, backup:trash and backup:browse show times in the zone of the daily
schedule and name it, instead of the server's UTC.
- backup:restore --to-directory of a file restored nothing: the path got a "/" appended.
- A restore removes "<name>.ngb-restore-<hex>" temporaries that a stopped restore left in the
folders it writes to.
- Roadmap: the cron warning (#13), reading ahead from S3/SMB and --salvage move to 1.0.2.
backup:user:restore <location> <uid> only said "is not a recorded user export": the argument
is the manifest path, which only backup:user:backup printed once. backup:user:list shows the
exports again; a bare user id works when that user has exactly one export, otherwise the error
lists them. The export is resolved before the replace confirmation.
Fixes#22
On S3, index files and the tree segments read back when a snapshot is
written are fetched 16 at a time through the storage's S3 client
(IBackend::getRanges(), getMany()). Blobs that follow each other in a pack
are read as one range on every location. The snapshot is written in a step
of its own and the run shows "commit" meanwhile. On a 1 TB production
repository the check went from about 4 hours to 3.5 minutes.
backup:status and the web interface list the runs of the last 7 days,
running ones and at least the newest 5; --all / "Show older runs" list
all. Durations of a day or more show the days.
Each 30-second step opened the location again and read its list of
index files. Over SMB that cost a request per file, so on a large
repository opening took longer than the step: each step stored one
4 MiB pack and the list grew further. A first backup of nc01 to a
Storage Box over SMB managed 12 GiB in 35 hours.
occ backup:run and system cron now keep the repository open for the
whole run (reopened when another process took a step or the location
was edited), opening no longer counts towards the step's time, and
SMB folders are listed in one request.
occ backup:cancel stops a running backup: no new step starts and the
step in progress finishes first. A run of an earlier version, which
does not hold back its next step, is marked cancelled after two
minutes unless it is recording its snapshot.
A changed password or host name could only be handled by adding a new
location, which starts an empty repository and, with a retention lock at
the provider, keeps the old one billed. Edit now changes a location in
place (web interface and occ backup:target:edit). The settings are only
saved when they lead to the same repository; a failed connection and a
missing repository give separate errors, and the folder is not created
while checking. Stored passwords are never sent to the browser: an empty
password field keeps the stored value.
An unreadable blob may still be a truncated pack, so don't rule out damage
in the verify message; README: Hetzner needs WebDAV/SMB and external
reachability enabled for the sub-account.
Tested both on sub-accounts: works end to end; WebDAV needs an existing
root, SMB needs port 445 and the smbclient extension; fixed cost per run
and slow deep verify over remote storage.
The open points from the pre-beta review are on the backlog in PLAN.md and
listed as known limitations in the README; none block 0.9.0-beta or 1.0.0.
The how-to warns against pointing a test server at a live location and
explains that removing a slot does not revoke access.
backup:user:backup and backup:user:restore now ask the administrator before
installing or enabling user_migration (or take --install-user-migration), so
nothing changes on the server without consent. The restore warns that
user_migration imports only its allowlisted app settings.
Also: a clear error for unknown locations instead of a raw database message;
prune reports the number of affected users, not the number of kept ones; and
--user-last rejects values that are not whole numbers. Adds docs/HOWTO.md and
updates the README and PLAN.md for the beta.
No ng_backup code changes needed: files_external's generic backend/auth adapter (the same
ExternalStorageFactory/StorageBackend already used for S3/SFTP) already covers FTP and SMB, and
NFS was never a files_external backend to begin with (PLAN.md 3.5) -- the "local" backend plus an
admin-managed OS-level mount already does the job. This commit is test coverage and documentation.
tests/integration/targets-matrix.sh now also runs the full round trip against:
- FTP (stilliard/pure-ftpd). A first attempt with delfer/alpine-ftp-server (vsftpd) failed with a
generic "create directory operation failed": files_external's FTP backend always builds
absolute paths, which that server's chroot rejects even via plain php ftp_mkdir() with an
absolute path (a relative path on the same server works fine) -- not a files_external or
ng_backup bug, just a server-specific quirk, confirmed by switching test servers.
- SMB (dperson/samba), via a small Nextcloud image built on the fly with the smbclient PECL
extension (libsmbclient-dev + pecl install), in its own temp directory as the build context
(never /tmp directly). The backend *lists* as available with just the smbclient CLI binary too
(files_external's shell-wrapped fallback), but that fallback's stream cannot seek, and ng_backup
reads blobs back by byte range while backing up -- confirmed live, it fails the very first
(small) backup with "Seek failed in packs/...". The native extension is genuinely required here,
not just preferred.
README: new "Storage backend experiences" entries for both findings (and a pointer from the
"no extra PHP extensions" intro line, since SMB is the one real exception), plus NFS guidance
(mount it yourself, point the local backend at the mount point).
PLAN.md: FTP/SMB marked done; SMB/NFS testing on the user's own OMV instance stays open (internal,
not something CI can cover).
Local review (Codex + CodeRabbit) found three issues, all fixed: the README's first-pass SMB claim
("CLI tool is not enough") was imprecise about *why* -- it lists fine without the extension, it
just doesn't work for ng_backup's read pattern; the cleanup trap could abort before removing the
network if an earlier docker rm failed, and didn't remove the built image/Dockerfile; and the
Docker build used all of /tmp as its context instead of a dedicated directory.
dillard