72 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dillard	a8da8cb54a	info.xml: website points to the GitHub repository ... The app store shows <website> as the app's homepage; pointing it at the store page itself made the link go nowhere.	2026-10-08 02:23:59 +02:00
Dillard Blom	bfd54447b0	Merge pull request #6: Release 0.9.2-beta.3 v0.9.2-beta.3	2026-10-08 01:28:52 +02:00
dillard	0b50bb6b81	Verify: retry storage exceptions only, let programming errors surface ... Catch \Exception (storage adapters throw their own exception types) instead of \Throwable, so an \Error such as a TypeError is not retried and reported as an unreadable blob.	2026-10-08 01:21:15 +02:00
dillard	f415bab519	Verify, README: address review notes ... An unreadable blob may still be a truncated pack, so don't rule out damage in the verify message; README: Hetzner needs WebDAV/SMB and external reachability enabled for the sub-account.	2026-10-08 01:11:21 +02:00
dillard	27eb0b32fc	Release 0.9.2-beta.3	2026-10-08 01:00:52 +02:00
dillard	b1d4f6a469	README: Hetzner Storage Box over WebDAV and SMB ... Tested both on sub-accounts: works end to end; WebDAV needs an existing root, SMB needs port 445 and the smbclient extension; fixed cost per run and slow deep verify over remote storage.	2026-10-08 01:00:46 +02:00
dillard	fe7ad9b4b5	Verify: report read errors as unreadable, not as corrupted ... verifyBlob() caught every exception from loading a blob and counted it as a failed (corrupted or tampered) blob, so a transient network error during a deep verify over WebDAV looked like data corruption. Read the range separately, retry it, and report a read that keeps failing as unreadable with its error; only a blob that was read but does not decrypt or match its id counts as failed.	2026-10-08 00:53:31 +02:00
dillard	695d972785	Backups: let the runs table scroll on narrow screens ... The table could not shrink below its content width and ran off the page, cutting off the last columns; wrap it in an overflow-x container.	2026-10-08 00:47:47 +02:00
dillard	972a52e665	Settings: align the tabs with the sections ... The tabs had no inline margin, so they sat against the edge while every NcSettingsSection is inset by 7 grid baselines.	2026-10-08 00:47:47 +02:00
dillard	7c08a10621	Locations: ask for password confirmation before adding ... addTarget requires a recent password confirmation, but the form never asked for one. Within 30 minutes of logging in that went unnoticed; after that, adding a location failed with a bare 403.	2026-10-08 00:40:48 +02:00
Dillard Blom	209520165e	Merge pull request #5: Release 0.9.1-beta.2 v0.9.1-beta.2	2026-10-07 18:09:23 +02:00
dillard	9554e774e5	CHANGELOG: correct why info.xml carries no -beta suffix	2026-10-07 18:01:20 +02:00
dillard	8b2dca28e3	Release 0.9.1-beta.2	2026-10-07 17:47:40 +02:00
dillard	5742f77253	App store listing: screenshots, links and UI fixes ... Add three screenshots (backups, restore, settings) taken on a demo instance, plus website, repository and the HOWTO as admin documentation in info.xml. Fixes found while taking them: - the "new folder" restore option lost "<date>" (l10n sanitizes it as a tag); show the folder name pattern via a placeholder instead - the restore-point dropdown cut off the time - passphrase actions stacked vertically in the slot table - HOWTO: SFTP example used "sftp::password", the auth mechanism is "password::password"	2026-10-07 17:43:41 +02:00
Dillard Blom	550249570b	Merge pull request #4: README key rotation note ... README: key rotation for revoking old kits is on the backlog	2026-10-07 02:26:19 +02:00
dillard	0eb23f56e7	README: note that key rotation for revoking old kits is on the backlog	2026-10-07 02:18:45 +02:00
Dillard Blom	a0c1f1cd58	Merge pull request #3: Release 0.9.0-beta.1 ... Release 0.9.0-beta.1 v0.9.0-beta.1	2026-10-07 01:15:19 +02:00
dillard	26cc9a897e	Release 0.9.0-beta.1 ... Version 0.9.0 in info.xml, the app description for the beta, and the CHANGELOG entry for 0.9.0-beta.1.	2026-10-07 01:07:55 +02:00
Dillard Blom	fe0385ad8c	Merge pull request #2: Pre-beta review ... Pre-beta review: data safety, safer commands, known limitations	2026-10-07 01:06:40 +02:00
dillard	b37a953cfd	Detect a busy run by identity, and clean up a failed kit write ... step() hands back the same Run object when another process holds the lease; comparing update times per second could stop the scheduled job early after a quick step. A failed write of the recovery kit file now closes the handle and removes the partial file.	2026-10-07 00:48:11 +02:00
dillard	b89b5b5fa6	Document known limitations for the beta and record the review backlog ... The open points from the pre-beta review are on the backlog in PLAN.md and listed as known limitations in the README; none block 0.9.0-beta or 1.0.0. The how-to warns against pointing a test server at a live location and explains that removing a slot does not revoke access.	2026-10-07 00:19:46 +02:00
dillard	16e09f4fde	Fix trash on SFTP and the disaster recovery test after the review fixes ... forget() and untrash() uploaded a remote read stream directly, which the new size check could not judge on SFTP, so cleanup failed there. They now copy through a local buffer, and the size check only trusts local streams (php://temp, php://memory, plain files). The disaster recovery test calls backup:restore:full with --no-interaction: docker exec without a terminal still counts as interactive, so the new confirmation question read an empty answer and cancelled.	2026-10-06 23:59:17 +02:00
dillard	45069994e9	Pre-beta review: safer destructive commands and hardening ... - backup:restore:full refuses on an installation with more than one user and asks before it starts (--force skips both, also to rerun after a failure halfway), and warns when the snapshot's database dump was not one consistent transaction. - backup:user:restore --mode=replace asks before deleting an existing account; non-interactive use needs --force. - The recovery kit file is created owner-only and never over an existing file or symlink; backup:key:import shows the deletion delay it read from the kit. - Adding a location through the web API needs password confirmation, like removing one. - Key derivation parameters from a kit or repository config are bounded, and a stalled network stream no longer spins at full CPU. - Floating-point columns are dumped exactly instead of with 14 digits.	2026-10-06 23:48:41 +02:00
dillard	0f4a238414	Pre-beta review: data safety in storage, catalog and backup runs ... Storage and repository: - Listing skipped every name containing ".part-", directories too, so the exports of a user like "john.part-time" were invisible to prune on S3, SFTP, FTP and SMB and their data was deleted. Only real temporary uploads ("<name>.part-<8 hex>") are skipped now. - prune refuses to run while a recorded per-user export's manifest is missing, instead of freeing that export's data. - forget() and untrash() copy first, switch the catalog, and delete the originals last, so a crash can no longer leave the catalog pointing at a missing snapshot or database dump. - Writes are checked against the expected size, so a short upload is an error instead of a truncated pack; raw restores check rename() and touch(). - A catalog write refuses to overwrite a generation another process just wrote, and the catalog lock lasts 300 s, so opening a large repository inside it no longer lets the lease run out. Backup runs and scheduling: - A file that keeps changing is read again at most three times; then the last read is kept and reported, instead of restarting forever and blocking cleanup. - Unreadable files and folders are recorded, logged, shown by occ backup:run, stored in the snapshot and notified to admins, instead of being skipped silently. - After a failed scheduled backup, at most three attempts a day, an hour apart, instead of a new run and notification every five minutes. - The job no longer spins on a run whose lease another process holds, and the weekly cleanup catches up after a Sunday spent backing up. - Notifications for the per-user export retention events.	2026-10-06 23:48:40 +02:00
Dillard Blom	f39ca71dab	Merge pull request #1: Phase 2 ... Phase 2: disaster recovery, per-user backup, verify, FTP/SMB/NFS	2026-10-06 23:21:55 +02:00
dillard	e8ecb77df5	Refresh the location lease while a per-user restore reads its export ... A long import could outlive the lease, letting a cleanup forget the export being restored and free its blobs mid-read.	2026-10-06 22:59:49 +02:00
dillard	70965dd4bd	Keep leases alive during long restores and exports; PR review fixes ... Full restore, raw directory restore, per-user export and the safety export held a fixed-TTL lease on the location that nothing refreshed. A restore or export running longer than that TTL let a cleanup take over and delete data still in use. They now refresh the lease while they work (per blob, per row batch, per written chunk), throttled to once a minute. Also: - backup:prune still applies snapshot retention when user-export retention fails for a location. - The recovery kit carries the deletion delay, so a disaster recovery keeps the original protection instead of falling back to 7 days. Older kits still import with the default. - CI checkouts no longer persist the token in the git config. - recoveryKit() has its real return type; the stale baseline entry is gone.	2026-10-06 22:38:47 +02:00
dillard	cc09d74942	Commit full-backup snapshots to the catalog under the catalog lock ... A full backup opened its repository at the start of a step and wrote the catalog at the end of it, without the catalog lock. A per-user export or restore that wrote the catalog in between could lose its entry, or the snapshot could lose its own. The final step now only stores the snapshot object and moves the run to a new 'commit' phase. BackupService then records the snapshot under the catalog lock, from a repository opened inside that lock; if the lock is busy, the next step retries the commit. Repository::writeSnapshot() keeps its old behaviour as putSnapshot() plus recordSnapshot().	2026-10-06 17:58:21 +02:00
dillard	eb9100a74d	Fix review findings: catalog writes, user restore consent, retention validation ... Catalog: forgetting a user export or purging trash now writes the catalog before deleting the objects, so a crash leaves at most an unreferenced file instead of a catalog entry pointing at nothing. Per-user backup and the safety export reopen the repository inside the catalog lock, so they build on the latest catalog generation. Commands: backup:user:backup and backup:user:restore resolve the location before asking to install user_migration, and a successful install always stops with a rerun instruction. backup:retention rejects a malformed --user-last before changing any policy. backup:key:import fails clearly when the passphrase file cannot be read.	2026-10-06 17:40:51 +02:00
dillard	614caf9f63	Document migrated- calendar names in the how-to; add backlog items	2026-10-06 16:59:45 +02:00
dillard	943c95bf07	Ask before installing user_migration; add the beta how-to ... backup:user:backup and backup:user:restore now ask the administrator before installing or enabling user_migration (or take --install-user-migration), so nothing changes on the server without consent. The restore warns that user_migration imports only its allowlisted app settings. Also: a clear error for unknown locations instead of a raw database message; prune reports the number of affected users, not the number of kept ones; and --user-last rejects values that are not whole numbers. Adds docs/HOWTO.md and updates the README and PLAN.md for the beta.	2026-10-06 16:57:56 +02:00
dillard	ba7f70e0f8	Rename per-user backup/restore commands to backup:user:backup and backup:user:restore	2026-10-06 03:51:35 +02:00
dillard	d70a869070	Phase 2: verify per-user exports, refuse SMB without the native extension, matrix user CI ... backup:verify accepts a user export path (users/<uid>/<export>): every blob its manifest references is checked the same way as for a snapshot, shallow or --deep. The SMB backend is refused at backup:target:add when the native smbclient PECL extension is not loaded, instead of registering and then failing the first backup with "Seek failed": with only the smbclient binary files_external falls back to a shell stream that cannot seek. The per-user integration job now runs on Nextcloud 32-35, not just 35: it calls user_migration's internal UserMigrationService, whose API may change between versions.	2026-10-06 03:47:02 +02:00
dillard	c9f82a79e4	Phase 2: retention for per-user exports (occ backup:retention --user-last) ... PruneService::userExportRetention()/setUserExportRetention(): a single, app-wide "keep the newest N" setting (default 3, same default as the snapshot policy's "last"), deliberately simpler than the full daily/weekly/monthly RetentionPolicy -- per-user exports are on-demand only (occ backup:backup:user, see the fold-out task about whether they ever join the automatic schedule), so there is no regular cadence yet to bucket by day/week/month. PruneService::applyUserExports(): groups a location's user exports by uid (read from each manifest's own 'time' field, the same field snapshot retention already reads), keeps the newest N per user, forgets the rest. No trash, unlike a snapshot: these are not yet the primary safety net the deletion delay protects. Runs under the same exclusive lock as snapshot retention. occ backup:retention gained --user-last=N; occ backup:prune now also applies it for every target it processes, reporting counts the same way it already does for snapshots and blob packs. Verified live against the dev rig (four real exports across four users, confirmed exactly the expected ones were kept/forgotten at two different --user-last values) and added coverage to tests/integration/user-roundtrip.sh (an account's manual export plus the automatic safety-export a replace-mode restore makes of it, pruned down to --user-last=2).	2026-10-06 03:18:14 +02:00
dillard	932f60bc5e	Phase 2: FTP, SMB and NFS storage backends ... No ng_backup code changes needed: files_external's generic backend/auth adapter (the same ExternalStorageFactory/StorageBackend already used for S3/SFTP) already covers FTP and SMB, and NFS was never a files_external backend to begin with (PLAN.md 3.5) -- the "local" backend plus an admin-managed OS-level mount already does the job. This commit is test coverage and documentation. tests/integration/targets-matrix.sh now also runs the full round trip against: - FTP (stilliard/pure-ftpd). A first attempt with delfer/alpine-ftp-server (vsftpd) failed with a generic "create directory operation failed": files_external's FTP backend always builds absolute paths, which that server's chroot rejects even via plain php ftp_mkdir() with an absolute path (a relative path on the same server works fine) -- not a files_external or ng_backup bug, just a server-specific quirk, confirmed by switching test servers. - SMB (dperson/samba), via a small Nextcloud image built on the fly with the smbclient PECL extension (libsmbclient-dev + pecl install), in its own temp directory as the build context (never /tmp directly). The backend *lists* as available with just the smbclient CLI binary too (files_external's shell-wrapped fallback), but that fallback's stream cannot seek, and ng_backup reads blobs back by byte range while backing up -- confirmed live, it fails the very first (small) backup with "Seek failed in packs/...". The native extension is genuinely required here, not just preferred. README: new "Storage backend experiences" entries for both findings (and a pointer from the "no extra PHP extensions" intro line, since SMB is the one real exception), plus NFS guidance (mount it yourself, point the local backend at the mount point). PLAN.md: FTP/SMB marked done; SMB/NFS testing on the user's own OMV instance stays open (internal, not something CI can cover). Local review (Codex + CodeRabbit) found three issues, all fixed: the README's first-pass SMB claim ("CLI tool is not enough") was imprecise about *why* -- it lists fine without the extension, it just doesn't work for ng_backup's read pattern; the cleanup trap could abort before removing the network if an earlier docker rm failed, and didn't remove the built image/Dockerfile; and the Docker build used all of /tmp as its context instead of a dedicated directory.	2026-10-06 00:31:29 +02:00
dillard	935c7b6a08	Phase 2: checksum audit (occ backup:verify) ... Repository::verify(): every blob a snapshot's files and database dump reference must be in a freshly re-downloaded, re-authenticated index (never trusted from the local cache, same reasoning as prune()) and have its pack still present on the location -- shallow, cheap, no blob content read, catches a deleted pack, a tampered/rolled-back index, or a database manifest that went missing (reported as a failure, not silently skipped). With $deep, each blob is also downloaded and decrypted, same as loadBlob() already does on every real read (AEAD authentication tag, content hash against its blob id), done here deliberately for everything in the snapshot rather than waiting for it to come up during an actual restore. A blob referenced by more than one file is only checked once; pack presence is cached per pack, not re-checked per blob. occ backup:verify <target> <snapshot> [--deep], meant to run periodically (cron, a sample of recent snapshots), per PLAN.md's "Verificatie: periodiek checksums controleren, steekproef-restore". The lease covering the audit is refreshed roughly every half of its TTL while blobs are checked, rather than relying on a single long-lived lease like the other new phase-2 commands do. Verified against a real S3-backed 1.4 GiB / 603-blob snapshot (shallow: instant; --deep: ~28s, fully decrypted) in addition to the new unit tests and the roundtrip.sh addition (shallow and deep on a snapshot with real, varied file sizes, now covered on every CI matrix combination). Local review (Codex) on the first pass found three real issues, all fixed: a missing database manifest was silently skipped instead of reported; shallow verification trusted the (possibly stale or unauthenticated) local index cache instead of re-checking the authenticated index on the location; and the lease had no refresh, so a --deep audit of a very large snapshot could outrun its TTL and run unprotected against a concurrent prune.	2026-10-05 23:56:16 +02:00
dillard	f18b4f0862	Phase 2: disaster recovery (occ backup:restore:full) ... New commands, per PLAN.md 3.7: - occ backup:key:import: adopt the master key from a recovery kit on a fresh installation (instead of occ backup:key:init generating a new one), so occ backup:target:add opens the existing repository on the location rather than creating an empty one. - occ backup:restore:full: replace the database (consistent, one transaction), restore the data directory in place, and merge config.php -- refuses up front and lists what's missing when the Nextcloud major version or any app enabled at backup time doesn't match here (their tables must exist before the database restore can use them), rather than guessing or partially applying. Repository::restore() gained an optional $stripPrefix (write "data/<uid>/..." straight into the real data directory, not a "data/" subdirectory of it) -- restoring "data/" and "config/" this way needs no temporary copy of what can be the largest part of an instance. occ backup:target:add now backfills the local snapshot cache (ngb_snapshots, what occ backup:list actually reads) when it adopts an existing repository: without this, a freshly connected location showed no restore points at all, even though the repository itself had them. One unreadable snapshot is skipped rather than aborting the backfill or the target add itself. Three self-referential problems found and fixed by testing this against two real, separate Nextcloud instances sharing only the backup location (tests/integration/full-restore-roundtrip.sh covers this; the per-process nature of these bugs meant they were invisible from inside a single occ invocation, including the one used to write the fixes, which is exactly why a second, fresh process re-running occ backup:status etc. was necessary to catch them): - The full database dump includes ng_backup's own tables and appconfig rows; restoring them verbatim would overwrite the key/target that is actively performing the restore, on THIS installation, with the backed-up instance's. Both are now preserved: ng_backup's own tables (and, on PostgreSQL, their sequences -- excluding a table does not exclude its sequence) are excluded from what gets restored, and its appconfig rows are captured before the database restore and written back after, in their own transaction. - config.php's 'secret' is merged in from the backup (needed so already-encrypted content in the restored database, e.g. share tokens, stays decryptable), but Target::options, the master key, the wrapped key and the recovery kit code are all encrypted with that same instance secret (OCP's ICrypto, or Nextcloud AppConfig's own "sensitive" flag underneath), not with ng_backup's own key. Changing it without re-encrypting them first left all four permanently undecryptable the moment any later occ command ran. Fixed with two helpers (TargetService::reencryptOptionsAround(), KeyService::reencryptForSecretRotation()) that decrypt before and re-encrypt after whatever callback actually changes the secret; config.php is applied dead last in restoreFull(), after every other repository access. Not crash-safe (a process kill mid-rotation would leave a key missing, not just stale) -- disaster recovery is run once by hand, right after occ maintenance:install, a window too narrow yet to justify a two-phase write. - A subtler variant of the same bug: Nextcloud's own AppConfig additionally encrypts a "sensitive" value itself, tied to the same secret, underneath our own encryption. Overwriting an existing sensitive key makes it read the old value first to compare, which then also fails against the new secret. Deleting the key before setting the new value avoids that read entirely. Roadmap (PLAN.md 9): phase 3 (in-place full restore, user self-service) moved out to v1.3.0; phase 2 continues toward v1.2.0 with FTP/SMB, a verification/checksum-audit command, user-export retention, and deciding whether per-user export joins the automatic schedule.	2026-10-05 23:31:08 +02:00
dillard	6cbc5d79d4	Phase 2: per-user backup and restore via user_migration ... Wires the existing RepositoryExportDestination/RepositoryImportSource (see lib/UserMigration/) into user_migration's own UserMigrationService, resolved on demand through a new UserMigrationServiceLocator since the app is an optional dependency (PLAN.md 3.8) -- without it, only this feature is unavailable, the full backup still works. New: UserBackupService::backupUser() (occ backup:backup:user) streams a user_migration export straight into the repository, under the same shared location lock as a regular backup. UserRestoreService::restoreUser() (occ backup:restore:user) never imports over a live account: --mode=replace safety-exports the current state first, then deletes and recreates the account from the backup under the same uid; --mode=as-backup restores into a new "<uid>-bak" account instead, since Nextcloud cannot rename a uid. The catalog now also tracks user export manifests (Repository::recordUserExport/ listUserExports/forgetUserExport/isRecordedUserExport), rollback-protected like a snapshot: listUserExports() verifies each one is still present on the location, and restore only ever reads a manifest the verified catalog actually recorded, never an arbitrary caller-supplied path. prune()'s existing raw users/ scan for blob retention is unchanged. A new short exclusive "catalog" lock (PruneService::catalogLockKey()) serializes the catalog write itself, so two concurrent per-user backups (or one alongside a full backup) can no longer race to write the same next generation and silently drop one of them -- the broader per-target shared lock only ever protected against a concurrent cleanup, not against other shared holders. Two real bugs found by testing this against a live Nextcloud instance, not by unit tests (RepositoryExportDestination/RepositoryImportSource implement OCP interfaces that plain PHPUnit can't load, see tests/Unit/Repository/UserExportTest.php's docblock; the new tests/integration/user-roundtrip.sh covers the real round trip instead): - RepositoryImportSource::copyToFolder() double-closed the stream it handed to putContent()/newFile() (both already close it themselves), which throws a TypeError on PHP 8 and broke every restored file with content. - A replace-mode restore failed on NotPermittedException: IRootFolder::getUserFolder() caches a Folder per uid for the process lifetime, so after delete+recreate of the same uid the import still got the deleted account's (now storage-less) folder. Fixed by dispatching FilesystemTornDownEvent, which Root already listens for to clear exactly that cache -- no internal OC\ classes needed. Local review (CodeRabbit + Codex) against this diff found three more real issues, all fixed: the catalog write race above, a failed as-backup import leaving a broken stub account behind (now deleted on failure), and an export stream leak on a read/backend failure in RepositoryExportDestination::copyFolder() (now closed in a finally). Deliberately left out: wiring per-user export into the automatic backup schedule (on-demand via occ only, for now) and retention/pruning of old user exports -- both open design questions, not yet decided. The lease covering a single export/import still has a fixed TTL (3600s) with no mid-call refresh, since UserMigrationService exposes no progress; a resumable, step-based per-user backup (like the main BackupService run) would remove that cap but is a bigger change than this pass.	2026-10-05 17:36:46 +02:00
dillard	58711b67ef	CHANGELOG: add the release link reference	2026-10-05 12:17:10 +02:00
dillard	e7229626f8	Prepare the 0.1.0-alpha.1 release ... info.xml's version field is restricted by Nextcloud's own info.xsd to plain numeric SemVer, no pre-release suffix, so the alpha/beta/rc stage from PLAN.md's release table lives in the git tag and CHANGELOG instead; noted that in PLAN.md so it isn't rediscovered later. Bumped the version, wrote an honest description for an alpha (feature-complete for phase 1, not for production), and added CHANGELOG.md. README: new "Storage backend experiences" section, seeded with the Hetzner Storage Box sub-account root-path quirk found while testing a real SFTP target (root must be "home", not "/"), open for others to extend via PR. v0.1.0-alpha.1	2026-10-05 11:45:48 +02:00
dillard	807353cbe4	Wire Psalm into CI, fix a real bug it found in user-migration import ... RepositoryImportSource::copyToFolder() called putContent() on the generic return type of Node::get(), which doesn't declare that method -- only File does. Existing-file overwrite during a per-user restore would have fatally errored at runtime; added an instanceof check and a resource leak fix (finally-close) while there. Also guarded getFileAsStream() against fopen() returning false. Everything else Psalm (strict level) currently flags is mechanical (missing `final` on leaf classes -- applied everywhere safe) or pre-existing array-shape/type gaps, captured in psalm-baseline.xml so CI fails only on new issues from here on, not the whole backlog at once.	2026-10-05 11:45:37 +02:00
dillard	48ea6bb33d	CI: round trip against real S3 and SFTP backends, not just a local path ... Added tests/integration/targets-matrix.sh, which runs the full backup/restore round trip through files_external's AmazonS3 and SFTP backend classes (S3 via a disposable s3mock container, SFTP via atmoz/sftp), since the existing matrix only ever exercised the local backend. This caught a real bug: moving a snapshot to the trash read it from the location and streamed it straight back into a write on the same connection, which SFTP can't do reliably (phpseclib fails to interleave a read and a write on one connection). Backends now expose a native move() and the id-bound snapshot object is moved with that instead of a read+write copy; the path-bound database dump still goes through decrypt/re-encrypt, since its AEAD authentication is tied to its path.	2026-10-05 03:15:34 +02:00
dillard	9d97f9c1f1	Admin UI: locations under Settings, fix the missing restore-point date ... Move backup locations onto the Settings tab next to the key setup, since adding one is a one-time step like the key, not daily use. The "Backup and restore" tab points there when no location exists yet. The restore-point dropdown relied on NcSelect's get-option-label, which it does not render for the selected value, so no date showed. Precompute a displayLabel per snapshot and select on that instead, the same way the location dropdown already does. Signed-off-by: Dillard Blom <dillard.blom@ensembia.com>	2026-10-04 11:15:45 +02:00
dillard	b6ba09b078	Admin UI: restore point dates and a settings/daily-use split ... Show the date of the restore point a restore run used, both in the backup history table and when confirming a restore. Split the admin page into a "Backup and restore" tab (locations, backups, restore) and a "Settings" tab (key, slots, recovery kit), so daily work no longer sits behind the key setup at the top of the page. Signed-off-by: Dillard Blom <dillard.blom@ensembia.com>	2026-10-02 17:11:34 +02:00
dillard	85e835bfc2	Storage limit: 80% warning and limit in the web interface ... - after each successful backup the stored size is recorded; at 80% of a location's limit administrators get a notification (once a day) - locations list shows stored size and limit; the add form has a 'maximum size' field that explains that empty means unlimited and billed by the provider; PUT /api/targets/{id}/limit (password confirmation) Tested on NC34 with the OVH test bucket: a backup above the limit stops, a successful one at 81% sends the warning.	2026-10-02 14:50:16 +02:00
dillard	2610c1ad2f	Expiring leases instead of long-held locks ... A backup step killed by a time limit, OOM or kill -9 kept Nextcloud's lock on its location for up to an hour, blocking every new start and the run itself. Locations and runs are now guarded by leases in ngb_leases with an expiry: a backup step holds them for its duration plus two minutes, restores refresh theirs every minute, prune holds 30 minutes, untrash 5. Nextcloud's lock is only used for the milliseconds in which a lease is checked and written. Also: starting a backup while one is already running simply continues it (no exclusive lease needed), and running to completion waits instead of spinning while a crashed step's lease has not expired. Tested on NC34: a backup killed four times at random moments resumes on its own after the lease expires (about two minutes), and restores bit-identically.	2026-10-02 14:44:52 +02:00
dillard	532b23482e	Storage limit per location ... S3 has no bucket quota, so NG Backup enforces one: before a pack is uploaded, the stored size plus the pack must stay under the location's limit, otherwise the backup stops with an error (and administrators are notified through the failed run). Repacking during prune is exempt, as it is how space is freed. occ backup:target:limit NAME [SIZE|none]. Also ignore credential files in the repository.	2026-10-02 14:23:25 +02:00
dillard	066873a818	Plan: release stages and version numbers	2026-10-02 13:16:54 +02:00
dillard	c54b2cebed	M5: end-to-end test matrix and CI ... - tests/integration/roundtrip.sh: key, kit code and confirmation, location, full and unchanged incremental backup, restore new-folder and replace (files identical, extra file in the trash), database dump and restore, retention, trash and restore from trash, no files_external errors in the log - tests/integration/matrix.sh: fresh Nextcloud per version and database (SQLite, MariaDB, PostgreSQL), removes only its own containers and their volumes; locally green on NC32 and NC35 with all three databases - GitHub Actions: unit tests on PHP 8.2-8.4, round trip on NC32-35 x SQLite/MariaDB/PostgreSQL Fixes found by the matrix: - files_external is no longer booted when it is disabled: only its classes are loaded and a private BackendService gets its backends from its Application class (booting registered its mount provider, which failed on installations where its tables were never created) - SQLite: the database file is excluded from the file backup (it is covered by the logical dump and copying the live file is inconsistent) - occ backup:key:confirm accepts --code and --confirm for scripts	2026-10-02 11:56:57 +02:00
dillard	48bb829d45	M5: resumable database dump and faster database restore ... - DbDumper::dumpStep(): tables are dumped until the step's deadline in one REPEATABLE READ transaction and resumed at the last primary key in the next step; the running table checksum is carried over. A dump that fits in one step is fully consistent, otherwise the snapshot records dbConsistent=false. BackupService uses it, so the database phase no longer needs one long request (hosted Nextcloud, web cron) - restore inserts 100 rows per statement and leaves tables whose checksum already matches the dump untouched - spike C: 30-step dump gives the same checksums as the one-shot dump; restore rewrote only the 3 damaged tables of 147; 300k-row table restored in 14.6 s instead of 46 s	2026-10-02 11:34:59 +02:00