diff --git a/CHANGELOG.md b/CHANGELOG.md index e8e3938..34f51d0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,25 @@ the app store accepts a `-beta` suffix, but Nextcloud servers then only offer th beta/daily update channel, which would hide the app from most admins. The stability stage is instead carried by the app description, the git tag, the release, and this file. +## [0.9.2-beta.3] - 2026-10-08 + +Bugfix release. + +### Added +- README: notes on using a Hetzner Storage Box over WebDAV and SMB. + +### Fixed +- Adding a backup location in the web interface failed with "Request failed with status code 403" + when the login was more than 30 minutes old: the form did not ask for password confirmation. +- Settings page: the "Backup and restore" / "Settings" tabs line up with the sections below them. +- Backups: the runs table scrolls horizontally on narrow screens instead of running off the page. +- `occ backup:verify --deep`: a blob that could not be read (network or storage error) was reported + as "corrupted or tampered". Reads are now retried, and a read that keeps failing is reported + separately as unreadable, with the error (run the check again; if the same blobs stay unreadable, + their pack may be damaged or truncated). Only a blob that was read but fails decryption or its + hash check counts as corrupted. Found on a Hetzner Storage Box over WebDAV, where the blob + turned out to be intact. + ## [0.9.1-beta.2] - 2026-10-07 Second beta, the first release in the Nextcloud app store (signed). @@ -103,6 +122,7 @@ and not implemented yet. - See the "Storage backend experiences" section of the README for provider-specific quirks found so far (e.g. Hetzner Storage Box sub-accounts). +[0.9.2-beta.3]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.2-beta.3 [0.9.1-beta.2]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.1-beta.2 [0.9.0-beta.1]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.0-beta.1 [0.1.0-alpha.1]: https://github.com/dillardblom/ng_backup/releases/tag/v0.1.0-alpha.1 diff --git a/README.md b/README.md index 5984046..d7691f3 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ installations and hosted Nextcloud. (Choosing SMB as a storage backend is the on see "Storage backend experiences" below — that requirement comes from files_external, not from ng_backup, and applies to any Nextcloud app that writes to an SMB share.) -**Status: beta (0.9.1).** Tested thoroughly, usable alongside your current backup, but not yet a +**Status: beta (0.9.2).** Tested thoroughly, usable alongside your current backup, but not yet a drop-in replacement for it. See "Known limitations" below, `PLAN.md` for the design and `docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. @@ -126,6 +126,20 @@ Contributions welcome via PR — add what you ran into with your own provider. verified whether this also applies to a Hetzner Storage Box **main** account (only tested with a sub-account so far) — if you've tried a main account, please say so in a PR. +- **Hetzner Storage Box over WebDAV and SMB (sub-accounts, 2026-10-07):** both work end to end + (backup, incremental run, restore of a folder, verify), tested from outside the Hetzner network + with about 66 MiB of data. In the Hetzner Console, enable WebDAV or SMB support for the + sub-account, and "External reachability" when your server is not in the Hetzner network. + WebDAV: host `https://uXXXXX-subN.your-storagebox.de`, and the `root` + must be a folder that already exists (or empty): WebDAV does not create missing parent folders, + so a root that doesn't exist yet fails `backup:target:add` with `Sabre\HTTP\ClientHttpException: + Conflict`. SMB: share = the sub-account name, port 445 must be reachable from your server (it was + from a home connection here, but some providers block it), and the `smbclient` PECL extension is + required (see the SMB entry below). Both have a noticeable fixed cost per run: an incremental run + that uploads nothing took about 20 s over WebDAV and 50 s over SMB. A deep verify is slow over + remote storage, since each blob is fetched with its own request (19 minutes for 66 MiB over + WebDAV); run it occasionally, not after every backup. + - **FTP, server-dependent absolute-path handling:** files_external's FTP backend always builds absolute paths (e.g. `MKD /repo`) for the configured root, never relative ones. `atmoz/sftp`-style `vsftpd` test servers (chrooted) can reject that with a generic "create directory operation diff --git a/appinfo/info.xml b/appinfo/info.xml index fce3237..14e41ff 100644 --- a/appinfo/info.xml +++ b/appinfo/info.xml @@ -9,7 +9,7 @@ directory to S3, SFTP, WebDAV, FTP, SMB or a local path, with restore of files a single users (via user_migration) and of the whole server onto a fresh installation. Tested thoroughly and usable alongside your current backup, but not yet a drop-in replacement for it. See the README for known limitations and the recovery kit before relying on it.]]> - 0.9.1 + 0.9.2 agpl Dillard Blom NgBackup diff --git a/lib/Command/Verify.php b/lib/Command/Verify.php index a9d1a7c..37997ab 100644 --- a/lib/Command/Verify.php +++ b/lib/Command/Verify.php @@ -66,7 +66,13 @@ HELP); if ($result['failed'] !== []) { $output->writeln('' . count($result['failed']) . ' blob(s) failed verification (corrupted or tampered): ' . implode(', ', $result['failed'])); } - if ($result['missing'] === [] && $result['failed'] === []) { + if ($result['unreadable'] !== []) { + $output->writeln('' . count($result['unreadable']) . ' blob(s) could not be read from the location (often a temporary storage ' + . 'or network error: run the check again; if the same blobs stay unreadable, their pack may be damaged or truncated): ' + . implode(', ', $result['unreadable'])); + $output->writeln('Last read error: ' . $result['readError']); + } + if ($result['missing'] === [] && $result['failed'] === [] && $result['unreadable'] === []) { $output->writeln('OK'); return 0; } diff --git a/lib/Repository/Repository.php b/lib/Repository/Repository.php index b312d30..9a383ae 100644 --- a/lib/Repository/Repository.php +++ b/lib/Repository/Repository.php @@ -28,6 +28,8 @@ final class Repository { public const FORMAT = 1; public const BLOB_SIZE = 4 * 1048576; private const FLAG_DEFLATE = 1; + /** Reads per blob in a deep verify before it is reported as unreadable (not corrupted). */ + private const VERIFY_READ_ATTEMPTS = 3; private StreamCipher $cipher; private BlobIndex $index; @@ -696,8 +698,13 @@ final class Repository { } private function loadBlob(string $id): string { - [$pack, $offset, $length, $raw, $flags] = $this->index->get($id); - $encrypted = $this->backend->getRange('packs/' . substr($pack, 0, 2) . '/' . $pack, $offset, $length); + [$pack, $offset, $length] = $this->index->get($id); + return $this->decodeBlob($id, $this->backend->getRange('packs/' . substr($pack, 0, 2) . '/' . $pack, $offset, $length)); + } + + /** Decrypts and checks a blob read from its pack; throws when it is corrupted or tampered with. */ + private function decodeBlob(string $id, string $encrypted): string { + [, , , $raw, $flags] = $this->index->get($id); $payload = $this->cipher->decryptString($encrypted, 'blob:' . $id); $data = ($flags & self::FLAG_DEFLATE) ? gzinflate($payload) : $payload; if ($data === false || strlen($data) !== $raw || $this->keys->blobId($data) !== $id) { @@ -733,13 +740,13 @@ final class Repository { * * @param ?callable $heartbeat called after every blob (shallow) or every blob read (deep); * callers use it to keep a time-boxed lease alive during a large audit - * @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list} + * @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list, unreadable:list, readError:string} */ public function verify(string $snapshotId, bool $deep = false, ?callable $heartbeat = null): array { $heartbeat ??= static function (): void { }; $this->index->load(true); - $stats = ['filesChecked' => 0, 'blobsChecked' => 0, 'bytesChecked' => 0, 'missing' => [], 'failed' => []]; + $stats = ['filesChecked' => 0, 'blobsChecked' => 0, 'bytesChecked' => 0, 'missing' => [], 'failed' => [], 'unreadable' => [], 'readError' => '']; $seen = []; $packSeen = []; // pack id => exists on the backend; avoids one exists() call per blob foreach ($this->streamEntries($snapshotId) as $entry) { @@ -778,7 +785,7 @@ final class Repository { * Same checks as verify(), for a user export: every blob its manifest references must be in * the freshly re-authenticated index with its pack present; $deep also decrypts each. * - * @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list} + * @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list, unreadable:list, readError:string} */ public function verifyUserExport(string $manifestPath, bool $deep = false, ?callable $heartbeat = null): array { if (!$this->isRecordedUserExport($manifestPath)) { @@ -787,7 +794,7 @@ final class Repository { $heartbeat ??= static function (): void { }; $this->index->load(true); - $stats = ['filesChecked' => 0, 'blobsChecked' => 0, 'bytesChecked' => 0, 'missing' => [], 'failed' => []]; + $stats = ['filesChecked' => 0, 'blobsChecked' => 0, 'bytesChecked' => 0, 'missing' => [], 'failed' => [], 'unreadable' => [], 'readError' => '']; $seen = []; $packSeen = []; $manifest = json_decode($this->getObject($manifestPath), true, 512, JSON_THROW_ON_ERROR); @@ -824,8 +831,25 @@ final class Repository { if (!$deep) { return; } + // Reading and checking are kept apart: a read that fails is usually a network or + // storage error, so it is retried and reported as unreadable (a pack that stays + // unreadable may still be truncated). Only a blob that was read but does not + // decrypt or match its id counts as failed: that is proof of damage. + [, $offset, $length] = $this->index->get($id); + $encrypted = null; + for ($attempt = 1; $attempt <= self::VERIFY_READ_ATTEMPTS && $encrypted === null; $attempt++) { + try { + $encrypted = $this->backend->getRange('packs/' . substr($pack, 0, 2) . '/' . $pack, $offset, $length); + } catch (\Exception $e) { // storage adapters throw their own exceptions; \Error (a bug) is not retried + $stats['readError'] = $e->getMessage(); + } + } + if ($encrypted === null) { + $stats['unreadable'][] = $id; + return; + } try { - $stats['bytesChecked'] += strlen($this->loadBlob($id)); + $stats['bytesChecked'] += strlen($this->decodeBlob($id, $encrypted)); } catch (\Throwable) { $stats['failed'][] = $id; } diff --git a/lib/Service/VerifyService.php b/lib/Service/VerifyService.php index bef188e..fcfd68e 100644 --- a/lib/Service/VerifyService.php +++ b/lib/Service/VerifyService.php @@ -18,12 +18,12 @@ final class VerifyService { ) { } - /** @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list} */ + /** @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list, unreadable:list, readError:string} */ public function verify(Target $target, string $snapshotId, bool $deep = false): array { return $this->run($target, $deep, fn (\OCA\NgBackup\Repository\Repository $repo, callable $heartbeat): array => $repo->verify($snapshotId, $deep, $heartbeat)); } - /** @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list} */ + /** @return array{filesChecked:int, blobsChecked:int, bytesChecked:int, missing:list, failed:list, unreadable:list, readError:string} */ public function verifyUserExport(Target $target, string $manifestPath, bool $deep = false): array { return $this->run($target, $deep, fn (\OCA\NgBackup\Repository\Repository $repo, callable $heartbeat): array => $repo->verifyUserExport($manifestPath, $deep, $heartbeat)); } diff --git a/src/AdminSettings.vue b/src/AdminSettings.vue index 9f0b458..c66dcf7 100644 --- a/src/AdminSettings.vue +++ b/src/AdminSettings.vue @@ -82,6 +82,7 @@ onBeforeUnmount(() => clearTimeout(timer)) .ngb-tabs { display: flex; gap: 8px; - margin: 0 0 12px; + /* Same inline margin as NcSettingsSection, so the tabs line up with the section content */ + margin: 0 calc(var(--default-grid-baseline) * 7) 12px; } diff --git a/src/components/Backups.vue b/src/components/Backups.vue index 9baf14f..3a09a28 100644 --- a/src/components/Backups.vue +++ b/src/components/Backups.vue @@ -30,30 +30,32 @@ - - - - - - - - - - - - - - - - - - - - - - - -
{{ t('ng_backup', 'Started') }}{{ t('ng_backup', 'Location') }}{{ t('ng_backup', 'Kind') }}{{ t('ng_backup', 'Restore point') }}{{ t('ng_backup', 'Status') }}{{ t('ng_backup', 'Files') }}{{ t('ng_backup', 'Uploaded') }}
{{ new Date(run.startedAt * 1000).toLocaleString() }}{{ targetName(run.targetId) }}{{ run.kind }}{{ run.snapshot ? new Date(run.snapshot.createdAt * 1000).toLocaleString() + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}{{ run.status }}{{ run.status === 'running' ? ' (' + run.phase + ')' : '' }}{{ run.error ? ': ' + run.error : '' }}{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}
+
+ + + + + + + + + + + + + + + + + + + + + + + +
{{ t('ng_backup', 'Started') }}{{ t('ng_backup', 'Location') }}{{ t('ng_backup', 'Kind') }}{{ t('ng_backup', 'Restore point') }}{{ t('ng_backup', 'Status') }}{{ t('ng_backup', 'Files') }}{{ t('ng_backup', 'Uploaded') }}
{{ new Date(run.startedAt * 1000).toLocaleString() }}{{ targetName(run.targetId) }}{{ run.kind }}{{ run.snapshot ? new Date(run.snapshot.createdAt * 1000).toLocaleString() + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}{{ run.status }}{{ run.status === 'running' ? ' (' + run.phase + ')' : '' }}{{ run.error ? ': ' + run.error : '' }}{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}
+
@@ -126,6 +128,7 @@ async function saveSchedule() {