Compare commits

...

5 Commits

Author SHA1 Message Date
dillard a8b5e85c9b All folders get their original modification time back on a restore
The backup records the time of the folders with something in them, packed in the meta record,
so an older version still reads the snapshot. A restore through Nextcloud leaves a folder that
was already there alone. The round trip checks what Nextcloud shows, not only the disk.
2026-11-07 01:07:09 +01:00
dillard 36f2427e9a Tell administrators about a location without a schedule
A notification when a schedule is turned off, then a reminder: once a week while another
location is backed up, every day when none is. The round trip now also checks the text of
these notifications and of the one about names, and that one location never gets two runs.
2026-11-07 00:23:28 +01:00
dillard e58b86fc48 A schedule per location, and locations backed up one after the other
Off, once a day or every N hours from a time, each in its own time zone. Existing locations
keep the time that held for all of them. A backup that has to wait shows as queued, and its
restore point has the time it really began.
2026-11-07 00:04:00 +01:00
dillard 193f65940b Pin a restore point and keep a note on it
Both live in the catalog on the location, so a new server sees them; the database holds
a local copy for the lists. The retention policy leaves a pinned restore point alone and
does not count it. Available on the restore page and as occ backup:pin and backup:note.
2026-11-06 23:49:50 +01:00
dillard 6540a9314a A name a snapshot cannot hold gets a notification of its own
Reported apart from unreadable files, with example paths and the advice to rename
them, instead of advice about permissions. Start of the 1.1.0 plan in PLAN.md.
2026-11-06 23:38:40 +01:00
45 changed files with 1349 additions and 125 deletions
+26
View File
@@ -11,6 +11,32 @@ Each heading starts with that `<version>`, once: the app store shows a release t
whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
shown under 1.0.0. The git tag of a release candidate goes after the date. shown under 1.0.0. The git tag of a release candidate goes after the date.
## [Unreleased]
### Added
- Pin a restore point, on the restore page or with `occ backup:pin`: the retention policy leaves it
alone, for example the one from before an upgrade. Unpinning notifies all administrators.
- A note on a restore point that you can change afterwards, on the restore page or with
`occ backup:note`. Pins and notes are stored encrypted on the location, so a new server sees them.
An older version of NG Backup can still read the location, but drops them when it writes to it.
- A schedule per location: off, once a day, or every N hours from a given time, on the page under
Backups or with `occ backup:target:schedule`. After the update every location keeps the time that
held for all of them; a new location starts with the schedule of the first one.
- All folders get their original modification time back on a restore, not only the empty ones, from
the first backup made with this version. An older version of NG Backup can still read new restore
points. A restore through Nextcloud into a folder that is already there leaves its time alone.
### Changed
- Locations are backed up one after the other: a backup that has to wait shows as queued. A restore
point has the time its backup really began, not the time it was queued.
- Choosing a time no longer starts a backup right away when that time has already passed today.
- A location without a schedule no longer gets the notification that it has had no recent backup.
Instead, all administrators are told when a schedule is turned off, and reminded once a week while
another location is still backed up, or every day when none is.
- A file or folder that is left out because of its name now gets a notification of its own, in the
log and in `occ backup:run`. It says the item is missing from the backup, gives example paths and
asks to rename it; before, it was reported as unreadable, with advice about permissions.
## [1.0.3] - 2026-11-06 ## [1.0.3] - 2026-11-06
### Fixed ### Fixed
+18 -5
View File
@@ -472,9 +472,12 @@ in README "Known limitations":
tijd terug. Hoort zo, vooral bij oudere verzamelingen: de maptijd wordt gebruikt om in te schatten tijd terug. Hoort zo, vooral bij oudere verzamelingen: de maptijd wordt gebruikt om in te schatten
wanneer een map is aangemaakt. De backup moet dan de tijd van alle mappen vastleggen en de restore wanneer een map is aangemaakt. De backup moet dan de tijd van alle mappen vastleggen en de restore
die aan het eind terugzetten; werkt pas vanaf de eerste backup met die versie. Op de roadmap onder 1.1.0. die aan het eind terugzetten; werkt pas vanaf de eerste backup met die versie. Op de roadmap onder 1.1.0.
**1.0.3 uitgebracht (2026-11-06)**, ook op nc01 geïnstalleerd.
- **Branches (besluit 2026-11-06):** werk voor een versie op een branch met die naam (`1.0.3`); - **Branches (besluit 2026-11-06):** werk voor een versie op een branch met die naam (`1.0.3`);
`polish` alleen voor kleine cosmetische zaken, per geval bekijken of en wanneer die gemerged `polish` alleen voor kleine cosmetische zaken, per geval bekijken of en wanneer die gemerged
worden. SMB staat uit op sub5 en sub6; sub6 is het testaccount, eerst aanzetten voor een SMB-test. worden. SMB staat uit op sub5 en sub6; sub6 is het testaccount, eerst aanzetten voor een SMB-test.
Na 1.0.3 gaat het werk naar `1.1.0`. Een spoedfix die niet kan wachten gaat op `1.0.4` vanaf `main`,
krijgt een eigen merge en release; daarna `main` in `1.1.0` mergen, zodat daar met de fix getest wordt.
- **1.0.1** (uitgebracht 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar - **1.0.1** (uitgebracht 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november, 1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november,
niet aangekondigd en los van de release. Voor 1.0.2 ook: nieuwe screenshots (het labelveld bij niet aangekondigd en los van de release. Voor 1.0.2 ook: nieuwe screenshots (het labelveld bij
@@ -569,7 +572,18 @@ in README "Known limitations":
- **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3, - **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3,
4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan 4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan
een paar bestanden. een paar bestanden.
- **1.1.0:** - **1.1.0** (geen harde datum: een week was een voorzet, langer mag; de releasefrequentie mag omlaag,
besluit 2026-11-06):
**Betere melding voor namen die een snapshot niet kan bevatten** (backslash, stuurteken, geen UTF-8):
een eigen melding in plaats van "unreadable" met het advies over rechten. Zegt dat deze bestanden of
mappen in de backup ontbreken, geeft voorbeelden van de paden en vraagt ze te hernoemen; de volgende
backup neemt ze mee. Alleen als Nextcloud-melding: de app mailt niet zelf, of meldingen gemaild worden
is de keuze van de beheerder (besluit 2026-11-06).
**Stand op branch `1.1.0` (2026-11-07):** klaar: betere melding, punten 4, 5, 6 en 8. Pin en notitie staan
in de catalogus, met een kopie in de database voor de lijsten. Een moment van vóór het instellen van
een schema start niets. Geen schema: geen melding "geen recente backup", wel een melding aan alle
beheerders bij het uitzetten en een herinnering, wekelijks zolang een andere locatie een schema heeft,
anders dagelijks (besluit 2026-11-07). Nog niet getest: de update vanaf 1.0.3, de pagina in een browser.
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit 1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
`target:add` en sleutelimport (het gevoeligste pad), daarom niet in rc1. Daarna de `target:add` en sleutelimport (het gevoeligste pad), daarom niet in rc1. Daarna de
@@ -601,11 +615,10 @@ in README "Known limitations":
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de `occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly. opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like 9. **Feedbackvraag weghalen in 1.2.0** (besluit 2026-11-07, niet in 1.1.0): de regel "We would like
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die dan staan. Er komt weinig feedback bij zo'n 150 downloads per versie; geen bericht is goed
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan bericht. Het formulier (`experience.yml`) in de issues blijft.
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met - **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
PostgreSQL 18)** — voor **1.0.0-rc.2**: PostgreSQL 18)** — voor **1.0.0-rc.2**:
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE, 1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
+5 -2
View File
@@ -293,8 +293,9 @@ On the backlog; know them before you rely on NG Backup:
- **Empty folders only come back from backups made with 1.0.3 or newer.** Older restore points do - **Empty folders only come back from backups made with 1.0.3 or newer.** Older restore points do
not contain them. not contain them.
- **Folders with files in them show the time of the restore**, not their original modification time. - **Folders with files in them only get their original modification time back from backups made with
The files keep theirs. On the roadmap. 1.1.0 or newer.** From older restore points they show the time of the restore; the files keep theirs.
A restore through Nextcloud into a folder that is already there leaves that folder's time alone.
- **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in - **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
`config.php`) is not supported: a backup refuses to start there. Folders mounted through the `config.php`) is not supported: a backup refuses to start there. Folders mounted through the
External storage app are not in the data directory and are **not included**, without a warning. External storage app are not in the data directory and are **not included**, without a warning.
@@ -329,10 +330,12 @@ On the backlog; know them before you rely on NG Backup:
| `backup:key:init`, `backup:key:kit`, `backup:key:confirm` | create the key, recovery kit, confirmation | | `backup:key:init`, `backup:key:kit`, `backup:key:confirm` | create the key, recovery kit, confirmation |
| `backup:key:slots`, `backup:key:slot:add/replace/remove` | passphrase slots | | `backup:key:slots`, `backup:key:slot:add/replace/remove` | passphrase slots |
| `backup:target:backends/add/edit/list/remove/append-only` | backup locations | | `backup:target:backends/add/edit/list/remove/append-only` | backup locations |
| `backup:target:schedule` | when a location is backed up: off, daily, or every N hours |
| `backup:run`, `backup:status`, `backup:list` | back up, status (runs of the last 7 days; `--all` for all), restore points | | `backup:run`, `backup:status`, `backup:list` | back up, status (runs of the last 7 days; `--all` for all), restore points |
| `backup:cancel <run>` | stop a running backup | | `backup:cancel <run>` | stop a running backup |
| `backup:browse`, `backup:restore` | browse and restore | | `backup:browse`, `backup:restore` | browse and restore |
| `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash | | `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash |
| `backup:pin`, `backup:note` | keep a restore point, a note you can change later |
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) | | `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) | | `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
| `backup:key:import` | import a recovery kit on a fresh installation | | `backup:key:import` | import a recovery kit on a fresh installation |
+3
View File
@@ -83,11 +83,14 @@ you need help with restoring a backup after a crash.]]></description>
<command>OCA\NgBackup\Command\ListSnapshots</command> <command>OCA\NgBackup\Command\ListSnapshots</command>
<command>OCA\NgBackup\Command\Browse</command> <command>OCA\NgBackup\Command\Browse</command>
<command>OCA\NgBackup\Command\Restore</command> <command>OCA\NgBackup\Command\Restore</command>
<command>OCA\NgBackup\Command\Pin</command>
<command>OCA\NgBackup\Command\Note</command>
<command>OCA\NgBackup\Command\Prune</command> <command>OCA\NgBackup\Command\Prune</command>
<command>OCA\NgBackup\Command\Retention</command> <command>OCA\NgBackup\Command\Retention</command>
<command>OCA\NgBackup\Command\TargetAppendOnly</command> <command>OCA\NgBackup\Command\TargetAppendOnly</command>
<command>OCA\NgBackup\Command\TargetTrustCurrent</command> <command>OCA\NgBackup\Command\TargetTrustCurrent</command>
<command>OCA\NgBackup\Command\TargetLimit</command> <command>OCA\NgBackup\Command\TargetLimit</command>
<command>OCA\NgBackup\Command\TargetSchedule</command>
<command>OCA\NgBackup\Command\Trash</command> <command>OCA\NgBackup\Command\Trash</command>
<command>OCA\NgBackup\Command\TrashRestore</command> <command>OCA\NgBackup\Command\TrashRestore</command>
<command>OCA\NgBackup\Command\KeySlots</command> <command>OCA\NgBackup\Command\KeySlots</command>
+3 -1
View File
@@ -25,8 +25,10 @@ return [
['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'], ['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'],
['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'], ['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'],
['name' => 'AdminApi#tick', 'url' => '/api/tick', 'verb' => 'POST'], ['name' => 'AdminApi#tick', 'url' => '/api/tick', 'verb' => 'POST'],
['name' => 'AdminApi#setSchedule', 'url' => '/api/schedule', 'verb' => 'PUT'], ['name' => 'AdminApi#setTargetSchedule', 'url' => '/api/targets/{id}/schedule', 'verb' => 'PUT'],
['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'], ['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'],
['name' => 'AdminApi#setSnapshotNote', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/note', 'verb' => 'PUT'],
['name' => 'AdminApi#setSnapshotPinned', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/pin', 'verb' => 'PUT'],
['name' => 'AdminApi#browse', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/browse', 'verb' => 'GET'], ['name' => 'AdminApi#browse', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/browse', 'verb' => 'GET'],
['name' => 'AdminApi#startRestore', 'url' => '/api/restores', 'verb' => 'POST'], ['name' => 'AdminApi#startRestore', 'url' => '/api/restores', 'verb' => 'POST'],
], ],
+8 -1
View File
@@ -79,7 +79,9 @@ occ backup:list
Later runs upload only changed data. An unchanged run uploads (almost) nothing; the output shows Later runs upload only changed data. An unchanged run uploads (almost) nothing; the output shows
the number of uploaded bytes, so check that value when you test incremental behaviour. the number of uploaded bytes, so check that value when you test incremental behaviour.
Schedule `occ backup:run` from system cron (or use the Administration settings, if you prefer). Set when each location is backed up in the Administration settings, or with
`occ backup:target:schedule offsite 02:00` (once a day), `02:00/6` (every 6 hours from 02:00) or `off`.
Locations are backed up one after the other.
## 4. Check the backup regularly ## 4. Check the backup regularly
@@ -173,8 +175,13 @@ occ backup:prune offsite --dry-run # show first
occ backup:prune offsite # apply the policy; forgotten snapshots go to the trash first occ backup:prune offsite # apply the policy; forgotten snapshots go to the trash first
occ backup:trash # list the trash occ backup:trash # list the trash
occ backup:trash:restore offsite <id> occ backup:trash:restore offsite <id>
occ backup:pin offsite <id> # keep this restore point; --unpin to undo
occ backup:note offsite <id> "text" # a note you can change later; without text it is removed
``` ```
The retention policy leaves a pinned restore point alone, and a pinned one does not take the place of
a daily, weekly or monthly one. Pin and note are stored on the location, so a new server sees them too.
Snapshots are kept according to the policy, and their data is deleted only after the deletion delay Snapshots are kept according to the policy, and their data is deleted only after the deletion delay
(set with `backup:key:init --delete-delay-days`). `occ backup:prune offsite --dry-run` shows what would be (set with `backup:key:init --delete-delay-days`). `occ backup:prune offsite --dry-run` shows what would be
removed without changing anything. removed without changing anything.
+15 -16
View File
@@ -43,25 +43,27 @@ final class BackupJob extends TimedJob {
$this->setTimeSensitivity(IJob::TIME_SENSITIVE); $this->setTimeSensitivity(IJob::TIME_SENSITIVE);
} }
private const MAX_DAILY_ATTEMPTS = 3;
protected function run($argument): void { protected function run($argument): void {
$budget = PHP_SAPI === 'cli' ? 240.0 : 20.0; $budget = PHP_SAPI === 'cli' ? 240.0 : 20.0;
$deadline = microtime(true) + $budget; $deadline = microtime(true) + $budget;
$this->startScheduled(); $this->startScheduled();
foreach ($this->runs->findRunning(null, BackupService::KIND_FULL) as $run) { // One after the other: runs that read the same disks at the same time are not faster together.
foreach ($this->backups->runsInTurn() as $run) {
while (microtime(true) < $deadline && $run->getStatus() === 'running') { while (microtime(true) < $deadline && $run->getStatus() === 'running') {
$next = $this->backups->step($run, min($deadline, microtime(true) + $budget)); $next = $this->backups->step($run, min($deadline, microtime(true) + $budget));
if ($next === $run) { if ($next === $run) {
break; // step() hands back the same object when another process holds the run's lease break 2; // step() hands back the same object when another process holds the run's lease
} }
$run = $next; $run = $next;
} }
} }
$this->failStaleRestores(); $this->failStaleRestores();
$this->weeklyPrune(); $this->weeklyPrune();
foreach ($this->targets->list() as $t) { $targets = $this->targets->list();
foreach ($targets as $t) {
$this->alerts->checkStale($t); $this->alerts->checkStale($t);
$othersActive = array_filter($targets, static fn ($o) => $o->getId() !== $t->getId() && (string)$o->getSchedule() !== '') !== [];
$this->alerts->checkOff($t, $othersActive);
} }
} }
@@ -102,21 +104,18 @@ final class BackupJob extends TimedJob {
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due); $this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
} }
/** Daily schedule: "HH:MM" in the schedule's time zone (see Schedule); empty = no schedule. */ /** Start the backups the schedules of the locations ask for; they run in turn (see LocationSchedule). */
private function startScheduled(): void { private function startScheduled(): void {
$todayAt = $this->schedule->todayAt(); $now = time();
if ($todayAt === null || time() < $todayAt) {
return;
}
foreach ($this->targets->list() as $t) { foreach ($this->targets->list() as $t) {
$last = $this->runs->lastFinished($t->getId(), BackupService::KIND_FULL); $moment = \OCA\NgBackup\Service\LocationSchedule::lastMoment((string)$t->getSchedule(), $this->schedule->zoneOr($t->getScheduleZone()), $now);
if ($this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== [] || ($last !== null && $last->getStartedAt() >= $todayAt)) { if ($moment === null || $this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== []) {
continue; continue;
} }
// After a failure: retry at most MAX_DAILY_ATTEMPTS times a day, an hour apart, instead $attempts = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $moment);
// of starting (and notifying about) a new full run every five minutes all day. if (!\OCA\NgBackup\Service\LocationSchedule::isDue($moment, (int)$t->getScheduleSince(),
$today = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $todayAt); $this->runs->lastFinished($t->getId(), BackupService::KIND_FULL)?->getStartedAt(),
if (count($today) >= self::MAX_DAILY_ATTEMPTS || ($today !== [] && time() - (int)$today[0]->getFinishedAt() < 3600)) { count($attempts), ($attempts[0] ?? null)?->getFinishedAt(), $now)) {
continue; continue;
} }
try { try {
+2 -2
View File
@@ -40,14 +40,14 @@ final class ListSnapshots extends Command {
foreach ($targets as $t) { foreach ($targets as $t) {
foreach ($this->snapshots->findForTarget($t->getId()) as $s) { foreach ($this->snapshots->findForTarget($t->getId()) as $s) {
$rows[] = [$t->getName(), $s->getSnapshotId(), Format::time($s->getCreatedAt(), $zone), $s->getKind(), $rows[] = [$t->getName(), $s->getSnapshotId(), Format::time($s->getCreatedAt(), $zone), $s->getKind(),
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getLabel() ?? '']; $s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getPinned() ? 'yes' : '', $s->getLabel() ?? '', $s->getNote() ?? ''];
} }
} }
if ($rows === []) { if ($rows === []) {
$output->writeln('No snapshots yet (occ backup:run).'); $output->writeln('No snapshots yet (occ backup:run).');
return 0; return 0;
} }
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'label'])->setRows($rows)->render(); (new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'pinned', 'label', 'note'])->setRows($rows)->render();
return 0; return 0;
} }
} }
+44
View File
@@ -0,0 +1,44 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\PruneService;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
final class Note extends Command {
public function __construct(
private TargetService $targets,
private PruneService $prune,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:note')
->setDescription('Set or remove the note on a snapshot')
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
->addArgument('note', InputArgument::OPTIONAL, 'The note, at most ' . PruneService::NOTE_MAX . ' characters; without it the note is removed', '');
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$note = (string)$input->getArgument('note');
try {
$this->prune->setNote($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), $note);
} catch (\Throwable $e) {
$output->writeln('<error>' . $e->getMessage() . '</error>');
return 1;
}
$output->writeln(trim($note) === '' ? 'Note removed.' : 'Note saved.');
return 0;
}
}
+45
View File
@@ -0,0 +1,45 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\PruneService;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
final class Pin extends Command {
public function __construct(
private TargetService $targets,
private PruneService $prune,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:pin')
->setDescription('Keep a snapshot: the retention policy does not remove a pinned one')
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
->addOption('unpin', null, InputOption::VALUE_NONE, 'Leave the snapshot to the retention policy again');
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$unpin = (bool)$input->getOption('unpin');
try {
$this->prune->setPinned($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), !$unpin);
} catch (\Throwable $e) {
$output->writeln('<error>' . $e->getMessage() . '</error>');
return 1;
}
$output->writeln($unpin ? 'Snapshot unpinned; the retention policy applies to it again.' : 'Snapshot pinned; the retention policy leaves it alone.');
return 0;
}
}
+3
View File
@@ -79,6 +79,9 @@ HELP);
if (($f['unreadable'] ?? 0) > 0) { if (($f['unreadable'] ?? 0) > 0) {
$output->writeln(sprintf('<comment> warning: %d unreadable files or folders are missing from this backup (see the server log)</comment>', $f['unreadable'])); $output->writeln(sprintf('<comment> warning: %d unreadable files or folders are missing from this backup (see the server log)</comment>', $f['unreadable']));
} }
if (($f['badNames'] ?? 0) > 0) {
$output->writeln(sprintf('<comment> warning: %d files or folders are missing from this backup because their name is not supported; rename them and the next backup includes them (see the server log)</comment>', $f['badNames']));
}
if (($f['changing'] ?? 0) > 0) { if (($f['changing'] ?? 0) > 0) {
$output->writeln(sprintf('<comment> warning: %d files kept changing and were saved as read (see the server log)</comment>', $f['changing'])); $output->writeln(sprintf('<comment> warning: %d files kept changing and were saved as read (see the server log)</comment>', $f['changing']));
} }
+4 -1
View File
@@ -31,6 +31,7 @@ final class Status extends Command {
private VerifyService $verify, private VerifyService $verify,
private Schedule $schedule, private Schedule $schedule,
private CronHealth $cron, private CronHealth $cron,
private \OCA\NgBackup\Service\BackupService $backups,
) { ) {
parent::__construct(); parent::__construct();
} }
@@ -61,9 +62,11 @@ final class Status extends Command {
$rows = []; $rows = [];
$all = (bool)$input->getOption('all'); $all = (bool)$input->getOption('all');
$shown = $this->runs->findShown($all ? null : time() - self::RECENT_DAYS * 86400); $shown = $this->runs->findShown($all ? null : time() - self::RECENT_DAYS * 86400);
$queued = array_map(static fn (\OCA\NgBackup\Db\Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1));
foreach ($shown as $r) { foreach ($shown as $r) {
$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? []; $s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus(), $rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(),
in_array($r->getId(), $queued, true) ? 'queued' : ($r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus()),
Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '', Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '', $s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')]; isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
+2 -1
View File
@@ -31,7 +31,8 @@ final class TargetList extends Base {
$rows = []; $rows = [];
foreach ($this->targets->list() as $t) { foreach ($this->targets->list() as $t) {
$row = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(), $row = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt())]; 'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt()),
'schedule' => (string)$t->getSchedule() !== '' ? $t->getSchedule() : 'off'];
if ($input->getOption('test')) { if ($input->getOption('test')) {
try { try {
$repo = $this->targets->repository($t); $repo = $this->targets->repository($t);
+50
View File
@@ -0,0 +1,50 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Command;
use OCA\NgBackup\Service\Schedule;
use OCA\NgBackup\Service\TargetService;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
final class TargetSchedule extends Command {
public function __construct(
private TargetService $targets,
private Schedule $schedule,
) {
parent::__construct();
}
protected function configure(): void {
$this->setName('backup:target:schedule')
->setDescription('Show or set when a location is backed up')
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
->addArgument('schedule', InputArgument::OPTIONAL, '"off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM; omit to show the current one')
->addOption('timezone', null, InputOption::VALUE_REQUIRED, 'Time zone of the time, e.g. Europe/Amsterdam');
}
protected function execute(InputInterface $input, OutputInterface $output): int {
$t = $this->targets->get($input->getArgument('target'));
$schedule = $input->getArgument('schedule');
$zone = $input->getOption('timezone') ?? $this->schedule->zoneOr($t->getScheduleZone());
if ($schedule !== null) {
try {
$t = $this->targets->setSchedule($t, $schedule, (string)$zone);
} catch (\InvalidArgumentException $e) {
$output->writeln('<error>' . $e->getMessage() . '</error>');
return 1;
}
}
$current = (string)$t->getSchedule();
$output->writeln($t->getName() . ': ' . ($current === '' ? 'off' : $current . ' (' . $zone . ')'));
return 0;
}
}
+34 -16
View File
@@ -59,10 +59,12 @@ final class AdminApiController extends Controller {
foreach ($this->targets->list() as $t) { foreach ($this->targets->list() as $t) {
$targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(), $targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt(), 'appendOnly' => (bool)$t->getAppendOnly(), 'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt(), 'appendOnly' => (bool)$t->getAppendOnly(),
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t)]; 'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t),
'schedule' => (string)$t->getSchedule(), 'scheduleZone' => $this->schedule->zoneOr($t->getScheduleZone())];
} }
$shown = $this->runs->findShown($allRuns ? null : time() - \OCA\NgBackup\Command\Status::RECENT_DAYS * 86400); $shown = $this->runs->findShown($allRuns ? null : time() - \OCA\NgBackup\Command\Status::RECENT_DAYS * 86400);
$runs = array_map(fn (Run $r) => $this->runJson($r), $shown); $queued = array_flip(array_map(static fn (Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1)));
$runs = array_map(fn (Run $r) => $this->runJson($r) + ['queued' => isset($queued[$r->getId()])], $shown);
return new JSONResponse([ return new JSONResponse([
'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(), 'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(),
'confirmation' => $this->keys->recoveryKitConfirmation(), 'everConfirmed' => $this->keys->isInitialized() && $this->keys->everConfirmed(), 'confirmation' => $this->keys->recoveryKitConfirmation(), 'everConfirmed' => $this->keys->isInitialized() && $this->keys->everConfirmed(),
@@ -74,8 +76,6 @@ final class AdminApiController extends Controller {
'runs' => $runs, 'runs' => $runs,
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown), 'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS, 'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
'schedule' => $this->schedule->get()['time'],
'scheduleTimezone' => $this->schedule->get()['timezone'],
'cron' => $this->cron->check(), 'cron' => $this->cron->check(),
]); ]);
} }
@@ -97,16 +97,16 @@ final class AdminApiController extends Controller {
} }
$deadline = microtime(true) + self::TICK_SECONDS; $deadline = microtime(true) + self::TICK_SECONDS;
$stepped = 0; $stepped = 0;
$running = $this->runs->findRunning(null, BackupService::KIND_FULL); foreach ($this->backups->runsInTurn() as $run) {
// Longest wait first: a run that takes all of a tick does not keep another from its turn.
usort($running, static fn ($a, $b) => $a->getUpdatedAt() <=> $b->getUpdatedAt());
foreach ($running as $run) {
if (microtime(true) >= $deadline) { if (microtime(true) >= $deadline) {
break; break;
} }
// step() returns at once when the background job (or another tab) holds the run. // step() returns at once when the background job (or another tab) holds the run.
$this->backups->step($run, $deadline); $next = $this->backups->step($run, $deadline);
$stepped++; $stepped++;
if ($next->getStatus() === Run::RUNNING) {
break; // not finished: the next one waits for its turn
}
} }
return new JSONResponse(['stepped' => $stepped]); return new JSONResponse(['stepped' => $stepped]);
} }
@@ -278,25 +278,43 @@ final class AdminApiController extends Controller {
return new JSONResponse($this->runJson($run)); return new JSONResponse($this->runJson($run));
} }
/** @param string $timezone the administrator's zone (the browser's); empty: keep the current one */ /** @param string $timezone the administrator's zone (the browser's); empty: the default one */
public function setSchedule(string $time, string $timezone = ''): JSONResponse { public function setTargetSchedule(int $id, string $schedule, string $timezone = ''): JSONResponse {
try { try {
$this->schedule->set($time, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']); $t = $this->targets->setSchedule($this->targets->get((string)$id), $schedule, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
} catch (\InvalidArgumentException $e) { } catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST); return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
} }
return new JSONResponse(['schedule' => $time, 'scheduleTimezone' => $this->schedule->get()['timezone']]); return new JSONResponse(['schedule' => $t->getSchedule(), 'scheduleZone' => $t->getScheduleZone()]);
} }
public function snapshots(int $targetId): JSONResponse { public function snapshots(int $targetId): JSONResponse {
$out = []; $out = [];
foreach ($this->snapshots->findForTarget($targetId) as $s) { foreach ($this->snapshots->findForTarget($targetId) as $s) {
$out[] = ['id' => $s->getSnapshotId(), 'kind' => $s->getKind(), 'label' => $s->getLabel(), 'createdAt' => $s->getCreatedAt(), $out[] = ['id' => $s->getSnapshotId(), 'kind' => $s->getKind(), 'label' => $s->getLabel(), 'createdAt' => $s->getCreatedAt(),
'files' => $s->getFiles(), 'bytes' => $s->getBytes()]; 'files' => $s->getFiles(), 'bytes' => $s->getBytes(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
} }
return new JSONResponse($out); return new JSONResponse($out);
} }
public function setSnapshotNote(int $targetId, string $snapshotId, string $note = ''): JSONResponse {
try {
$this->prune->setNote($this->targets->get((string)$targetId), $snapshotId, $note);
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
return new JSONResponse(['ok' => true]);
}
public function setSnapshotPinned(int $targetId, string $snapshotId, bool $pinned = true): JSONResponse {
try {
$this->prune->setPinned($this->targets->get((string)$targetId), $snapshotId, $pinned);
} catch (\Throwable $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
return new JSONResponse(['ok' => true]);
}
public function browse(int $targetId, string $snapshotId, string $path = ''): JSONResponse { public function browse(int $targetId, string $snapshotId, string $path = ''): JSONResponse {
try { try {
return new JSONResponse($this->restore->browse($this->targets->get((string)$targetId), $snapshotId, $path)); return new JSONResponse($this->restore->browse($this->targets->get((string)$targetId), $snapshotId, $path));
@@ -341,7 +359,7 @@ final class AdminApiController extends Controller {
if ($r->getSnapshotId() !== null) { if ($r->getSnapshotId() !== null) {
try { try {
$s = $this->snapshots->findOne($r->getTargetId(), $r->getSnapshotId()); $s = $this->snapshots->findOne($r->getTargetId(), $r->getSnapshotId());
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel()]; $snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
} catch (\Throwable) { } catch (\Throwable) {
// Snapshot already pruned or trashed: the run keeps its id, just not the date. // Snapshot already pruned or trashed: the run keeps its id, just not the date.
} }
+7
View File
@@ -24,6 +24,10 @@ use OCP\AppFramework\Db\Entity;
* @method void setFiles(?int $v) * @method void setFiles(?int $v)
* @method int|null getBytes() * @method int|null getBytes()
* @method void setBytes(?int $v) * @method void setBytes(?int $v)
* @method string|null getNote()
* @method void setNote(?string $v)
* @method bool|null getPinned()
* @method void setPinned(?bool $v)
*/ */
final class Snapshot extends Entity { final class Snapshot extends Entity {
protected int $targetId = 0; protected int $targetId = 0;
@@ -33,8 +37,11 @@ final class Snapshot extends Entity {
protected int $createdAt = 0; protected int $createdAt = 0;
protected ?int $files = null; protected ?int $files = null;
protected ?int $bytes = null; protected ?int $bytes = null;
protected ?string $note = null;
protected ?bool $pinned = null;
public function __construct() { public function __construct() {
$this->addType('pinned', 'boolean');
$this->addType('targetId', 'integer'); $this->addType('targetId', 'integer');
$this->addType('createdAt', 'integer'); $this->addType('createdAt', 'integer');
$this->addType('files', 'integer'); $this->addType('files', 'integer');
+10
View File
@@ -28,6 +28,12 @@ use OCP\AppFramework\Db\Entity;
* @method void setMaxBytes(?int $v) * @method void setMaxBytes(?int $v)
* @method int getCreatedAt() * @method int getCreatedAt()
* @method void setCreatedAt(int $v) * @method void setCreatedAt(int $v)
* @method string|null getSchedule()
* @method void setSchedule(?string $v)
* @method string|null getScheduleZone()
* @method void setScheduleZone(?string $v)
* @method int|null getScheduleSince()
* @method void setScheduleSince(?int $v)
*/ */
final class Target extends Entity { final class Target extends Entity {
protected string $name = ''; protected string $name = '';
@@ -39,8 +45,12 @@ final class Target extends Entity {
protected int $createdAt = 0; protected int $createdAt = 0;
protected ?bool $appendOnly = null; protected ?bool $appendOnly = null;
protected ?int $maxBytes = null; protected ?int $maxBytes = null;
protected ?string $schedule = null;
protected ?string $scheduleZone = null;
protected ?int $scheduleSince = null;
public function __construct() { public function __construct() {
$this->addType('scheduleSince', 'integer');
$this->addType('createdAt', 'integer'); $this->addType('createdAt', 'integer');
$this->addType('appendOnly', 'boolean'); $this->addType('appendOnly', 'boolean');
$this->addType('maxBytes', 'integer'); $this->addType('maxBytes', 'integer');
+46 -5
View File
@@ -52,19 +52,35 @@ final class BackupRun {
$stats = $state['stats']; $stats = $state['stats'];
$walker = new TreeWalker($state['roots'], $state['exclude'], function (string $logical) use (&$state, &$stats): void { $walker = new TreeWalker($state['roots'], $state['exclude'], function (string $logical) use (&$state, &$stats): void {
self::recordUnreadable($state, $stats, $logical); self::recordUnreadable($state, $stats, $logical);
}); }, true);
$parent = new ParentCursor($repo, $state['parent'], $state['parentPos']); $parent = new ParentCursor($repo, $state['parent'], $state['parentPos']);
$lines = []; $lines = [];
$emptyDirs = []; $emptyDirs = [];
$folders = [];
$finishedWalk = true; $finishedWalk = true;
foreach ($walker->walk($state['after']) as $logical => $path) { foreach ($walker->walk($state['after']) as $logical => $path) {
if (!mb_check_encoding($logical, 'UTF-8') || !Repository::isSafePath($logical)) { $safe = mb_check_encoding($logical, 'UTF-8') && Repository::isSafePath($logical);
if ($walker->atFolderWithContent()) {
// Only its time: what is in it is reported by itself, and a bad name in the path of
// each of those is what gets reported. The cursor stays for such a name, see below.
$mtime = $safe ? @filemtime($path) : false;
if ($safe && $mtime !== false) {
$folders[] = json_encode([$logical, $mtime], JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
$state['after'] = $logical;
if (microtime(true) >= $deadline) {
$finishedWalk = false;
break;
}
}
continue;
}
if (!$safe) {
// A name a snapshot cannot hold (a backslash, a control character, not UTF-8). One // A name a snapshot cannot hold (a backslash, a control character, not UTF-8). One
// such path must not make a backup fail, or the whole snapshot unreadable later. // such path must not make a backup fail, or the whole snapshot unreadable later.
// The cursor stays where it is: the state is JSON, which cannot hold such a name. A // The cursor stays where it is: the state is JSON, which cannot hold such a name. A
// next step meets it again, and it is reported once. // next step meets it again, and it is reported once.
self::recordUnreadable($state, $stats, mb_scrub($logical, 'UTF-8')); self::recordBadName($state, $stats, $logical);
continue; continue;
} }
if (is_dir($path)) { if (is_dir($path)) {
@@ -164,6 +180,11 @@ final class BackupRun {
$writer->write(implode("\n", $emptyDirs) . "\n"); $writer->write(implode("\n", $emptyDirs) . "\n");
$state['dirSegments'][] = $writer->finish(); $state['dirSegments'][] = $writer->finish();
} }
if ($folders !== []) {
$writer = $repo->blobWriter($stats);
$writer->write(implode("\n", $folders) . "\n");
$state['folderSegments'][] = $writer->finish();
}
$repo->flushPacks(); $repo->flushPacks();
$state['parentPos'] = $parent->position(); $state['parentPos'] = $parent->position();
$state['stats'] = $stats; $state['stats'] = $stats;
@@ -186,10 +207,14 @@ final class BackupRun {
$emptyDirs[] = json_decode($line, true, 8, JSON_THROW_ON_ERROR); $emptyDirs[] = json_decode($line, true, 8, JSON_THROW_ON_ERROR);
} }
} }
// The times of the other folders too, packed: every look at a snapshot's meta record reads
// this, and a string of a few hundred KiB costs that far less than tens of thousands of arrays.
$folders = isset($state['folderSegments']) ? implode("\n", iterator_to_array($repo->readLines(array_values(array_merge(...$state['folderSegments']))), false)) : '';
$repo->putSnapshot($state['snapshot'], $repo->putSnapshot($state['snapshot'],
['time' => $state['started'], 'parent' => $state['parent'], 'label' => $state['label'], ['time' => $state['started'], 'parent' => $state['parent'], 'label' => $state['label'],
'roots' => array_keys($state['roots']), 'stats' => $state['stats'], 'roots' => array_keys($state['roots']), 'stats' => $state['stats'],
'warnings' => self::warnings($state)] + ($emptyDirs === [] ? [] : ['emptyDirs' => $emptyDirs]) + $state['meta'], 'warnings' => self::warnings($state)] + ($emptyDirs === [] ? [] : ['emptyDirs' => $emptyDirs])
+ ($folders === '' ? [] : ['folderTimes' => base64_encode((string)gzdeflate($folders))]) + $state['meta'],
$repo->readLines(array_values(array_merge(...$state['segments'])))); $repo->readLines(array_values(array_merge(...$state['segments']))));
$state['phase'] = 'commit'; $state['phase'] = 'commit';
} }
@@ -250,10 +275,26 @@ final class BackupRun {
$stats['unreadable'] = ($stats['unreadable'] ?? 0) + 1; $stats['unreadable'] = ($stats['unreadable'] ?? 0) + 1;
} }
/** @return array{unreadable: list<string>, changing: list<string>} examples for the report (first 50 each) */ /**
* A file or directory left out because of its name. Remembered like an unreadable one, in a form
* that fits the JSON state and is safe to show: control characters and invalid bytes replaced.
*/
private static function recordBadName(array &$state, array &$stats, string $logical): void {
$shown = (string)preg_replace('/[\x00-\x1f\x7f]/', '?', mb_scrub($logical, 'UTF-8'));
if (isset($state['badNames'][$shown])) {
return;
}
if (count($state['badNames'] ?? []) < 1000) {
$state['badNames'][$shown] = true;
}
$stats['badNames'] = ($stats['badNames'] ?? 0) + 1;
}
/** @return array{unreadable: list<string>, badNames: list<string>, changing: list<string>} examples for the report (first 50 each) */
public static function warnings(array $state): array { public static function warnings(array $state): array {
return [ return [
'unreadable' => array_slice(array_map('strval', array_keys($state['unreadable'] ?? [])), 0, 50), 'unreadable' => array_slice(array_map('strval', array_keys($state['unreadable'] ?? [])), 0, 50),
'badNames' => array_slice(array_map('strval', array_keys($state['badNames'] ?? [])), 0, 50),
'changing' => array_slice(array_map('strval', array_values($state['changing'] ?? [])), 0, 50), 'changing' => array_slice(array_map('strval', array_values($state['changing'] ?? [])), 0, 50),
]; ];
} }
+17 -1
View File
@@ -17,6 +17,8 @@ namespace OCA\NgBackup\Job;
* and be merge-joined with the previous snapshot without holding any list in memory. * and be merge-joined with the previous snapshot without holding any list in memory.
*/ */
final class TreeWalker { final class TreeWalker {
private bool $atFolder = false;
/** /**
* @param array<string, string> $roots name => absolute directory, walked in this order * @param array<string, string> $roots name => absolute directory, walked in this order
* @param list<string> $exclude path prefixes ("<root>/<relative>") that are skipped entirely; * @param list<string> $exclude path prefixes ("<root>/<relative>") that are skipped entirely;
@@ -27,9 +29,16 @@ final class TreeWalker {
private array $exclude = [], private array $exclude = [],
/** Called with the logical path of a directory that cannot be listed. */ /** Called with the logical path of a directory that cannot be listed. */
private ?\Closure $onUnreadable = null, private ?\Closure $onUnreadable = null,
/** Also yield directories with something in them, right before the first thing in them. */
private bool $withFolders = false,
) { ) {
} }
/** Whether what was just yielded is a directory with something in it (see $withFolders). */
public function atFolderWithContent(): bool {
return $this->atFolder;
}
/** @return \Generator<string, string> logical path => absolute path, strictly after $after */ /** @return \Generator<string, string> logical path => absolute path, strictly after $after */
public function walk(?string $after = null): \Generator { public function walk(?string $after = null): \Generator {
foreach ($this->roots as $name => $dir) { foreach ($this->roots as $name => $dir) {
@@ -75,7 +84,14 @@ final class TreeWalker {
if (is_link($childAbs)) { if (is_link($childAbs)) {
continue; // never follow symlinks out of the data directory continue; // never follow symlinks out of the data directory
} }
if (is_dir($childAbs)) { $childIsDir = is_dir($childAbs);
if (($childIsDir || is_file($childAbs)) && $empty && !$this->isExcluded($childLogical, $childIsDir)
&& $this->withFolders && $logical !== '' && ($after === null || self::compare($logical, $after) > 0)) {
$this->atFolder = true;
yield $logical => $abs;
$this->atFolder = false;
}
if ($childIsDir) {
$empty = $empty && $this->isExcluded($childLogical, true); $empty = $empty && $this->isExcluded($childLogical, true);
// Skip a directory entirely if everything in it sorts before the cursor. // Skip a directory entirely if everything in it sorts before the cursor.
if ($after !== null && self::compare($childLogical, $after) < 0 && !self::isPrefix($childLogical, $after)) { if ($after !== null && self::compare($childLogical, $after) < 0 && !self::isPrefix($childLogical, $after)) {
@@ -0,0 +1,33 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Migration;
use Closure;
use OCP\DB\ISchemaWrapper;
use OCP\DB\Types;
use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep;
/** Local copy of what the catalog on the location says about a snapshot: its note and pin. */
final class Version000007Date20261107000000 extends SimpleMigrationStep {
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */
$schema = $schemaClosure();
$t = $schema->getTable('ngb_snapshots');
$changed = false;
if (!$t->hasColumn('note')) {
$t->addColumn('note', Types::STRING, ['notnull' => false, 'length' => 1000]);
$changed = true;
}
if (!$t->hasColumn('pinned')) {
$t->addColumn('pinned', Types::BOOLEAN, ['notnull' => false, 'default' => false]);
$changed = true;
}
return $changed ? $schema : null;
}
}
@@ -0,0 +1,49 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Migration;
use Closure;
use OCA\NgBackup\Service\LocationSchedule;
use OCA\NgBackup\Service\Schedule;
use OCP\DB\ISchemaWrapper;
use OCP\DB\Types;
use OCP\IDBConnection;
use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep;
/** A schedule per location. Each existing location gets the daily time that held for all of them. */
final class Version000008Date20261107010000 extends SimpleMigrationStep {
public function __construct(
private IDBConnection $db,
private Schedule $schedule,
) {
}
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */
$schema = $schemaClosure();
$t = $schema->getTable('ngb_targets');
if ($t->hasColumn('schedule')) {
return null;
}
$t->addColumn('schedule', Types::STRING, ['notnull' => false, 'length' => 16]);
$t->addColumn('schedule_zone', Types::STRING, ['notnull' => false, 'length' => 64]);
$t->addColumn('schedule_since', Types::BIGINT, ['notnull' => false, 'unsigned' => true]);
return $schema;
}
public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options): void {
['time' => $time, 'timezone' => $zone] = $this->schedule->get();
$qb = $this->db->getQueryBuilder();
$qb->update('ngb_targets')
->set('schedule', $qb->createNamedParameter(Schedule::validTime($time) ? LocationSchedule::normalize($time) : ''))
->set('schedule_zone', $qb->createNamedParameter($zone))
->where($qb->expr()->isNull('schedule'));
$qb->executeStatement();
}
}
+12 -1
View File
@@ -46,6 +46,10 @@ final class Notifier implements INotifier {
$notification->setParsedSubject($l->t('Backup to %1$s finished with warnings: %2$s unreadable, %3$s changed while being read', [$p['target'], $p['unreadable'], $p['changing']])); $notification->setParsedSubject($l->t('Backup to %1$s finished with warnings: %2$s unreadable, %3$s changed while being read', [$p['target'], $p['unreadable'], $p['changing']]));
$notification->setParsedMessage($l->t('Unreadable files and folders are missing from this backup (check their permissions). Files that kept changing were saved as they were while being read. Examples: %s', [$p['examples'] ?? ''])); $notification->setParsedMessage($l->t('Unreadable files and folders are missing from this backup (check their permissions). Files that kept changing were saved as they were while being read. Examples: %s', [$p['examples'] ?? '']));
break; break;
case 'run_bad_names':
$notification->setParsedSubject($l->n('Backup to %s finished, but %n item is missing because of its name', 'Backup to %s finished, but %n items are missing because of their names', (int)$p['count'], [$p['target']]));
$notification->setParsedMessage($l->t('Names with a backslash or a control character, or that are not valid UTF-8, are not supported, so these files and folders are not in this backup. Rename them; the next backup includes them. Examples: %s', [$p['examples'] ?? '']));
break;
case 'quota_warning': case 'quota_warning':
$notification->setParsedSubject($l->t('Backup location %1$s is at %2$s%% of its limit (%3$s of %4$s)', [$p['target'], $p['percent'], $p['used'], $p['limit']])); $notification->setParsedSubject($l->t('Backup location %1$s is at %2$s%% of its limit (%3$s of %4$s)', [$p['target'], $p['percent'], $p['used'], $p['limit']]));
$notification->setParsedMessage($l->t('Backups stop when the limit is reached. Raise the limit, or let the retention policy remove older backups.')); $notification->setParsedMessage($l->t('Backups stop when the limit is reached. Raise the limit, or let the retention policy remove older backups.'));
@@ -57,13 +61,20 @@ final class Notifier implements INotifier {
: $l->t('No successful backup to %1$s since %2$s', [$p['target'], $p['since']])); : $l->t('No successful backup to %1$s since %2$s', [$p['target'], $p['since']]));
$notification->setParsedMessage($l->t('Check the backup settings and the server log.')); $notification->setParsedMessage($l->t('Check the backup settings and the server log.'));
break; break;
case 'schedule_off':
$notification->setParsedSubject($l->t('Backup location %s has no schedule', [$p['target']]));
$notification->setParsedMessage(($p['others'] ?? '') === '1'
? $l->t('No backups run to this location until it gets a schedule or someone starts one. If that is what you want, you can ignore this reminder.')
: $l->t('No location has a schedule, so no backups run at all unless someone starts one. Set a schedule under Backups.'));
break;
case 'security_event': case 'security_event':
$who = (string)($p['actor'] ?? ''); $who = (string)($p['actor'] ?? '');
$notification->setParsedSubject(match ($p['event'] ?? '') { $notification->setParsedSubject(match ($p['event'] ?? '') {
'kit_downloaded' => $l->t('%s downloaded the backup recovery kit', [$who]), 'kit_downloaded' => $l->t('%s downloaded the backup recovery kit', [$who]),
'target_removed' => $l->t('%1$s removed backup location %2$s', [$who, $p['target'] ?? '']), 'target_removed' => $l->t('%1$s removed backup location %2$s', [$who, $p['target'] ?? '']),
'target_changed' => $l->t('%1$s changed the connection settings of backup location %2$s', [$who, $p['target'] ?? '']), 'target_changed' => $l->t('%1$s changed the connection settings of backup location %2$s', [$who, $p['target'] ?? '']),
'append_only_off' => $l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']), 'schedule_off' => $l->t('%1$s turned off the schedule of backup location %2$s', [$who, $p['target'] ?? '']),
'append_only_off' =>$l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
'retention_changed' => $l->t('%1$s changed the backup retention to %2$s', [$who, $p['policy'] ?? '']), 'retention_changed' => $l->t('%1$s changed the backup retention to %2$s', [$who, $p['policy'] ?? '']),
'snapshots_forgotten' => $l->t('%1$s moved %2$s snapshots of %3$s to the trash; they are deleted permanently after %4$s unless restored', [$who, $p['count'] ?? '', $p['target'] ?? '', $p['until'] ?? '']), 'snapshots_forgotten' => $l->t('%1$s moved %2$s snapshots of %3$s to the trash; they are deleted permanently after %4$s unless restored', [$who, $p['count'] ?? '', $p['target'] ?? '', $p['until'] ?? '']),
'snapshot_untrashed' => $l->t('%1$s restored snapshot %2$s of %3$s from the trash', [$who, $p['snapshot'] ?? '', $p['target'] ?? '']), 'snapshot_untrashed' => $l->t('%1$s restored snapshot %2$s of %3$s from the trash', [$who, $p['snapshot'] ?? '', $p['target'] ?? '']),
+10 -1
View File
@@ -30,6 +30,10 @@ final class Catalog {
public array $trash = []; public array $trash = [];
/** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */ /** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */
public array $userExports = []; public array $userExports = [];
/** @var array<string, true> snapshots the retention policy leaves alone */
public array $pinned = [];
/** @var array<string, string> snapshot id => note, also while the snapshot is in the trash */
public array $notes = [];
public string $hash = ''; public string $hash = '';
public function __construct( public function __construct(
@@ -82,6 +86,8 @@ final class Catalog {
// Catalogs written before user exports existed have no such key: default to empty rather // Catalogs written before user exports existed have no such key: default to empty rather
// than adopting the location's current users/ listing (that would skip verification). // than adopting the location's current users/ listing (that would skip verification).
$this->userExports = array_fill_keys($data['userExports'] ?? [], true); $this->userExports = array_fill_keys($data['userExports'] ?? [], true);
$this->pinned = array_fill_keys($data['pinned'] ?? [], true);
$this->notes = $data['notes'] ?? [];
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash); $this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
} }
@@ -93,9 +99,12 @@ final class Catalog {
if ($this->backend->exists($this->path($next))) { if ($this->backend->exists($this->path($next))) {
throw new RepositoryException("Catalog generation $next was written by another process at the same time; try again"); throw new RepositoryException("Catalog generation $next was written by another process at the same time; try again");
} }
// A pin ends when its snapshot leaves the list, a note when the snapshot is gone for good.
$this->pinned = array_intersect_key($this->pinned, $this->snapshots);
$this->notes = array_intersect_key($this->notes, $this->snapshots + $this->trash);
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash, $plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash), 'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash),
'userExports' => array_keys($this->userExports)], JSON_THROW_ON_ERROR); 'userExports' => array_keys($this->userExports), 'pinned' => array_keys($this->pinned), 'notes' => $this->notes], JSON_THROW_ON_ERROR);
$encrypted = $this->cipher->encryptString($plain, $this->ad($next)); $encrypted = $this->cipher->encryptString($plain, $this->ad($next));
$stream = fopen('php://memory', 'w+b'); $stream = fopen('php://memory', 'w+b');
fwrite($stream, $encrypted); fwrite($stream, $encrypted);
+75
View File
@@ -134,6 +134,49 @@ final class Repository {
return $this->catalog->gen; return $this->catalog->gen;
} }
/** @return list<string> snapshots the retention policy leaves alone */
public function pinnedSnapshots(): array {
return array_map('strval', array_keys($this->catalog->pinned));
}
/** @return array<string, string> snapshot id => note */
public function snapshotNotes(): array {
return $this->catalog->notes;
}
public function setPinned(string $snapshotId, bool $pinned): void {
$this->requireSnapshot($snapshotId);
if (isset($this->catalog->pinned[$snapshotId]) === $pinned) {
return;
}
if ($pinned) {
$this->catalog->pinned[$snapshotId] = true;
} else {
unset($this->catalog->pinned[$snapshotId]);
}
$this->catalog->write();
}
/** A note kept in the catalog: unlike the label inside the snapshot, it can change afterwards. */
public function setSnapshotNote(string $snapshotId, string $note): void {
$this->requireSnapshot($snapshotId);
if (($this->catalog->notes[$snapshotId] ?? '') === $note) {
return;
}
if ($note === '') {
unset($this->catalog->notes[$snapshotId]);
} else {
$this->catalog->notes[$snapshotId] = $note;
}
$this->catalog->write();
}
private function requireSnapshot(string $snapshotId): void {
if (!isset($this->catalog->snapshots[$snapshotId])) {
throw new \InvalidArgumentException('There is no snapshot ' . $snapshotId . ' on this location');
}
}
/** Index files downloaded when this repository was opened (0 when the local cache was complete). */ /** Index files downloaded when this repository was opened (0 when the local cache was complete). */
public function indexDownloads(): int { public function indexDownloads(): int {
return $this->index->downloaded; return $this->index->downloaded;
@@ -392,9 +435,41 @@ final class Repository {
self::restoreEmptyDir($target, $dest, $path, $mtime); self::restoreEmptyDir($target, $dest, $path, $mtime);
} }
} }
// Very last, nothing is written in them any more. Also on a run that goes on after a stop, and
// the deepest first. The time is not worth stopping for: all the files are there by now.
foreach (array_reverse($this->folderTimes($snapshotId)) as [$path, $mtime]) {
if (!self::underPrefix($path, $prefix) || ($stripPrefix && $path === rtrim($prefix, '/'))) {
continue; // the target itself is not ours to change
}
if ($heartbeat !== null) {
$heartbeat();
}
$dest = self::destination($target, $path, $prefix, $stripPrefix);
@touch(self::directoryInTarget($target, $dest, $path), $mtime);
}
return $count; return $count;
} }
/**
* The folders with something in them that a snapshot recorded, parents before children (none
* before 1.1.0). Damage here is no reason to give up on the files of a restore.
*
* @return list<array{0: string, 1: int}> path, modification time
*/
public function folderTimes(string $snapshotId): array {
$meta = $this->lastMeta[$snapshotId] ?? $this->snapshotMeta($snapshotId);
$packed = is_string($meta['folderTimes'] ?? null) ? base64_decode($meta['folderTimes'], true) : false;
$plain = $packed === false ? false : @gzinflate($packed);
$folders = [];
foreach ($plain === false ? [] : explode("\n", $plain) as $line) {
$folder = json_decode($line, true);
if (is_array($folder) && is_string($folder[0] ?? null) && self::isSafePath($folder[0]) && is_int($folder[1] ?? null)) {
$folders[] = [$folder[0], $folder[1]];
}
}
return $folders;
}
/** /**
* The folders without anything in them that a snapshot recorded (none before 1.0.3). * The folders without anything in them that a snapshot recorded (none before 1.0.3).
* *
+38 -1
View File
@@ -61,6 +61,16 @@ final class AlertService {
'examples' => mb_strimwidth(implode(', ', array_merge($examples['unreadable'], $examples['changing'])), 0, 300, '…')]); 'examples' => mb_strimwidth(implode(', ', array_merge($examples['unreadable'], $examples['changing'])), 0, 300, '…')]);
} }
/**
* A backup that finished but left out files or folders because of their name.
*
* @param list<string> $examples
*/
public function runBadNames(Run $run, string $targetName, int $count, array $examples): void {
$this->notifyAdmins('run_bad_names', 'run-' . $run->getId(), ['target' => $targetName, 'count' => (string)$count,
'examples' => mb_strimwidth(implode(', ', $examples), 0, 300, '…')]);
}
/** /**
* Remember how much is stored on a location (for the settings page) and warn administrators * Remember how much is stored on a location (for the settings page) and warn administrators
* once a day when it is at 80% of its limit or more. * once a day when it is at 80% of its limit or more.
@@ -91,9 +101,36 @@ final class AlertService {
return $v === '' ? null : (int)$v; return $v === '' ? null : (int)$v;
} }
/**
* Whether it is time to remind administrators of a location without a schedule: once a week
* while another location is still backed up, every day when none is. An hour of margin, so
* that the moment of the cron run does not push a reminder to the next day.
*/
public static function offReminderDue(int $last, int $now, bool $othersActive): bool {
return $now - $last >= ($othersActive ? 7 * 86400 : 86400) - 3600;
}
public function checkOff(Target $target, bool $othersActive): void {
if ((string)$target->getSchedule() !== '') {
return;
}
$key = 'off_notified_' . $target->getId();
$now = time();
$last = max((int)$target->getScheduleSince(), $this->appConfig->getValueInt(Application::APP_ID, $key, 0));
if (!self::offReminderDue($last, $now, $othersActive)) {
return;
}
$this->appConfig->setValueInt(Application::APP_ID, $key, $now);
$this->notifyAdmins('schedule_off', 'off-' . $target->getId() . '-' . date('Y-m-d', $now),
['target' => $target->getName(), 'others' => $othersActive ? '1' : '0']);
}
/** At most one notification per location per day. */ /** At most one notification per location per day. */
public function checkStale(Target $target): void { public function checkStale(Target $target): void {
$last = $this->runs->lastFinished($target->getId(), BackupService::KIND_FULL); if ((string)$target->getSchedule() === '') {
return; // no schedule is a choice, as for a location that is only connected now and then
}
$last =$this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
$lastTime = $last?->getFinishedAt() ?? $target->getCreatedAt(); $lastTime = $last?->getFinishedAt() ?? $target->getCreatedAt();
if (time() - $lastTime < self::STALE_AFTER || $this->runs->findRunning($target->getId(), BackupService::KIND_FULL) !== []) { if (time() - $lastTime < self::STALE_AFTER || $this->runs->findRunning($target->getId(), BackupService::KIND_FULL) !== []) {
return; return;
+22 -1
View File
@@ -58,6 +58,18 @@ final class BackupService {
$this->openedWith = $opened; $this->openedWith = $opened;
} }
/**
* The running backups in the order they get their turn: first come, first served. The first one
* runs, the others are queued; only one at a time reads the disks.
*
* @return list<Run>
*/
public function runsInTurn(): array {
$running = $this->runs->findRunning(null, self::KIND_FULL);
usort($running, static fn (Run $a, Run $b): int => [$a->getStartedAt(), $a->getId()] <=> [$b->getStartedAt(), $b->getId()]);
return $running;
}
public function start(Target $target, string $label = ''): Run { public function start(Target $target, string $label = ''): Run {
$label = trim($label); $label = trim($label);
if (mb_strlen($label) > self::LABEL_MAX) { if (mb_strlen($label) > self::LABEL_MAX) {
@@ -210,6 +222,10 @@ final class BackupService {
$state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR); $state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
if ($run->getPhase() === 'db') { if ($run->getPhase() === 'db') {
if (!isset($state['dump'])) {
// It may have been queued for hours: the restore point has the time of its data.
$state['files']['started'] = gmdate('c');
}
$dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_')); $dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_'));
$dbStats = $state['dbStats'] ?? []; $dbStats = $state['dbStats'] ?? [];
// With system cron the whole dump usually fits in one step (one transaction, fully // With system cron the whole dump usually fits in one step (one transaction, fully
@@ -253,8 +269,13 @@ final class BackupService {
try { try {
$this->alerts->storedBytes($target, $repo->storedBytes()); $this->alerts->storedBytes($target, $repo->storedBytes());
$stats = $state['files']['stats']; $stats = $state['files']['stats'];
$warnings = BackupRun::warnings($state['files']);
if (($stats['badNames'] ?? 0) > 0) {
$this->logger->warning(sprintf('NG Backup run %d to %s: %d left out because their name is not supported (rename them): %s',
$run->getId(), $target->getName(), $stats['badNames'], implode(', ', $warnings['badNames'])));
$this->alerts->runBadNames($run, $target->getName(), (int)$stats['badNames'], $warnings['badNames']);
}
if (($stats['unreadable'] ?? 0) > 0 || ($stats['changing'] ?? 0) > 0) { if (($stats['unreadable'] ?? 0) > 0 || ($stats['changing'] ?? 0) > 0) {
$warnings = BackupRun::warnings($state['files']);
$this->logger->warning(sprintf('NG Backup run %d to %s: %d unreadable (missing from the snapshot): %s; %d changed while being read: %s', $this->logger->warning(sprintf('NG Backup run %d to %s: %d unreadable (missing from the snapshot): %s; %d changed while being read: %s',
$run->getId(), $target->getName(), $stats['unreadable'] ?? 0, implode(', ', $warnings['unreadable']), $run->getId(), $target->getName(), $stats['unreadable'] ?? 0, implode(', ', $warnings['unreadable']),
$stats['changing'] ?? 0, implode(', ', $warnings['changing']))); $stats['changing'] ?? 0, implode(', ', $warnings['changing'])));
+78
View File
@@ -0,0 +1,78 @@
<?php
declare(strict_types=1);
// SPDX-FileCopyrightText: 2026 Dillard Blom
// SPDX-License-Identifier: AGPL-3.0-or-later
namespace OCA\NgBackup\Service;
/**
* When a location is backed up: "" (off), "HH:MM" (daily) or "HH:MM/N" (from HH:MM every N hours,
* starting over at HH:MM the next day), in the location's time zone.
*/
final class LocationSchedule {
/** Attempts per moment after a failure, an hour apart, instead of one every few minutes. */
public const MAX_ATTEMPTS = 3;
public const RETRY_AFTER = 3600;
/** @throws \InvalidArgumentException */
public static function normalize(string $schedule): string {
$schedule = trim($schedule);
if ($schedule === '' || strtolower($schedule) === 'off') {
return '';
}
if (preg_match('/^(([01]\d|2[0-3]):[0-5]\d)(?:\/(\d{1,2}))?$/', $schedule, $m) !== 1) {
throw new \InvalidArgumentException('Use "off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM');
}
$hours = (int)($m[3] ?? 24);
if ($hours < 1 || $hours > 24) {
throw new \InvalidArgumentException('Every 1 to 24 hours');
}
return $hours === 24 ? $m[1] : $m[1] . '/' . $hours;
}
/**
* The most recent scheduled moment at or before $now; null without a schedule.
*
* @param non-empty-string $zone
*/
public static function lastMoment(string $schedule, string $zone, int $now): ?int {
if ($schedule === '') {
return null;
}
$tz = new \DateTimeZone($zone);
$today = (new \DateTimeImmutable('@' . $now))->setTimezone($tz);
$moments = array_merge(self::moments($schedule, $today->modify('-1 day'), $tz), self::moments($schedule, $today, $tz));
$last = null;
foreach ($moments as $moment) {
if ($moment <= $now && ($last === null || $moment > $last)) {
$last = $moment;
}
}
return $last;
}
/** @return list<int> the moments of one day */
private static function moments(string $schedule, \DateTimeImmutable $day, \DateTimeZone $tz): array {
[$time, $every] = array_pad(explode('/', $schedule), 2, '24');
[$hour, $minute] = array_map('intval', explode(':', $time));
$moments = [];
for ($h = $hour; $h < 24; $h += (int)$every) {
$moments[] = (new \DateTimeImmutable($day->format('Y-m-d') . sprintf(' %02d:%02d', $h, $minute), $tz))->getTimestamp();
}
return $moments;
}
/**
* Whether a backup has to start for $moment. One that was made since then is enough, also when
* several moments went by during a long backup. A moment from before the schedule was set does
* not count: choosing a time must not start a backup right away.
*/
public static function isDue(int $moment, int $since, ?int $lastDoneStarted, int $attempts, ?int $lastAttemptFinished, int $now): bool {
if ($moment < $since || ($lastDoneStarted !== null && $lastDoneStarted >= $moment) || $attempts >= self::MAX_ATTEMPTS) {
return false;
}
return $attempts === 0 || $now - (int)$lastAttemptFinished >= self::RETRY_AFTER;
}
}
+51 -1
View File
@@ -11,6 +11,7 @@ use OCA\NgBackup\AppInfo\Application;
use OCA\NgBackup\Db\RunMapper; use OCA\NgBackup\Db\RunMapper;
use OCA\NgBackup\Db\SnapshotMapper; use OCA\NgBackup\Db\SnapshotMapper;
use OCA\NgBackup\Db\Target; use OCA\NgBackup\Db\Target;
use OCA\NgBackup\Repository\Repository;
use OCP\IAppConfig; use OCP\IAppConfig;
use OCP\Lock\ILockingProvider; use OCP\Lock\ILockingProvider;
use OCP\Lock\LockedException; use OCP\Lock\LockedException;
@@ -33,6 +34,54 @@ final class PruneService {
) { ) {
} }
/** Characters of a note on a snapshot (the column holds 1000) */
public const NOTE_MAX = 500;
/** Pin or unpin a snapshot: the retention policy leaves a pinned one alone. */
public function setPinned(Target $target, string $snapshotId, bool $pinned): void {
$changed = $this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $pinned): bool {
$was = in_array($snapshotId, $repo->pinnedSnapshots(), true);
$repo->setPinned($snapshotId, $pinned);
return $was !== $pinned;
});
$this->targets->syncSnapshot($target, $snapshotId, pinned: $pinned);
if ($changed) {
// Unpinning opens the way to removing it, so every administrator hears about that.
$this->alerts->securityEvent($pinned ? 'snapshot_pinned' : 'snapshot_unpinned', ['target' => $target->getName(), 'snapshot' => $snapshotId], !$pinned);
}
}
public function setNote(Target $target, string $snapshotId, string $note): void {
$note = trim((string)preg_replace('/[\x00-\x1f\x7f]+/', ' ', $note));
if (!mb_check_encoding($note, 'UTF-8')) {
throw new \InvalidArgumentException('A note must be valid UTF-8');
}
if (mb_strlen($note) > self::NOTE_MAX) {
throw new \InvalidArgumentException('A note has at most ' . self::NOTE_MAX . ' characters');
}
$this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $note): bool {
$repo->setSnapshotNote($snapshotId, $note);
return true;
});
$this->targets->syncSnapshot($target, $snapshotId, note: $note);
}
/**
* Write to the catalog next to a running backup or restore, which only takes it for a moment to
* commit. Not during a cleanup, which holds a repository opened earlier and writes from that.
*
* @param callable(Repository): bool $change
*/
private function changeCatalog(Target $target, callable $change): bool {
try {
return $this->leases->with(self::lockKey($target), LeaseService::SHARED, 300,
fn (): bool => $this->leases->with(self::catalogLockKey($target), LeaseService::EXCLUSIVE, 300,
fn (): bool => $change($this->targets->repository($target))));
} catch (LockedException) {
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' or another change to its restore points is in progress; try again later');
}
}
public function untrash(Target $target, string $snapshotId): void { public function untrash(Target $target, string $snapshotId): void {
$lease = $this->lockExclusive($target, 300); $lease = $this->lockExclusive($target, 300);
try { try {
@@ -87,6 +136,7 @@ final class PruneService {
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now'))); $s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
$s->setFiles((int)($meta['stats']['files'] ?? 0)); $s->setFiles((int)($meta['stats']['files'] ?? 0));
$s->setBytes((int)($meta['stats']['bytes'] ?? 0)); $s->setBytes((int)($meta['stats']['bytes'] ?? 0));
$s->setNote($repo->snapshotNotes()[$snapshotId] ?? null);
try { try {
$this->snapshots->insert($s); $this->snapshots->insert($s);
} catch (\Throwable) { } catch (\Throwable) {
@@ -208,7 +258,7 @@ final class PruneService {
$meta = $repo->snapshotMeta($id); $meta = $repo->snapshotMeta($id);
$times[$id] = (int)strtotime((string)($meta['time'] ?? '@0')); $times[$id] = (int)strtotime((string)($meta['time'] ?? '@0'));
} }
$keep = $this->policy()->keep($times); $keep = $this->policy()->keepWithPinned($times, $repo->pinnedSnapshots());
$forget = array_values(array_diff(array_keys($times), array_keys($keep))); $forget = array_values(array_diff(array_keys($times), array_keys($keep)));
$actor = $scheduled ? 'retention policy' : $this->alerts->actor(); $actor = $scheduled ? 'retention policy' : $this->alerts->actor();
$purged = []; $purged = [];
+18
View File
@@ -155,6 +155,17 @@ final class RestoreService {
$dest = $this->ensureFolder($userFolder, $parentRel); $dest = $this->ensureFolder($userFolder, $parentRel);
$names = $single ? [basename($rel) => $entries['']] : $entries; $names = $single ? [basename($rel) => $entries['']] : $entries;
// The folders with something in them that this restore creates get their time back; one that
// was already there keeps its own, as its content changes now. Noted before anything is written.
/** @var array<array-key, int> $newFolders a folder called "2026" is an integer key */
$newFolders = [];
foreach ($single ? [] : $repo->folderTimes($snapshotId) as [$dir, $mtime]) {
$relative = $dir === $source ? '' : (str_starts_with($dir, $source . '/') ? substr($dir, strlen($source) + 1) : null);
if ($relative !== null && ($relative === '' ? !$destExisted : !$dest->nodeExists($relative))) {
$newFolders[$relative] = $mtime;
}
}
$this->cleaned = []; $this->cleaned = [];
$restored = 0; $restored = 0;
foreach ($names as $relPath => $entry) { foreach ($names as $relPath => $entry) {
@@ -190,6 +201,13 @@ final class RestoreService {
if ($mode === self::MODE_REPLACE && !$single) { if ($mode === self::MODE_REPLACE && !$single) {
$trashed = $this->trashExtras($dest, '', array_flip(array_keys($names)) + $emptyDirs); $trashed = $this->trashExtras($dest, '', array_flip(array_keys($names)) + $emptyDirs);
} }
// Last and deepest first: a change in a folder also touches the folders above it.
foreach (array_reverse($newFolders, true) as $dir => $mtime) {
$this->ensureFolder($dest, (string)$dir)->touch($mtime);
if ($progress !== null) {
$progress($restored, count($names), (string)$dir); // keeps the lease alive too
}
}
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())]; return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
} }
+13
View File
@@ -28,6 +28,19 @@ final class RetentionPolicy {
return ['last' => $this->last, 'daily' => $this->daily, 'weekly' => $this->weekly, 'monthly' => $this->monthly]; return ['last' => $this->last, 'daily' => $this->daily, 'weekly' => $this->weekly, 'monthly' => $this->monthly];
} }
/**
* keep() for the snapshots that are not pinned, plus the pinned ones. A pinned snapshot does not
* take the place of a daily, weekly or monthly one.
*
* @param array<string, int> $snapshots id => unix time
* @param list<string> $pinned
* @return array<string, list<string>>
*/
public function keepWithPinned(array $snapshots, array $pinned): array {
$pins = array_intersect_key($snapshots, array_flip($pinned));
return $this->keep(array_diff_key($snapshots, $pins)) + array_fill_keys(array_keys($pins), ['pinned']);
}
/** /**
* @param array<string, int> $snapshots id => unix time * @param array<string, int> $snapshots id => unix time
* @return array<string, list<string>> id => reasons it is kept (ids not in the result are removed) * @return array<string, list<string>> id => reasons it is kept (ids not in the result are removed)
+8 -25
View File
@@ -13,10 +13,11 @@ use OCP\IConfig;
use OCP\IGroupManager; use OCP\IGroupManager;
/** /**
* The daily backup time: "HH:MM" in a time zone. Nextcloud runs PHP in UTC, so a time without a * The time zone times are shown in, and the daily time from before each location had its own
* zone meant UTC (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone * schedule (see LocationSchedule). Nextcloud runs PHP in UTC, so a time without a zone meant UTC
* is converted once to the zone of the server or of an administrator, at today's offset: it then * (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone is converted once
* shows as it effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer). * to the zone of the server or of an administrator, at today's offset: it then shows as it
* effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
*/ */
final class Schedule { final class Schedule {
public const TIME_KEY = 'schedule_time'; public const TIME_KEY = 'schedule_time';
@@ -44,27 +45,9 @@ final class Schedule {
return ['time' => $time, 'timezone' => $zone]; return ['time' => $time, 'timezone' => $zone];
} }
/** @throws \InvalidArgumentException for a malformed time or an unknown zone */ /** @return non-empty-string the zone of a location's schedule, or else this one */
public function set(string $time, string $zone): void { public function zoneOr(?string $zone): string {
if ($time !== '' && !self::validTime($time)) { return $zone !== null && self::validZone($zone) ? $zone : $this->get()['timezone'];
throw new \InvalidArgumentException('Use HH:MM or empty');
}
if (!self::validZone($zone)) {
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
}
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
}
/** Today's scheduled moment as a Unix time, in the schedule's zone; null without a schedule. */
public function todayAt(?int $now = null): ?int {
['time' => $time, 'timezone' => $zone] = $this->get();
if ($time === '' || !self::validTime($time)) {
return null;
}
$tz = new \DateTimeZone($zone);
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone($tz)->format('Y-m-d');
return (new \DateTimeImmutable($day . ' ' . $time, $tz))->getTimestamp();
} }
/** /**
+46
View File
@@ -35,6 +35,7 @@ final class TargetService {
private \OCA\NgBackup\Db\DbIndexCache $indexCache, private \OCA\NgBackup\Db\DbIndexCache $indexCache,
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor, private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
private SnapshotMapper $snapshots, private SnapshotMapper $snapshots,
private AlertService $alerts,
) { ) {
} }
@@ -78,6 +79,11 @@ final class TargetService {
$target->setBasePath(trim($basePath, '/')); $target->setBasePath(trim($basePath, '/'));
$target->setRepositoryId($repo->id()); $target->setRepositoryId($repo->id());
$target->setCreatedAt(time()); $target->setCreatedAt(time());
// Like the location before it: a backup scheme is usually one scheme.
$first = $this->mapper->findAll()[0] ?? null;
$target->setSchedule($first?->getSchedule() ?? '');
$target->setScheduleZone($first?->getScheduleZone());
$target->setScheduleSince(time());
$target = $this->mapper->insert($target); $target = $this->mapper->insert($target);
if (!$created) { if (!$created) {
// Disaster recovery: occ backup:list reads this local cache, not the repository // Disaster recovery: occ backup:list reads this local cache, not the repository
@@ -208,7 +214,25 @@ final class TargetService {
return $result; return $result;
} }
/** Update the local copy of a snapshot's note or pin, if the snapshot is listed here at all. */
public function syncSnapshot(Target $target, string $snapshotId, ?string $note = null, ?bool $pinned = null): void {
try {
$s = $this->snapshots->findOne($target->getId(), $snapshotId);
} catch (DoesNotExistException) {
return;
}
if ($note !== null) {
$s->setNote($note !== '' ? $note : null);
}
if ($pinned !== null) {
$s->setPinned($pinned);
}
$this->snapshots->update($s);
}
private function adoptSnapshots(Target $target, Repository $repo): void { private function adoptSnapshots(Target $target, Repository $repo): void {
$notes = $repo->snapshotNotes();
$pinned = array_flip($repo->pinnedSnapshots());
foreach ($repo->listSnapshots() as $snapshotId) { foreach ($repo->listSnapshots() as $snapshotId) {
try { try {
$this->snapshots->findOne($target->getId(), $snapshotId); $this->snapshots->findOne($target->getId(), $snapshotId);
@@ -227,6 +251,8 @@ final class TargetService {
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now'))); $s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
$s->setFiles((int)($meta['stats']['files'] ?? 0)); $s->setFiles((int)($meta['stats']['files'] ?? 0));
$s->setBytes((int)($meta['stats']['bytes'] ?? 0)); $s->setBytes((int)($meta['stats']['bytes'] ?? 0));
$s->setNote($notes[$snapshotId] ?? null);
$s->setPinned(isset($pinned[$snapshotId]));
$this->snapshots->insert($s); $this->snapshots->insert($s);
} catch (\Throwable $e) { } catch (\Throwable $e) {
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: snapshot ' . $snapshotId . ' on ' . $target->getName() . ' not listed: ' . $e->getMessage(), ['exception' => $e]); Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: snapshot ' . $snapshotId . ' on ' . $target->getName() . ' not listed: ' . $e->getMessage(), ['exception' => $e]);
@@ -248,6 +274,26 @@ final class TargetService {
} }
/** Size limit for the data stored on this location (null = none). */ /** Size limit for the data stored on this location (null = none). */
/**
* @param string $schedule see LocationSchedule
* @throws \InvalidArgumentException
*/
public function setSchedule(Target $target, string $schedule, string $zone): Target {
if (!Schedule::validZone($zone)) {
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
}
$new = LocationSchedule::normalize($schedule);
$turnedOff = $new === '' && (string)$target->getSchedule() !== '';
$target->setSchedule($new);
$target->setScheduleZone($zone);
$target->setScheduleSince(time());
$target = $this->mapper->update($target);
if ($turnedOff) {
$this->alerts->securityEvent('schedule_off', ['target' => $target->getName()]);
}
return $target;
}
public function setMaxBytes(Target $target, ?int $maxBytes): Target { public function setMaxBytes(Target $target, ?int $maxBytes): Target {
$target->setMaxBytes($maxBytes); $target->setMaxBytes($maxBytes);
return $this->mapper->update($target); return $this->mapper->update($target);
+16
View File
@@ -300,6 +300,22 @@
<code><![CDATA[$t]]></code> <code><![CDATA[$t]]></code>
</UndefinedDocblockClass> </UndefinedDocblockClass>
</file> </file>
<file src="lib/Migration/Version000008Date20261107010000.php">
<UndefinedDocblockClass>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
</UndefinedDocblockClass>
</file>
<file src="lib/Migration/Version000007Date20261107000000.php">
<UndefinedDocblockClass>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
<code><![CDATA[$t]]></code>
</UndefinedDocblockClass>
</file>
<file src="lib/Migration/Version000005Date20261002040000.php"> <file src="lib/Migration/Version000005Date20261002040000.php">
<UndefinedDocblockClass> <UndefinedDocblockClass>
<code><![CDATA[$t]]></code> <code><![CDATA[$t]]></code>
+3 -1
View File
@@ -34,8 +34,10 @@ export default {
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }), setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }), startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
tick: () => call('post', '/tick'), tick: () => call('post', '/tick'),
setSchedule: (time, timezone) => call('put', '/schedule', { time, timezone }), setTargetSchedule: (id, schedule, timezone) => call('put', `/targets/${id}/schedule`, { schedule, timezone }),
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`), snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
setSnapshotNote: (targetId, snapshotId, note) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/note`, { note }),
setSnapshotPinned: (targetId, snapshotId, pinned) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/pin`, { pinned }),
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }), browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }), startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
} }
+8 -24
View File
@@ -5,10 +5,7 @@
<template> <template>
<NcSettingsSection :name="t('ng_backup', 'Backups')" <NcSettingsSection :name="t('ng_backup', 'Backups')"
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')"> :description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
<div class="ngb-row"> <LocationScheduleRow v-for="target in status.targets" :key="target.id" :target="target" @changed="emit('changed')" @error="error = $event" />
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, {zone}, empty = off)', { zone: browserZone })" />
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
</div>
<div class="ngb-row"> <div class="ngb-row">
<NcTextField v-model="label" class="ngb-label" :maxlength="100" :label="t('ng_backup', 'Label (optional)')" <NcTextField v-model="label" class="ngb-label" :maxlength="100" :label="t('ng_backup', 'Label (optional)')"
:placeholder="t('ng_backup', 'e.g. before the upgrade to Nextcloud 35')" /> :placeholder="t('ng_backup', 'e.g. before the upgrade to Nextcloud 35')" />
@@ -37,7 +34,7 @@
<td>{{ formatDateTime(run.startedAt) }}</td> <td>{{ formatDateTime(run.startedAt) }}</td>
<td>{{ targetName(run.targetId) }}</td> <td>{{ targetName(run.targetId) }}</td>
<td>{{ run.kind }}</td> <td>{{ run.kind }}</td>
<td>{{ run.snapshot ? formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}</td> <td>{{ run.snapshot ? (run.snapshot.pinned ? '📌 ' : '') + formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') + (run.snapshot.note ? ' – ' + run.snapshot.note : '') : '' }}</td>
<td :class="'ngb-' + run.status">{{ runStatus(run) }}{{ run.error ? ': ' + run.error : '' }}</td> <td :class="'ngb-' + run.status">{{ runStatus(run) }}{{ run.error ? ': ' + run.error : '' }}</td>
<td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td> <td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
<td>{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td> <td>{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td>
@@ -60,24 +57,19 @@
</template> </template>
<script setup> <script setup>
import { ref, watch } from 'vue' import { ref } from 'vue'
import { n, t } from '@nextcloud/l10n' import { n, t } from '@nextcloud/l10n'
import NcButton from '@nextcloud/vue/components/NcButton' import NcButton from '@nextcloud/vue/components/NcButton'
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard' import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection' import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
import NcTextField from '@nextcloud/vue/components/NcTextField' import NcTextField from '@nextcloud/vue/components/NcTextField'
import LocationScheduleRow from './LocationScheduleRow.vue'
import api from '../api.js' import api from '../api.js'
import { formatDateTime, formatSize, toZone } from '../format.js' import { formatDateTime, formatSize } from '../format.js'
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } }) const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
const emit = defineEmits(['changed', 'show-all-runs']) const emit = defineEmits(['changed', 'show-all-runs'])
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
// zone is shown in this one, so it reads as the moment it runs here.
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
const shown = (s) => (s && props.status.scheduleTimezone && props.status.scheduleTimezone !== browserZone) ? toZone(s, props.status.scheduleTimezone, browserZone) : s
const schedule = ref(shown(props.status.schedule))
const error = ref('') const error = ref('')
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
const label = ref('') const label = ref('')
@@ -97,6 +89,9 @@ const runStatus = (run) => {
if (run.status !== 'running') { if (run.status !== 'running') {
return run.status return run.status
} }
if (run.queued) {
return t('ng_backup', 'queued, after the backup that is running now')
}
const detail = run.phase + percent(run) const detail = run.phase + percent(run)
if (run.waitingFor) { if (run.waitingFor) {
return t('ng_backup', 'waiting for background jobs ({detail}, last step {minutes} min ago)', { detail, minutes: Math.floor(run.waitingFor / 60) }) return t('ng_backup', 'waiting for background jobs ({detail}, last step {minutes} min ago)', { detail, minutes: Math.floor(run.waitingFor / 60) })
@@ -116,21 +111,10 @@ async function start(target) {
error.value = e.message error.value = e.message
} }
} }
async function saveSchedule() {
error.value = ''
try {
await api.setSchedule(schedule.value.trim(), browserZone)
emit('changed')
} catch (e) {
error.value = e.message
}
}
</script> </script>
<style scoped> <style scoped>
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; } .ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
.ngb-time { max-width: 260px; }
.ngb-label { max-width: 360px; } .ngb-label { max-width: 360px; }
.ngb-table-wrap { overflow-x: auto; } .ngb-table-wrap { overflow-x: auto; }
.ngb-table { width: 100%; border-collapse: collapse; } .ngb-table { width: 100%; border-collapse: collapse; }
+77
View File
@@ -0,0 +1,77 @@
<!--
- SPDX-FileCopyrightText: 2026 Dillard Blom
- SPDX-License-Identifier: AGPL-3.0-or-later
-->
<template>
<div class="ngb-row">
<strong class="ngb-name">{{ target.name }}</strong>
<NcSelect v-model="kind" :options="kinds" label="label" :input-label="t('ng_backup', 'Schedule')" :clearable="false" />
<NcTextField v-if="kind.id !== 'off'" v-model="time" class="ngb-time" :label="kind.id === 'daily' ? t('ng_backup', 'At (HH:MM, {zone})', { zone: browserZone }) : t('ng_backup', 'From (HH:MM, {zone})', { zone: browserZone })" />
<NcTextField v-if="kind.id === 'hours'" v-model="hours" class="ngb-hours" type="number" min="1" max="23" :label="t('ng_backup', 'Every … hours')" />
<NcButton :disabled="busy || wanted === current" @click="save">{{ t('ng_backup', 'Save') }}</NcButton>
</div>
</template>
<script setup>
import { computed, ref, watch } from 'vue'
import { t } from '@nextcloud/l10n'
import NcButton from '@nextcloud/vue/components/NcButton'
import NcSelect from '@nextcloud/vue/components/NcSelect'
import NcTextField from '@nextcloud/vue/components/NcTextField'
import api from '../api.js'
import { toZone } from '../format.js'
const props = defineProps({ target: { type: Object, required: true } })
const emit = defineEmits(['changed', 'error'])
const kinds = [
{ id: 'off', label: t('ng_backup', 'Off') },
{ id: 'daily', label: t('ng_backup', 'Once a day') },
{ id: 'hours', label: t('ng_backup', 'Every few hours') },
]
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
// zone is shown in this one, so it reads as the moment it runs here.
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
const kind = ref(kinds[0])
const time = ref('02:00')
const hours = ref('6')
const busy = ref(false)
const current = computed(() => {
const [at, every] = props.target.schedule.split('/')
return at === '' ? '' : (props.target.scheduleZone !== browserZone ? toZone(at, props.target.scheduleZone, browserZone) : at) + (every ? '/' + every : '')
})
const wanted = computed(() => {
if (kind.value.id === 'off') {
return ''
}
return time.value.trim() + (kind.value.id === 'hours' ? '/' + String(hours.value).trim() : '')
})
watch(current, (schedule) => {
const [at, every] = schedule.split('/')
kind.value = kinds[at === '' ? 0 : (every ? 2 : 1)]
time.value = at === '' ? time.value : at
hours.value = every ?? hours.value
}, { immediate: true })
async function save() {
busy.value = true
emit('error', '')
try {
await api.setTargetSchedule(props.target.id, wanted.value, browserZone)
emit('changed')
} catch (e) {
emit('error', e.message)
} finally {
busy.value = false
}
}
</script>
<style scoped>
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
.ngb-name { min-width: 120px; align-self: center; }
.ngb-time { max-width: 220px; }
.ngb-hours { max-width: 120px; }
</style>
+37 -2
View File
@@ -11,6 +11,13 @@
</div> </div>
<template v-if="snapshot"> <template v-if="snapshot">
<div class="ngb-row ngb-marks">
<NcTextField v-model="note" class="ngb-note" :maxlength="500" :label="t('ng_backup', 'Note (can be changed later)')" />
<NcButton :disabled="marking || note.trim() === (snapshot.note ?? '')" @click="saveNote">{{ t('ng_backup', 'Save note') }}</NcButton>
<NcCheckboxRadioSwitch :model-value="snapshot.pinned" :disabled="marking" @update:model-value="savePinned">
{{ t('ng_backup', 'Pin: the retention policy does not remove this restore point') }}
</NcCheckboxRadioSwitch>
</div>
<div class="ngb-crumbs"> <div class="ngb-crumbs">
<NcButton variant="tertiary" @click="open('data')">data</NcButton> <NcButton variant="tertiary" @click="open('data')">data</NcButton>
<template v-for="(crumb, i) in crumbs" :key="crumb.path"> <template v-for="(crumb, i) in crumbs" :key="crumb.path">
@@ -49,6 +56,7 @@ import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard' import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
import NcSelect from '@nextcloud/vue/components/NcSelect' import NcSelect from '@nextcloud/vue/components/NcSelect'
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection' import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
import NcTextField from '@nextcloud/vue/components/NcTextField'
import { confirmPassword } from '@nextcloud/password-confirmation' import { confirmPassword } from '@nextcloud/password-confirmation'
import '@nextcloud/password-confirmation/style.css' import '@nextcloud/password-confirmation/style.css'
import api from '../api.js' import api from '../api.js'
@@ -74,7 +82,10 @@ const modes = [
{ id: 'replace', label: t('ng_backup', 'Into the original place, overwrite and move files that are not in the backup to the trash') }, { id: 'replace', label: t('ng_backup', 'Into the original place, overwrite and move files that are not in the backup to the trash') },
] ]
const snapshotLabel = (s) => formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '') const note = ref('')
const marking = ref(false)
const snapshotLabel = (s) => (s.pinned ? '📌 ' : '') + formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '') + (s.note ? ' – ' + s.note : '')
const join = (name) => path.value + '/' + name const join = (name) => path.value + '/' + name
const restorable = (p) => /^data\/[^/]+\/files(\/.*)?$/.test(p) const restorable = (p) => /^data\/[^/]+\/files(\/.*)?$/.test(p)
const crumbs = computed(() => { const crumbs = computed(() => {
@@ -88,7 +99,29 @@ watch(target, async (tg) => {
snapshots.value = list.map(s => ({ ...s, displayLabel: snapshotLabel(s) })) snapshots.value = list.map(s => ({ ...s, displayLabel: snapshotLabel(s) }))
snapshot.value = snapshots.value[0] ?? null snapshot.value = snapshots.value[0] ?? null
}, { immediate: true }) }, { immediate: true })
watch(snapshot, () => { selected.value = ''; open('data') }) watch(snapshot, (s) => { selected.value = ''; note.value = s?.note ?? ''; open('data') })
// Changes the snapshot in place: choosing another one, which opens its folders again, is not needed
async function mark(call, change) {
const s = snapshot.value
marking.value = true
error.value = message.value = ''
try {
await call(target.value.id, s.id)
change(s)
s.displayLabel = snapshotLabel(s)
emit('changed')
} catch (e) {
error.value = e.message
} finally {
marking.value = false
}
}
const saveNote = () => {
const text = note.value.trim()
return mark((targetId, id) => api.setSnapshotNote(targetId, id, text), (s) => { s.note = text })
}
const savePinned = (pinned) => mark((targetId, id) => api.setSnapshotPinned(targetId, id, pinned), (s) => { s.pinned = pinned })
async function open(p) { async function open(p) {
if (!snapshot.value) { if (!snapshot.value) {
@@ -125,6 +158,8 @@ async function restore() {
<style scoped> <style scoped>
.ngb-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 12px; } .ngb-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 12px; }
.ngb-row :deep(.ngb-snapshot) { min-width: min(360px, 100%); } .ngb-row :deep(.ngb-snapshot) { min-width: min(360px, 100%); }
.ngb-marks { align-items: flex-end; }
.ngb-note { max-width: 360px; }
.ngb-crumbs { display: flex; align-items: center; flex-wrap: wrap; } .ngb-crumbs { display: flex; align-items: center; flex-wrap: wrap; }
.ngb-entries li { display: flex; align-items: center; gap: 12px; min-height: 36px; } .ngb-entries li { display: flex; align-items: center; gap: 12px; min-height: 36px; }
.ngb-entries li.selected { background: var(--color-primary-element-light); border-radius: var(--border-radius); } .ngb-entries li.selected { background: var(--color-primary-element-light); border-radius: var(--border-radius); }
+5 -1
View File
@@ -82,7 +82,11 @@ class EmptyDirsTest extends TestCase {
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR); $state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
} }
$this->assertSame('commit', $state['phase']); $this->assertSame('commit', $state['phase']);
$this->assertSame(4, $state['stats']['unreadable'], 'each reported once, over several steps'); $this->assertSame(4, $state['stats']['badNames'], 'each reported once, over several steps');
$this->assertArrayNotHasKey('unreadable', $state['stats'], 'not a problem with permissions');
$shown = BackupRun::warnings($state)['badNames'];
$this->assertContains('data/alice/files/line?break', $shown, 'no control character in a message');
$this->assertSame($shown, array_filter($shown, fn (string $name): bool => mb_check_encoding($name, 'UTF-8')));
$this->repo()->recordSnapshot($state['snapshot']); $this->repo()->recordSnapshot($state['snapshot']);
$this->assertSame(['data/alice/files/Hollow/Nested', 'data/skipped/inner'], array_column($this->repo()->emptyDirs($state['snapshot']), 0)); $this->assertSame(['data/alice/files/Hollow/Nested', 'data/skipped/inner'], array_column($this->repo()->emptyDirs($state['snapshot']), 0));
+127
View File
@@ -0,0 +1,127 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Repository;
use OCA\NgBackup\Backend\LocalBackend;
use OCA\NgBackup\Crypto\KeyRing;
use OCA\NgBackup\Job\BackupRun;
use OCA\NgBackup\Repository\Repository;
use OCA\NgBackup\Tests\Unit\TempDirTrait;
use PHPUnit\Framework\TestCase;
/** Folders with something in them get their original modification time back on a restore. */
class FolderTimesTest extends TestCase {
use TempDirTrait;
private const OLD = 1500000000;
private const OLDER = 1400000000;
private LocalBackend $backend;
private KeyRing $keys;
private string $src;
protected function setUp(): void {
$this->backend = new LocalBackend($this->tempDir());
$this->keys = KeyRing::generate();
Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
$this->src = $this->tempDir();
mkdir("$this->src/data/alice/files/Old/Deep", 0700, true);
mkdir("$this->src/data/alice/files/2026", 0700, true);
file_put_contents("$this->src/data/alice/files/Old/Deep/a.txt", 'a');
file_put_contents("$this->src/data/alice/files/Old/b.txt", 'b');
file_put_contents("$this->src/data/alice/files/2026/c.txt", 'c');
touch("$this->src/data/alice/files/Old/Deep", self::OLDER);
touch("$this->src/data/alice/files/Old", self::OLD);
touch("$this->src/data/alice/files/2026", self::OLD);
}
private function repo(): Repository {
return Repository::openWithKey($this->backend, $this->keys);
}
private function backup(float $seconds = 30.0): string {
$state = BackupRun::start(['data' => "$this->src/data"]);
for ($i = 0; $i < 500 && $state['phase'] !== 'commit'; $i++) {
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, microtime(true) + $seconds)), true);
}
$this->assertSame('commit', $state['phase']);
$this->repo()->recordSnapshot($state['snapshot']);
return $state['snapshot'];
}
private function time(string $path): int {
clearstatcache();
return (int)filemtime($path);
}
public function testEachFolderIsRecordedOnceAlsoInSmallSteps(): void {
$expected = ['data', 'data/alice', 'data/alice/files', 'data/alice/files/2026', 'data/alice/files/Old', 'data/alice/files/Old/Deep'];
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup()), 0));
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup(0.0)), 0), 'a step per entry');
}
public function testNoEmptyFolderIsAmongThem(): void {
mkdir("$this->src/data/alice/files/Hollow");
$snapshot = $this->backup();
$this->assertNotContains('data/alice/files/Hollow', array_column($this->repo()->folderTimes($snapshot), 0));
$this->assertContains('data/alice/files/Hollow', array_column($this->repo()->emptyDirs($snapshot), 0));
}
public function testARestoreBringsThemBack(): void {
$snapshot = $this->backup();
$target = $this->tempDir();
$this->repo()->restore($snapshot, $target);
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
$this->assertSame(self::OLDER, $this->time("$target/data/alice/files/Old/Deep"));
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/2026"));
}
public function testARestoreThatGoesOnAfterAStopToo(): void {
$snapshot = $this->backup();
$target = $this->tempDir();
$this->repo()->restore($snapshot, $target);
touch("$target/data/alice/files/Old", 1800000000);
$this->repo()->restore($snapshot, $target, keepUnchanged: true);
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
}
public function testAPrefixLeavesTheTargetAndOthersAlone(): void {
$snapshot = $this->backup();
$target = $this->tempDir();
$this->repo()->restore($snapshot, $target, 'data/alice/files/Old/', true);
$this->assertSame(self::OLDER, $this->time("$target/Deep"));
$this->assertNotSame(self::OLD, $this->time($target), 'the directory that was chosen is not ours to change');
$this->assertDirectoryDoesNotExist("$target/2026");
}
public function testALinkOutOfTheTargetIsRefused(): void {
$snapshot = $this->backup();
$target = $this->tempDir();
$outside = $this->tempDir();
touch($outside, 1800000000);
mkdir("$target/data/alice/files", 0700, true);
symlink($outside, "$target/data/alice/files/Old");
$this->expectException(\OCA\NgBackup\Repository\RestoreTargetException::class);
$this->repo()->restore($snapshot, $target);
}
public function testAFolderWithABadNameIsLeftOutAndWhatIsInItReported(): void {
mkdir("$this->src/data/odd\\folder");
file_put_contents("$this->src/data/odd\\folder/x", 'x');
$state = BackupRun::start(['data' => "$this->src/data"]);
for ($i = 0; $i < 50 && $state['phase'] !== 'commit'; $i++) {
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
}
$this->assertSame('commit', $state['phase']);
$this->assertSame(1, $state['stats']['badNames']);
$this->repo()->recordSnapshot($state['snapshot']);
$this->assertNotContains("data/odd\\folder", array_column($this->repo()->folderTimes($state['snapshot']), 0));
}
public function testASnapshotWithoutThemHasNone(): void {
$snapshot = $this->repo()->backupDirectory($this->src)['snapshot'];
$this->assertSame([], $this->repo()->folderTimes($snapshot));
}
}
@@ -0,0 +1,89 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Repository;
use OCA\NgBackup\Backend\LocalBackend;
use OCA\NgBackup\Crypto\KeyRing;
use OCA\NgBackup\Repository\Repository;
use OCA\NgBackup\Tests\Unit\TempDirTrait;
use PHPUnit\Framework\TestCase;
/** A pin and a note on a snapshot live in the catalog on the location, so a new server sees them too. */
class SnapshotMarksTest extends TestCase {
use TempDirTrait;
private LocalBackend $backend;
private KeyRing $keys;
private string $snapshot;
protected function setUp(): void {
$this->backend = new LocalBackend($this->tempDir());
$this->keys = KeyRing::generate();
$src = $this->tempDir();
file_put_contents("$src/a", 'x');
$this->snapshot = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'))->backupDirectory($src)['snapshot'];
}
private function repo(): Repository {
return Repository::openWithKey($this->backend, $this->keys);
}
public function testTheyAreThereAfterOpeningAgain(): void {
$this->repo()->setPinned($this->snapshot, true);
$this->repo()->setSnapshotNote($this->snapshot, 'before the upgrade');
$this->assertSame([$this->snapshot], $this->repo()->pinnedSnapshots());
$this->assertSame([$this->snapshot => 'before the upgrade'], $this->repo()->snapshotNotes());
}
public function testTheyCanBeTakenOffAgain(): void {
$this->repo()->setPinned($this->snapshot, true);
$this->repo()->setSnapshotNote($this->snapshot, 'x');
$this->repo()->setPinned($this->snapshot, false);
$this->repo()->setSnapshotNote($this->snapshot, '');
$this->assertSame([], $this->repo()->pinnedSnapshots());
$this->assertSame([], $this->repo()->snapshotNotes());
}
public function testNoChangeWritesNoCatalog(): void {
$repo = $this->repo();
$repo->setPinned($this->snapshot, true);
$generation = $repo->catalogGeneration();
$repo->setPinned($this->snapshot, true);
$repo->setSnapshotNote($this->snapshot, '');
$this->assertSame($generation, $repo->catalogGeneration());
}
public function testASnapshotThatIsNotThereIsRefused(): void {
$this->expectException(\InvalidArgumentException::class);
$this->repo()->setPinned('0123456789abcdef0123456789abcdef', true);
}
public function testATrashedSnapshotLosesItsPinAndKeepsItsNote(): void {
$repo = $this->repo();
$repo->setPinned($this->snapshot, true);
$repo->setSnapshotNote($this->snapshot, 'kept');
$repo->forget($this->snapshot);
$this->assertSame([], $this->repo()->pinnedSnapshots());
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
$repo = $this->repo();
$repo->untrash($this->snapshot);
$this->assertSame([], $this->repo()->pinnedSnapshots(), 'back in the list, as a normal one');
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
}
public function testThePurgeTakesTheNoteAlong(): void {
$repo = $this->repo();
$repo->setSnapshotNote($this->snapshot, 'kept');
$repo->forget($this->snapshot);
$repo->purgeTrash(0);
$this->assertSame([], $this->repo()->snapshotNotes());
}
public function testACatalogFromBeforeThisVersionReadsAsNone(): void {
$this->assertSame([], $this->repo()->pinnedSnapshots());
$this->assertSame([], $this->repo()->snapshotNotes());
}
}
@@ -0,0 +1,74 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Service;
use OCA\NgBackup\Service\LocationSchedule as S;
use PHPUnit\Framework\TestCase;
class LocationScheduleTest extends TestCase {
private const BERLIN = 'Europe/Berlin';
private static function at(string $time): int {
return (int)strtotime($time);
}
public function testWhatCanBeSaved(): void {
$this->assertSame('', S::normalize('off'));
$this->assertSame('', S::normalize(' '));
$this->assertSame('02:30', S::normalize('02:30'));
$this->assertSame('02:30', S::normalize('02:30/24'));
$this->assertSame('00:00/6', S::normalize('00:00/6'));
foreach (['2:30', '24:00', '02:30/0', '02:30/25', 'daily', '02:30/x'] as $bad) {
try {
S::normalize($bad);
$this->fail("$bad accepted");
} catch (\InvalidArgumentException) {
$this->addToAssertionCount(1);
}
}
}
public function testOffHasNoMoment(): void {
$this->assertNull(S::lastMoment('', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
}
public function testDailyIsInTheZoneOfTheLocation(): void {
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
$this->assertSame(self::at('2026-11-09 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 01:00 UTC')), 'not yet today');
}
public function testEveryNHoursStartsOverTheNextDay(): void {
$zone = 'UTC';
$this->assertSame(self::at('2026-11-10 12:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 13:00 UTC')));
$this->assertSame(self::at('2026-11-09 22:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 01:59 UTC')), '02, 07, 12, 17, 22, then 02 again');
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 02:00 UTC')), 'at the moment itself');
}
public function testAClockChangeInTheNightStillGivesOneMoment(): void {
$this->assertSame(self::at('2026-03-29 01:30 UTC'), S::lastMoment('02:30', self::BERLIN, self::at('2026-03-29 12:00 UTC')), '02:30 does not exist that night');
$this->assertNotNull(S::lastMoment('02:30', self::BERLIN, self::at('2026-10-25 12:00 UTC')));
}
public function testChoosingATimeDoesNotStartABackup(): void {
$moment = self::at('2026-11-10 02:00 UTC');
$this->assertFalse(S::isDue($moment, $moment + 1, null, 0, null, $moment + 3600));
$this->assertTrue(S::isDue($moment, $moment - 1, null, 0, null, $moment + 3600));
$this->assertTrue(S::isDue($moment, 0, null, 0, null, $moment), 'a location from before schedules per location');
}
public function testABackupSinceTheMomentIsEnough(): void {
$moment = 1000000;
$this->assertFalse(S::isDue($moment, 0, $moment, 1, $moment + 10, $moment + 20));
$this->assertTrue(S::isDue($moment, 0, $moment - 1, 0, null, $moment + 20), 'it started before this moment: one more');
}
public function testAfterAFailureOnlyAFewAttemptsAnHourApart(): void {
$moment = 1000000;
$finished = $moment + 60;
$this->assertFalse(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER - 1));
$this->assertTrue(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER));
$this->assertFalse(S::isDue($moment, 0, null, S::MAX_ATTEMPTS, $finished, $finished + 86400));
}
}
@@ -25,6 +25,17 @@ class RetentionPolicyTest extends TestCase {
$this->assertCount(3, $months); $this->assertCount(3, $months);
} }
public function testAPinnedSnapshotStaysAndDoesNotCount(): void {
$policy = new RetentionPolicy(1, 0, 0, 0);
$snaps = ['old' => 100, 'pinned' => 300, 'new' => 200];
$this->assertSame(['pinned'], array_keys($policy->keep($snaps)), 'without the pin it would take the place of "last"');
$keep = $policy->keepWithPinned($snaps, ['pinned', 'gone-already']);
$this->assertSame(['pinned'], $keep['pinned']);
$this->assertSame(['last'], $keep['new'], 'the newest one that is not pinned');
$this->assertArrayNotHasKey('old', $keep);
$this->assertArrayNotHasKey('gone-already', $keep);
}
public function testKeepsEverythingWhenFewSnapshots(): void { public function testKeepsEverythingWhenFewSnapshots(): void {
$keep = (new RetentionPolicy())->keep(['a' => 100, 'b' => 200]); $keep = (new RetentionPolicy())->keep(['a' => 100, 'b' => 200]);
$this->assertSame(['a', 'b'], array_values(array_intersect(['a', 'b'], array_keys($keep)))); $this->assertSame(['a', 'b'], array_values(array_intersect(['a', 'b'], array_keys($keep))));
@@ -0,0 +1,22 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Service;
use OCA\NgBackup\Service\AlertService;
use PHPUnit\Framework\TestCase;
/** A location without a schedule: a weekly reminder while another is backed up, else a daily one. */
class ScheduleOffReminderTest extends TestCase {
public function testOnceAWeekWhileAnotherLocationIsActive(): void {
$this->assertFalse(AlertService::offReminderDue(0, 6 * 86400, true));
$this->assertTrue(AlertService::offReminderDue(0, 7 * 86400, true));
}
public function testEveryDayWhenNoLocationIsActive(): void {
$this->assertFalse(AlertService::offReminderDue(0, 3600, false));
$this->assertTrue(AlertService::offReminderDue(0, 86400, false));
$this->assertTrue(AlertService::offReminderDue(0, 86400 - 60, false), 'a cron run a minute earlier than yesterday');
}
}
-11
View File
@@ -57,15 +57,4 @@ class ScheduleTest extends TestCase {
$this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get()); $this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get());
} }
public function testTodayAtIsTheMomentInTheSchedulesZone(): void {
$this->values = [Schedule::TIME_KEY => '03:00', Schedule::ZONE_KEY => 'Europe/Berlin'];
$at = $this->schedule()->todayAt(strtotime('2026-10-22 12:00 UTC'));
$this->assertSame(strtotime('2026-10-22 01:00 UTC'), $at, '03:00 in Berlin (summer time) is 01:00 UTC');
}
public function testAnUnknownZoneOrABadTimeIsRefused(): void {
$s = $this->schedule();
$this->expectException(\InvalidArgumentException::class);
$s->set('03:00', 'Mars/Olympus');
}
} }
+85 -5
View File
@@ -23,12 +23,15 @@ ok "key, kit code and confirmation"
occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add" occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add"
ok "location added" ok "location added"
# Test data: a user with files. # Test data: a user with files. No skeleton files on the first login, below, in the middle of the checks.
occ config:system:set skeletondirectory --value='' >/dev/null
occ config:system:set templatedirectory --value='' >/dev/null
$X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null" $X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
# large.bin (40 MB) fills more than one pack: a restore to a directory reads ahead over several ranges # large.bin (40 MB) fills more than one pack: a restore to a directory reads ahead over several ranges
$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs/Empty/Nested $DATA/$U/files/Hollow && touch -d @1700000000 $DATA/$U/files/Hollow && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin; head -c 40000000 /dev/urandom > $DATA/$U/files/large.bin" $X sh -c "mkdir -p $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs/Empty/Nested $DATA/$U/files/Hollow && touch -d @1700000000 $DATA/$U/files/Hollow && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin; head -c 40000000 /dev/urandom > $DATA/$U/files/large.bin"
$X sh -c "touch -d @1600000000 $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs"
occ files:scan "$U" >/dev/null occ files:scan "$U" >/dev/null
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum") SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
occ backup:run ci -l first >/dev/null || fail "first backup" occ backup:run ci -l first >/dev/null || fail "first backup"
SECOND=$(occ backup:run ci -l second) SECOND=$(occ backup:run ci -l second)
@@ -44,6 +47,14 @@ occ files:scan "$U" >/dev/null
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m new-folder >/dev/null || fail "restore new-folder" occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m new-folder >/dev/null || fail "restore new-folder"
$X sh -c "ls $DATA/$U/files | grep -q '^Restored '" || fail "no Restored folder" $X sh -c "ls $DATA/$U/files | grep -q '^Restored '" || fail "no Restored folder"
$X sh -c "test -d $DATA/$U/files/Restored\ */Docs/Empty/Nested" || fail "the empty folder is not in the Restored folder" $X sh -c "test -d $DATA/$U/files/Restored\ */Docs/Empty/Nested" || fail "the empty folder is not in the Restored folder"
# Folders with files: on disk and in what Nextcloud shows, the parent as well as the child.
RDIR=$($X sh -c "cd $DATA/$U/files && ls -d Restored\ *")
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs/Sub")" = 1600000000 ] || fail "a folder with files did not get its time back"
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs")" = 1600000000 ] || fail "the parent folder did not get its time back"
DAV=$($X curl -sf -u "$U:Roundtrip-pw-2026" -X PROPFIND -H 'Depth: 1' "http://localhost/remote.php/dav/files/$U/$(php -r 'echo rawurlencode($argv[1]);' "$RDIR")/Docs/") || fail "PROPFIND"
[ "$(echo "$DAV" | grep -o 'Sun, 13 Sep 2020 12:26:40 GMT' | wc -l)" = 2 ] || fail "Nextcloud does not show the time of the folders: $(echo "$DAV" | grep -o '<d:getlastmodified>[^<]*' | sort | uniq -c)"
occ backup:restore ci "$SNAP" "data/$U/files/Docs/Sub" -m merge >/dev/null || fail "restore into a folder that is there"
[ "$($X stat -c %Y "$DATA/$U/files/Docs/Sub")" != 1600000000 ] || fail "a folder that was already there got the old time"
occ backup:browse ci "$SNAP" "data/$U/files" | grep Hollow >/dev/null || fail "an empty folder is not shown when browsing the snapshot" occ backup:browse ci "$SNAP" "data/$U/files" | grep Hollow >/dev/null || fail "an empty folder is not shown when browsing the snapshot"
occ backup:restore ci "$SNAP" "data/$U/files/Hollow" -m merge >/dev/null || fail "restore of a folder that was empty" occ backup:restore ci "$SNAP" "data/$U/files/Hollow" -m merge >/dev/null || fail "restore of a folder that was empty"
$X sh -c "test -d $DATA/$U/files/Hollow" || fail "the empty folder did not come back by itself" $X sh -c "test -d $DATA/$U/files/Hollow" || fail "the empty folder did not come back by itself"
@@ -52,8 +63,8 @@ occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m replace >/dev/null || fail
$X sh -c "test -d $DATA/$U/files/Docs/Empty/Nested" || fail "replace did not bring back the empty folder, or moved it to the trash" $X sh -c "test -d $DATA/$U/files/Docs/Empty/Nested" || fail "replace did not bring back the empty folder, or moved it to the trash"
ok "empty folders: restored in each mode, by themselves and shown when browsing" ok "empty folders: restored in each mode, by themselves and shown when browsing"
$X sh -c "rm -rf '$DATA/$U/files/Restored '*" ; occ files:scan "$U" >/dev/null $X sh -c "rm -rf '$DATA/$U/files/Restored '*" ; occ files:scan "$U" >/dev/null
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum") SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore" [ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore: $(diff <(echo "$SUM_BEFORE") <(echo "$SUM_AFTER") | cut -c1-120)"
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash" $X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
ok "restore new-folder and replace: files identical, extra file in the trash" ok "restore new-folder and replace: files identical, extra file in the trash"
@@ -67,7 +78,7 @@ ok "restore of one file to a directory; the temporary of a stopped restore remov
# All of the user's files to a directory: over S3 and SFTP the packs are read ahead # All of the user's files to a directory: over S3 and SFTP the packs are read ahead
occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-dir >/dev/null || fail "restore of a folder to a directory" occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-dir >/dev/null || fail "restore of a folder to a directory"
SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum") SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
[ "$SUM_BEFORE" = "$SUM_DIR" ] || fail "the files restored to a directory differ" [ "$SUM_BEFORE" = "$SUM_DIR" ] || fail "the files restored to a directory differ"
$X sh -c "ls -A /tmp/ngb-rt-dir | grep -q ngb-restore-spool" && fail "the read-ahead spool was left in the directory" $X sh -c "ls -A /tmp/ngb-rt-dir | grep -q ngb-restore-spool" && fail "the read-ahead spool was left in the directory"
$X sh -c "test -d /tmp/ngb-rt-dir/data/$U/files/Docs/Empty/Nested" || fail "the empty folder is not in the directory" $X sh -c "test -d /tmp/ngb-rt-dir/data/$U/files/Docs/Empty/Nested" || fail "the empty folder is not in the directory"
@@ -101,13 +112,71 @@ ok "without system cron: warning in backup:status and the API, backup finished b
# The Bin reads in parts, as the page does (what is in the parts: CatalogTest) # The Bin reads in parts, as the page does (what is in the parts: CatalogTest)
api "$API/targets/$TID/trash" | php -r '$p = json_decode(stream_get_contents(STDIN), true); exit(is_int($p["total"] ?? null) && is_array($p["items"] ?? null) && count($p["items"]) <= $p["total"] ? 0 : 1);' || fail "the Bin API does not give a total and items" api "$API/targets/$TID/trash" | php -r '$p = json_decode(stream_get_contents(STDIN), true); exit(is_int($p["total"] ?? null) && is_array($p["items"] ?? null) && count($p["items"]) <= $p["total"] ? 0 : 1);' || fail "the Bin API does not give a total and items"
ok "the Bin API reads in parts, with a total" ok "the Bin API reads in parts, with a total"
# Locations are backed up one after the other: the second one started is queued until the first is done.
for L in 2 3; do
$X sh -c "mkdir -p /tmp/ngb-rt-target$L"
occ backup:target:add ci$L local -a null::null -o datadir=/tmp/ngb-rt-target$L --path=repo >/dev/null || fail "target add ci$L"
done
IDS=$(api "$API/status" | php -r 'foreach (json_decode(stream_get_contents(STDIN), true)["targets"] as $t) { if ($t["name"] === "ci2" || $t["name"] === "ci3") { echo $t["id"], " "; } }')
for ID in $IDS; do api -X POST -d "{\"targetId\": $ID, \"label\": \"turn\"}" "$API/backups" >/dev/null || fail "start a backup to location $ID"; done
# Never two runs to one location: a second start gets the run that is there.
ID=${IDS%% *}
RUNS=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
api -X POST -d "{\"targetId\": $ID, \"label\": \"again\"}" "$API/backups" >/dev/null || fail "second start"
AGAIN=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
[ "$RUNS" = 2 ] && [ "$AGAIN" = 2 ] || fail "a second start to one location: $RUNS, then $AGAIN running"
occ backup:status | grep "queued" >/dev/null || fail "the second backup is not queued"
QUEUED=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["queued"]));')
[ "$QUEUED" = 1 ] || fail "$QUEUED backups queued instead of 1"
for i in $(seq 1 60); do
api -X POST "$API/tick" >/dev/null || fail "tick"
RUNNING=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
[ "$RUNNING" != 0 ] || break
done
[ "$RUNNING" = 0 ] || fail "the queued backups did not finish"
occ backup:list ci3 | grep " turn " >/dev/null || fail "the queued backup made no snapshot"
ok "locations in turn: the second backup queued, and done after the first"
occ background:cron >/dev/null occ background:cron >/dev/null
$X php -f /var/www/html/cron.php >/dev/null 2>&1 || true $X php -f /var/www/html/cron.php >/dev/null 2>&1 || true
occ backup:status | grep "system cron" >/dev/null && fail "backup:status still warns after cron ran" occ backup:status | grep "system cron" >/dev/null && fail "backup:status still warns after cron ran"
sleep 8 # APCu app config cache of the web server sleep 8 # APCu app config cache of the web server
api "$API/status" | php -r 'exit(json_decode(stream_get_contents(STDIN), true)["cron"]["ok"] ? 0 : 1);' || fail "the status API still reports a missing cron after cron ran" api "$API/status" | php -r 'exit(json_decode(stream_get_contents(STDIN), true)["cron"]["ok"] ? 0 : 1);' || fail "the status API still reports a missing cron after cron ran"
ok "with a system cron that ran: no warning in backup:status or the API" ok "with a system cron that ran: no warning in backup:status or the API"
# A schedule per location. Choosing a time starts nothing; the moment itself does, and only there.
JOB=$(occ background-job:list --class='OCA\NgBackup\BackgroundJob\BackupJob' --output=json | php -r 'echo json_decode(stream_get_contents(STDIN), true)[0]["id"];')
job() { occ background-job:execute "$JOB" --force-execute >/dev/null; }
occ backup:target:schedule ci off >/dev/null
occ backup:target:schedule ci3 off >/dev/null
occ backup:target:schedule ci2 00:00 --timezone=UTC >/dev/null || fail "set a schedule"
occ backup:target:schedule ci2 | grep "00:00 (UTC)" >/dev/null || fail "schedule not shown"
job
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 0 ] || fail "choosing a time started a backup"
occ backup:target:schedule ci2 "$($X date -u -d '+1 minute' +%H:%M)/6" --timezone=UTC >/dev/null || fail "set every 6 hours"
sleep 65
job
occ backup:list ci2 | grep " scheduled " >/dev/null || fail "the scheduled moment started no backup"
[ "$(occ backup:list ci3 | grep -c ' scheduled ')" = 0 ] || fail "a location that is off was backed up"
job
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 1 ] || fail "one moment started more than one backup"
occ backup:target:schedule ci2 nonsense >/dev/null 2>&1 && fail "a bad schedule was accepted"
ok "schedule per location: nothing on choosing a time, one backup at the moment, none when off"
# What administrators are told, as Nextcloud shows it.
notes() { api "http://localhost/ocs/v2.php/apps/notifications/api/v2/notifications?format=json"; }
occ backup:target:schedule ci2 off >/dev/null
$X sh -c "touch '$DATA/ngb-rt-bad\\name.txt'"
occ backup:run ci3 -l names | grep "their name is not supported" >/dev/null || fail "occ does not report the name"
$X sh -c "rm '$DATA/ngb-rt-bad\\name.txt'"
NOTES=$(notes) || fail "read the notifications"
echo "$NOTES" | grep "turned off the schedule of backup location ci2" >/dev/null || fail "no notification about the schedule that was turned off"
echo "$NOTES" | grep "is missing because of its name" >/dev/null || fail "no notification about the name"
echo "$NOTES" | grep "Rename them" >/dev/null || fail "the notification about the name does not ask to rename"
job
notes | grep "has no schedule" >/dev/null && fail "a reminder right after turning it off"
for L in 2 3; do occ backup:target:remove ci$L >/dev/null || fail "target remove ci$L"; done
occ user:delete "$ADM" >/dev/null occ user:delete "$ADM" >/dev/null
ok "notifications: schedule turned off, name not supported, no early reminder"
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob). # Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify" occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify"
@@ -120,6 +189,17 @@ ok "database dump and restore"
# Retention, trash and catalog. # Retention, trash and catalog.
occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null
# A pinned restore point stays, and a note can change afterwards.
FIRST=$(occ backup:list ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
occ backup:pin ci "$FIRST" >/dev/null || fail "pin"
occ backup:note ci "$FIRST" "a note made later" >/dev/null || fail "note"
occ backup:list ci | grep " first " | grep " yes " | grep "a note made later" >/dev/null || fail "pin and note not listed"
occ backup:prune ci --dry-run | grep "would forget 1 " >/dev/null || fail "a pinned snapshot would be removed"
occ backup:pin ci "$FIRST" --unpin >/dev/null || fail "unpin"
occ backup:prune ci --dry-run | grep "would forget 2 " >/dev/null || fail "an unpinned snapshot would stay"
occ backup:note ci "$FIRST" >/dev/null || fail "remove note"
occ backup:list ci | grep -q "a note made later" && fail "note not removed"
ok "pin and note on a restore point"
occ backup:prune ci | grep "forgot 2" >/dev/null || fail "prune" # first and second; "page" (above) is the last occ backup:prune ci | grep "forgot 2" >/dev/null || fail "prune" # first and second; "page" (above) is the last
occ backup:trash ci | grep " first " >/dev/null || fail "trash" occ backup:trash ci | grep " first " >/dev/null || fail "trash"
TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}') TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')