Compare commits
5 Commits
v1.0.3
..
a8b5e85c9b
| Author | SHA1 | Date | |
|---|---|---|---|
| a8b5e85c9b | |||
| 36f2427e9a | |||
| e58b86fc48 | |||
| 193f65940b | |||
| 6540a9314a |
@@ -11,6 +11,32 @@ Each heading starts with that `<version>`, once: the app store shows a release t
|
||||
whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
|
||||
shown under 1.0.0. The git tag of a release candidate goes after the date.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Pin a restore point, on the restore page or with `occ backup:pin`: the retention policy leaves it
|
||||
alone, for example the one from before an upgrade. Unpinning notifies all administrators.
|
||||
- A note on a restore point that you can change afterwards, on the restore page or with
|
||||
`occ backup:note`. Pins and notes are stored encrypted on the location, so a new server sees them.
|
||||
An older version of NG Backup can still read the location, but drops them when it writes to it.
|
||||
- A schedule per location: off, once a day, or every N hours from a given time, on the page under
|
||||
Backups or with `occ backup:target:schedule`. After the update every location keeps the time that
|
||||
held for all of them; a new location starts with the schedule of the first one.
|
||||
- All folders get their original modification time back on a restore, not only the empty ones, from
|
||||
the first backup made with this version. An older version of NG Backup can still read new restore
|
||||
points. A restore through Nextcloud into a folder that is already there leaves its time alone.
|
||||
|
||||
### Changed
|
||||
- Locations are backed up one after the other: a backup that has to wait shows as queued. A restore
|
||||
point has the time its backup really began, not the time it was queued.
|
||||
- Choosing a time no longer starts a backup right away when that time has already passed today.
|
||||
- A location without a schedule no longer gets the notification that it has had no recent backup.
|
||||
Instead, all administrators are told when a schedule is turned off, and reminded once a week while
|
||||
another location is still backed up, or every day when none is.
|
||||
- A file or folder that is left out because of its name now gets a notification of its own, in the
|
||||
log and in `occ backup:run`. It says the item is missing from the backup, gives example paths and
|
||||
asks to rename it; before, it was reported as unreadable, with advice about permissions.
|
||||
|
||||
## [1.0.3] - 2026-11-06
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -472,9 +472,12 @@ in README "Known limitations":
|
||||
tijd terug. Hoort zo, vooral bij oudere verzamelingen: de maptijd wordt gebruikt om in te schatten
|
||||
wanneer een map is aangemaakt. De backup moet dan de tijd van alle mappen vastleggen en de restore
|
||||
die aan het eind terugzetten; werkt pas vanaf de eerste backup met die versie. Op de roadmap onder 1.1.0.
|
||||
**1.0.3 uitgebracht (2026-11-06)**, ook op nc01 geïnstalleerd.
|
||||
- **Branches (besluit 2026-11-06):** werk voor een versie op een branch met die naam (`1.0.3`);
|
||||
`polish` alleen voor kleine cosmetische zaken, per geval bekijken of en wanneer die gemerged
|
||||
worden. SMB staat uit op sub5 en sub6; sub6 is het testaccount, eerst aanzetten voor een SMB-test.
|
||||
Na 1.0.3 gaat het werk naar `1.1.0`. Een spoedfix die niet kan wachten gaat op `1.0.4` vanaf `main`,
|
||||
krijgt een eigen merge en release; daarna `main` in `1.1.0` mergen, zodat daar met de fix getest wordt.
|
||||
- **1.0.1** (uitgebracht 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
|
||||
1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november,
|
||||
niet aangekondigd en los van de release. Voor 1.0.2 ook: nieuwe screenshots (het labelveld bij
|
||||
@@ -569,7 +572,18 @@ in README "Known limitations":
|
||||
- **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3,
|
||||
4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan
|
||||
een paar bestanden.
|
||||
- **1.1.0:**
|
||||
- **1.1.0** (geen harde datum: een week was een voorzet, langer mag; de releasefrequentie mag omlaag,
|
||||
besluit 2026-11-06):
|
||||
**Betere melding voor namen die een snapshot niet kan bevatten** (backslash, stuurteken, geen UTF-8):
|
||||
een eigen melding in plaats van "unreadable" met het advies over rechten. Zegt dat deze bestanden of
|
||||
mappen in de backup ontbreken, geeft voorbeelden van de paden en vraagt ze te hernoemen; de volgende
|
||||
backup neemt ze mee. Alleen als Nextcloud-melding: de app mailt niet zelf, of meldingen gemaild worden
|
||||
is de keuze van de beheerder (besluit 2026-11-06).
|
||||
**Stand op branch `1.1.0` (2026-11-07):** klaar: betere melding, punten 4, 5, 6 en 8. Pin en notitie staan
|
||||
in de catalogus, met een kopie in de database voor de lijsten. Een moment van vóór het instellen van
|
||||
een schema start niets. Geen schema: geen melding "geen recente backup", wel een melding aan alle
|
||||
beheerders bij het uitzetten en een herinnering, wekelijks zolang een andere locatie een schema heeft,
|
||||
anders dagelijks (besluit 2026-11-07). Nog niet getest: de update vanaf 1.0.3, de pagina in een browser.
|
||||
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
|
||||
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
||||
`target:add` en sleutelimport (het gevoeligste pad), daarom niet in rc1. Daarna de
|
||||
@@ -601,11 +615,10 @@ in README "Known limitations":
|
||||
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
||||
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
||||
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
||||
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
|
||||
9. **Feedbackvraag weghalen in 1.2.0** (besluit 2026-11-07, niet in 1.1.0): de regel "We would like
|
||||
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
||||
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
|
||||
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
|
||||
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
|
||||
dan staan. Er komt weinig feedback bij zo'n 150 downloads per versie; geen bericht is goed
|
||||
bericht. Het formulier (`experience.yml`) in de issues blijft.
|
||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||
|
||||
@@ -293,8 +293,9 @@ On the backlog; know them before you rely on NG Backup:
|
||||
|
||||
- **Empty folders only come back from backups made with 1.0.3 or newer.** Older restore points do
|
||||
not contain them.
|
||||
- **Folders with files in them show the time of the restore**, not their original modification time.
|
||||
The files keep theirs. On the roadmap.
|
||||
- **Folders with files in them only get their original modification time back from backups made with
|
||||
1.1.0 or newer.** From older restore points they show the time of the restore; the files keep theirs.
|
||||
A restore through Nextcloud into a folder that is already there leaves that folder's time alone.
|
||||
- **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
|
||||
`config.php`) is not supported: a backup refuses to start there. Folders mounted through the
|
||||
External storage app are not in the data directory and are **not included**, without a warning.
|
||||
@@ -329,10 +330,12 @@ On the backlog; know them before you rely on NG Backup:
|
||||
| `backup:key:init`, `backup:key:kit`, `backup:key:confirm` | create the key, recovery kit, confirmation |
|
||||
| `backup:key:slots`, `backup:key:slot:add/replace/remove` | passphrase slots |
|
||||
| `backup:target:backends/add/edit/list/remove/append-only` | backup locations |
|
||||
| `backup:target:schedule` | when a location is backed up: off, daily, or every N hours |
|
||||
| `backup:run`, `backup:status`, `backup:list` | back up, status (runs of the last 7 days; `--all` for all), restore points |
|
||||
| `backup:cancel <run>` | stop a running backup |
|
||||
| `backup:browse`, `backup:restore` | browse and restore |
|
||||
| `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash |
|
||||
| `backup:pin`, `backup:note` | keep a restore point, a note you can change later |
|
||||
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
||||
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
||||
| `backup:key:import` | import a recovery kit on a fresh installation |
|
||||
|
||||
@@ -83,11 +83,14 @@ you need help with restoring a backup after a crash.]]></description>
|
||||
<command>OCA\NgBackup\Command\ListSnapshots</command>
|
||||
<command>OCA\NgBackup\Command\Browse</command>
|
||||
<command>OCA\NgBackup\Command\Restore</command>
|
||||
<command>OCA\NgBackup\Command\Pin</command>
|
||||
<command>OCA\NgBackup\Command\Note</command>
|
||||
<command>OCA\NgBackup\Command\Prune</command>
|
||||
<command>OCA\NgBackup\Command\Retention</command>
|
||||
<command>OCA\NgBackup\Command\TargetAppendOnly</command>
|
||||
<command>OCA\NgBackup\Command\TargetTrustCurrent</command>
|
||||
<command>OCA\NgBackup\Command\TargetLimit</command>
|
||||
<command>OCA\NgBackup\Command\TargetSchedule</command>
|
||||
<command>OCA\NgBackup\Command\Trash</command>
|
||||
<command>OCA\NgBackup\Command\TrashRestore</command>
|
||||
<command>OCA\NgBackup\Command\KeySlots</command>
|
||||
|
||||
+3
-1
@@ -25,8 +25,10 @@ return [
|
||||
['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'],
|
||||
['name' => 'AdminApi#tick', 'url' => '/api/tick', 'verb' => 'POST'],
|
||||
['name' => 'AdminApi#setSchedule', 'url' => '/api/schedule', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#setTargetSchedule', 'url' => '/api/targets/{id}/schedule', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'],
|
||||
['name' => 'AdminApi#setSnapshotNote', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/note', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#setSnapshotPinned', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/pin', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#browse', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/browse', 'verb' => 'GET'],
|
||||
['name' => 'AdminApi#startRestore', 'url' => '/api/restores', 'verb' => 'POST'],
|
||||
],
|
||||
|
||||
+8
-1
@@ -79,7 +79,9 @@ occ backup:list
|
||||
Later runs upload only changed data. An unchanged run uploads (almost) nothing; the output shows
|
||||
the number of uploaded bytes, so check that value when you test incremental behaviour.
|
||||
|
||||
Schedule `occ backup:run` from system cron (or use the Administration settings, if you prefer).
|
||||
Set when each location is backed up in the Administration settings, or with
|
||||
`occ backup:target:schedule offsite 02:00` (once a day), `02:00/6` (every 6 hours from 02:00) or `off`.
|
||||
Locations are backed up one after the other.
|
||||
|
||||
## 4. Check the backup regularly
|
||||
|
||||
@@ -173,8 +175,13 @@ occ backup:prune offsite --dry-run # show first
|
||||
occ backup:prune offsite # apply the policy; forgotten snapshots go to the trash first
|
||||
occ backup:trash # list the trash
|
||||
occ backup:trash:restore offsite <id>
|
||||
occ backup:pin offsite <id> # keep this restore point; --unpin to undo
|
||||
occ backup:note offsite <id> "text" # a note you can change later; without text it is removed
|
||||
```
|
||||
|
||||
The retention policy leaves a pinned restore point alone, and a pinned one does not take the place of
|
||||
a daily, weekly or monthly one. Pin and note are stored on the location, so a new server sees them too.
|
||||
|
||||
Snapshots are kept according to the policy, and their data is deleted only after the deletion delay
|
||||
(set with `backup:key:init --delete-delay-days`). `occ backup:prune offsite --dry-run` shows what would be
|
||||
removed without changing anything.
|
||||
|
||||
@@ -43,25 +43,27 @@ final class BackupJob extends TimedJob {
|
||||
$this->setTimeSensitivity(IJob::TIME_SENSITIVE);
|
||||
}
|
||||
|
||||
private const MAX_DAILY_ATTEMPTS = 3;
|
||||
|
||||
protected function run($argument): void {
|
||||
$budget = PHP_SAPI === 'cli' ? 240.0 : 20.0;
|
||||
$deadline = microtime(true) + $budget;
|
||||
$this->startScheduled();
|
||||
foreach ($this->runs->findRunning(null, BackupService::KIND_FULL) as $run) {
|
||||
// One after the other: runs that read the same disks at the same time are not faster together.
|
||||
foreach ($this->backups->runsInTurn() as $run) {
|
||||
while (microtime(true) < $deadline && $run->getStatus() === 'running') {
|
||||
$next = $this->backups->step($run, min($deadline, microtime(true) + $budget));
|
||||
if ($next === $run) {
|
||||
break; // step() hands back the same object when another process holds the run's lease
|
||||
break 2; // step() hands back the same object when another process holds the run's lease
|
||||
}
|
||||
$run = $next;
|
||||
}
|
||||
}
|
||||
$this->failStaleRestores();
|
||||
$this->weeklyPrune();
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$targets = $this->targets->list();
|
||||
foreach ($targets as $t) {
|
||||
$this->alerts->checkStale($t);
|
||||
$othersActive = array_filter($targets, static fn ($o) => $o->getId() !== $t->getId() && (string)$o->getSchedule() !== '') !== [];
|
||||
$this->alerts->checkOff($t, $othersActive);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -102,21 +104,18 @@ final class BackupJob extends TimedJob {
|
||||
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
|
||||
}
|
||||
|
||||
/** Daily schedule: "HH:MM" in the schedule's time zone (see Schedule); empty = no schedule. */
|
||||
/** Start the backups the schedules of the locations ask for; they run in turn (see LocationSchedule). */
|
||||
private function startScheduled(): void {
|
||||
$todayAt = $this->schedule->todayAt();
|
||||
if ($todayAt === null || time() < $todayAt) {
|
||||
return;
|
||||
}
|
||||
$now = time();
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$last = $this->runs->lastFinished($t->getId(), BackupService::KIND_FULL);
|
||||
if ($this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== [] || ($last !== null && $last->getStartedAt() >= $todayAt)) {
|
||||
$moment = \OCA\NgBackup\Service\LocationSchedule::lastMoment((string)$t->getSchedule(), $this->schedule->zoneOr($t->getScheduleZone()), $now);
|
||||
if ($moment === null || $this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== []) {
|
||||
continue;
|
||||
}
|
||||
// After a failure: retry at most MAX_DAILY_ATTEMPTS times a day, an hour apart, instead
|
||||
// of starting (and notifying about) a new full run every five minutes all day.
|
||||
$today = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $todayAt);
|
||||
if (count($today) >= self::MAX_DAILY_ATTEMPTS || ($today !== [] && time() - (int)$today[0]->getFinishedAt() < 3600)) {
|
||||
$attempts = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $moment);
|
||||
if (!\OCA\NgBackup\Service\LocationSchedule::isDue($moment, (int)$t->getScheduleSince(),
|
||||
$this->runs->lastFinished($t->getId(), BackupService::KIND_FULL)?->getStartedAt(),
|
||||
count($attempts), ($attempts[0] ?? null)?->getFinishedAt(), $now)) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
|
||||
@@ -40,14 +40,14 @@ final class ListSnapshots extends Command {
|
||||
foreach ($targets as $t) {
|
||||
foreach ($this->snapshots->findForTarget($t->getId()) as $s) {
|
||||
$rows[] = [$t->getName(), $s->getSnapshotId(), Format::time($s->getCreatedAt(), $zone), $s->getKind(),
|
||||
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getLabel() ?? ''];
|
||||
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getPinned() ? 'yes' : '', $s->getLabel() ?? '', $s->getNote() ?? ''];
|
||||
}
|
||||
}
|
||||
if ($rows === []) {
|
||||
$output->writeln('No snapshots yet (occ backup:run).');
|
||||
return 0;
|
||||
}
|
||||
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'label'])->setRows($rows)->render();
|
||||
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'pinned', 'label', 'note'])->setRows($rows)->render();
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\PruneService;
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class Note extends Command {
|
||||
public function __construct(
|
||||
private TargetService $targets,
|
||||
private PruneService $prune,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:note')
|
||||
->setDescription('Set or remove the note on a snapshot')
|
||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||
->addArgument('note', InputArgument::OPTIONAL, 'The note, at most ' . PruneService::NOTE_MAX . ' characters; without it the note is removed', '');
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$note = (string)$input->getArgument('note');
|
||||
try {
|
||||
$this->prune->setNote($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), $note);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln(trim($note) === '' ? 'Note removed.' : 'Note saved.');
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\PruneService;
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Input\InputOption;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class Pin extends Command {
|
||||
public function __construct(
|
||||
private TargetService $targets,
|
||||
private PruneService $prune,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:pin')
|
||||
->setDescription('Keep a snapshot: the retention policy does not remove a pinned one')
|
||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||
->addOption('unpin', null, InputOption::VALUE_NONE, 'Leave the snapshot to the retention policy again');
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$unpin = (bool)$input->getOption('unpin');
|
||||
try {
|
||||
$this->prune->setPinned($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), !$unpin);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln($unpin ? 'Snapshot unpinned; the retention policy applies to it again.' : 'Snapshot pinned; the retention policy leaves it alone.');
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -79,6 +79,9 @@ HELP);
|
||||
if (($f['unreadable'] ?? 0) > 0) {
|
||||
$output->writeln(sprintf('<comment> warning: %d unreadable files or folders are missing from this backup (see the server log)</comment>', $f['unreadable']));
|
||||
}
|
||||
if (($f['badNames'] ?? 0) > 0) {
|
||||
$output->writeln(sprintf('<comment> warning: %d files or folders are missing from this backup because their name is not supported; rename them and the next backup includes them (see the server log)</comment>', $f['badNames']));
|
||||
}
|
||||
if (($f['changing'] ?? 0) > 0) {
|
||||
$output->writeln(sprintf('<comment> warning: %d files kept changing and were saved as read (see the server log)</comment>', $f['changing']));
|
||||
}
|
||||
|
||||
@@ -31,6 +31,7 @@ final class Status extends Command {
|
||||
private VerifyService $verify,
|
||||
private Schedule $schedule,
|
||||
private CronHealth $cron,
|
||||
private \OCA\NgBackup\Service\BackupService $backups,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
@@ -61,9 +62,11 @@ final class Status extends Command {
|
||||
$rows = [];
|
||||
$all = (bool)$input->getOption('all');
|
||||
$shown = $this->runs->findShown($all ? null : time() - self::RECENT_DAYS * 86400);
|
||||
$queued = array_map(static fn (\OCA\NgBackup\Db\Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1));
|
||||
foreach ($shown as $r) {
|
||||
$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
|
||||
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus(),
|
||||
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(),
|
||||
in_array($r->getId(), $queued, true) ? 'queued' : ($r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus()),
|
||||
Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
|
||||
$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
|
||||
isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
|
||||
|
||||
@@ -31,7 +31,8 @@ final class TargetList extends Base {
|
||||
$rows = [];
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$row = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
||||
'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt())];
|
||||
'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt()),
|
||||
'schedule' => (string)$t->getSchedule() !== '' ? $t->getSchedule() : 'off'];
|
||||
if ($input->getOption('test')) {
|
||||
try {
|
||||
$repo = $this->targets->repository($t);
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\Schedule;
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Input\InputOption;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class TargetSchedule extends Command {
|
||||
public function __construct(
|
||||
private TargetService $targets,
|
||||
private Schedule $schedule,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:target:schedule')
|
||||
->setDescription('Show or set when a location is backed up')
|
||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||
->addArgument('schedule', InputArgument::OPTIONAL, '"off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM; omit to show the current one')
|
||||
->addOption('timezone', null, InputOption::VALUE_REQUIRED, 'Time zone of the time, e.g. Europe/Amsterdam');
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$t = $this->targets->get($input->getArgument('target'));
|
||||
$schedule = $input->getArgument('schedule');
|
||||
$zone = $input->getOption('timezone') ?? $this->schedule->zoneOr($t->getScheduleZone());
|
||||
if ($schedule !== null) {
|
||||
try {
|
||||
$t = $this->targets->setSchedule($t, $schedule, (string)$zone);
|
||||
} catch (\InvalidArgumentException $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
$current = (string)$t->getSchedule();
|
||||
$output->writeln($t->getName() . ': ' . ($current === '' ? 'off' : $current . ' (' . $zone . ')'));
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -59,10 +59,12 @@ final class AdminApiController extends Controller {
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
||||
'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt(), 'appendOnly' => (bool)$t->getAppendOnly(),
|
||||
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t)];
|
||||
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t),
|
||||
'schedule' => (string)$t->getSchedule(), 'scheduleZone' => $this->schedule->zoneOr($t->getScheduleZone())];
|
||||
}
|
||||
$shown = $this->runs->findShown($allRuns ? null : time() - \OCA\NgBackup\Command\Status::RECENT_DAYS * 86400);
|
||||
$runs = array_map(fn (Run $r) => $this->runJson($r), $shown);
|
||||
$queued = array_flip(array_map(static fn (Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1)));
|
||||
$runs = array_map(fn (Run $r) => $this->runJson($r) + ['queued' => isset($queued[$r->getId()])], $shown);
|
||||
return new JSONResponse([
|
||||
'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(),
|
||||
'confirmation' => $this->keys->recoveryKitConfirmation(), 'everConfirmed' => $this->keys->isInitialized() && $this->keys->everConfirmed(),
|
||||
@@ -74,8 +76,6 @@ final class AdminApiController extends Controller {
|
||||
'runs' => $runs,
|
||||
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
|
||||
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
|
||||
'schedule' => $this->schedule->get()['time'],
|
||||
'scheduleTimezone' => $this->schedule->get()['timezone'],
|
||||
'cron' => $this->cron->check(),
|
||||
]);
|
||||
}
|
||||
@@ -97,16 +97,16 @@ final class AdminApiController extends Controller {
|
||||
}
|
||||
$deadline = microtime(true) + self::TICK_SECONDS;
|
||||
$stepped = 0;
|
||||
$running = $this->runs->findRunning(null, BackupService::KIND_FULL);
|
||||
// Longest wait first: a run that takes all of a tick does not keep another from its turn.
|
||||
usort($running, static fn ($a, $b) => $a->getUpdatedAt() <=> $b->getUpdatedAt());
|
||||
foreach ($running as $run) {
|
||||
foreach ($this->backups->runsInTurn() as $run) {
|
||||
if (microtime(true) >= $deadline) {
|
||||
break;
|
||||
}
|
||||
// step() returns at once when the background job (or another tab) holds the run.
|
||||
$this->backups->step($run, $deadline);
|
||||
$next = $this->backups->step($run, $deadline);
|
||||
$stepped++;
|
||||
if ($next->getStatus() === Run::RUNNING) {
|
||||
break; // not finished: the next one waits for its turn
|
||||
}
|
||||
}
|
||||
return new JSONResponse(['stepped' => $stepped]);
|
||||
}
|
||||
@@ -278,25 +278,43 @@ final class AdminApiController extends Controller {
|
||||
return new JSONResponse($this->runJson($run));
|
||||
}
|
||||
|
||||
/** @param string $timezone the administrator's zone (the browser's); empty: keep the current one */
|
||||
public function setSchedule(string $time, string $timezone = ''): JSONResponse {
|
||||
/** @param string $timezone the administrator's zone (the browser's); empty: the default one */
|
||||
public function setTargetSchedule(int $id, string $schedule, string $timezone = ''): JSONResponse {
|
||||
try {
|
||||
$this->schedule->set($time, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
||||
} catch (\InvalidArgumentException $e) {
|
||||
$t = $this->targets->setSchedule($this->targets->get((string)$id), $schedule, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
||||
} catch (\Throwable $e) {
|
||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||
}
|
||||
return new JSONResponse(['schedule' => $time, 'scheduleTimezone' => $this->schedule->get()['timezone']]);
|
||||
return new JSONResponse(['schedule' => $t->getSchedule(), 'scheduleZone' => $t->getScheduleZone()]);
|
||||
}
|
||||
|
||||
public function snapshots(int $targetId): JSONResponse {
|
||||
$out = [];
|
||||
foreach ($this->snapshots->findForTarget($targetId) as $s) {
|
||||
$out[] = ['id' => $s->getSnapshotId(), 'kind' => $s->getKind(), 'label' => $s->getLabel(), 'createdAt' => $s->getCreatedAt(),
|
||||
'files' => $s->getFiles(), 'bytes' => $s->getBytes()];
|
||||
'files' => $s->getFiles(), 'bytes' => $s->getBytes(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
|
||||
}
|
||||
return new JSONResponse($out);
|
||||
}
|
||||
|
||||
public function setSnapshotNote(int $targetId, string $snapshotId, string $note = ''): JSONResponse {
|
||||
try {
|
||||
$this->prune->setNote($this->targets->get((string)$targetId), $snapshotId, $note);
|
||||
} catch (\Throwable $e) {
|
||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||
}
|
||||
return new JSONResponse(['ok' => true]);
|
||||
}
|
||||
|
||||
public function setSnapshotPinned(int $targetId, string $snapshotId, bool $pinned = true): JSONResponse {
|
||||
try {
|
||||
$this->prune->setPinned($this->targets->get((string)$targetId), $snapshotId, $pinned);
|
||||
} catch (\Throwable $e) {
|
||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||
}
|
||||
return new JSONResponse(['ok' => true]);
|
||||
}
|
||||
|
||||
public function browse(int $targetId, string $snapshotId, string $path = ''): JSONResponse {
|
||||
try {
|
||||
return new JSONResponse($this->restore->browse($this->targets->get((string)$targetId), $snapshotId, $path));
|
||||
@@ -341,7 +359,7 @@ final class AdminApiController extends Controller {
|
||||
if ($r->getSnapshotId() !== null) {
|
||||
try {
|
||||
$s = $this->snapshots->findOne($r->getTargetId(), $r->getSnapshotId());
|
||||
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel()];
|
||||
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
|
||||
} catch (\Throwable) {
|
||||
// Snapshot already pruned or trashed: the run keeps its id, just not the date.
|
||||
}
|
||||
|
||||
@@ -24,6 +24,10 @@ use OCP\AppFramework\Db\Entity;
|
||||
* @method void setFiles(?int $v)
|
||||
* @method int|null getBytes()
|
||||
* @method void setBytes(?int $v)
|
||||
* @method string|null getNote()
|
||||
* @method void setNote(?string $v)
|
||||
* @method bool|null getPinned()
|
||||
* @method void setPinned(?bool $v)
|
||||
*/
|
||||
final class Snapshot extends Entity {
|
||||
protected int $targetId = 0;
|
||||
@@ -33,8 +37,11 @@ final class Snapshot extends Entity {
|
||||
protected int $createdAt = 0;
|
||||
protected ?int $files = null;
|
||||
protected ?int $bytes = null;
|
||||
protected ?string $note = null;
|
||||
protected ?bool $pinned = null;
|
||||
|
||||
public function __construct() {
|
||||
$this->addType('pinned', 'boolean');
|
||||
$this->addType('targetId', 'integer');
|
||||
$this->addType('createdAt', 'integer');
|
||||
$this->addType('files', 'integer');
|
||||
|
||||
@@ -28,6 +28,12 @@ use OCP\AppFramework\Db\Entity;
|
||||
* @method void setMaxBytes(?int $v)
|
||||
* @method int getCreatedAt()
|
||||
* @method void setCreatedAt(int $v)
|
||||
* @method string|null getSchedule()
|
||||
* @method void setSchedule(?string $v)
|
||||
* @method string|null getScheduleZone()
|
||||
* @method void setScheduleZone(?string $v)
|
||||
* @method int|null getScheduleSince()
|
||||
* @method void setScheduleSince(?int $v)
|
||||
*/
|
||||
final class Target extends Entity {
|
||||
protected string $name = '';
|
||||
@@ -39,8 +45,12 @@ final class Target extends Entity {
|
||||
protected int $createdAt = 0;
|
||||
protected ?bool $appendOnly = null;
|
||||
protected ?int $maxBytes = null;
|
||||
protected ?string $schedule = null;
|
||||
protected ?string $scheduleZone = null;
|
||||
protected ?int $scheduleSince = null;
|
||||
|
||||
public function __construct() {
|
||||
$this->addType('scheduleSince', 'integer');
|
||||
$this->addType('createdAt', 'integer');
|
||||
$this->addType('appendOnly', 'boolean');
|
||||
$this->addType('maxBytes', 'integer');
|
||||
|
||||
+46
-5
@@ -52,19 +52,35 @@ final class BackupRun {
|
||||
$stats = $state['stats'];
|
||||
$walker = new TreeWalker($state['roots'], $state['exclude'], function (string $logical) use (&$state, &$stats): void {
|
||||
self::recordUnreadable($state, $stats, $logical);
|
||||
});
|
||||
}, true);
|
||||
$parent = new ParentCursor($repo, $state['parent'], $state['parentPos']);
|
||||
$lines = [];
|
||||
$emptyDirs = [];
|
||||
$folders = [];
|
||||
$finishedWalk = true;
|
||||
|
||||
foreach ($walker->walk($state['after']) as $logical => $path) {
|
||||
if (!mb_check_encoding($logical, 'UTF-8') || !Repository::isSafePath($logical)) {
|
||||
$safe = mb_check_encoding($logical, 'UTF-8') && Repository::isSafePath($logical);
|
||||
if ($walker->atFolderWithContent()) {
|
||||
// Only its time: what is in it is reported by itself, and a bad name in the path of
|
||||
// each of those is what gets reported. The cursor stays for such a name, see below.
|
||||
$mtime = $safe ? @filemtime($path) : false;
|
||||
if ($safe && $mtime !== false) {
|
||||
$folders[] = json_encode([$logical, $mtime], JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
|
||||
$state['after'] = $logical;
|
||||
if (microtime(true) >= $deadline) {
|
||||
$finishedWalk = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (!$safe) {
|
||||
// A name a snapshot cannot hold (a backslash, a control character, not UTF-8). One
|
||||
// such path must not make a backup fail, or the whole snapshot unreadable later.
|
||||
// The cursor stays where it is: the state is JSON, which cannot hold such a name. A
|
||||
// next step meets it again, and it is reported once.
|
||||
self::recordUnreadable($state, $stats, mb_scrub($logical, 'UTF-8'));
|
||||
self::recordBadName($state, $stats, $logical);
|
||||
continue;
|
||||
}
|
||||
if (is_dir($path)) {
|
||||
@@ -164,6 +180,11 @@ final class BackupRun {
|
||||
$writer->write(implode("\n", $emptyDirs) . "\n");
|
||||
$state['dirSegments'][] = $writer->finish();
|
||||
}
|
||||
if ($folders !== []) {
|
||||
$writer = $repo->blobWriter($stats);
|
||||
$writer->write(implode("\n", $folders) . "\n");
|
||||
$state['folderSegments'][] = $writer->finish();
|
||||
}
|
||||
$repo->flushPacks();
|
||||
$state['parentPos'] = $parent->position();
|
||||
$state['stats'] = $stats;
|
||||
@@ -186,10 +207,14 @@ final class BackupRun {
|
||||
$emptyDirs[] = json_decode($line, true, 8, JSON_THROW_ON_ERROR);
|
||||
}
|
||||
}
|
||||
// The times of the other folders too, packed: every look at a snapshot's meta record reads
|
||||
// this, and a string of a few hundred KiB costs that far less than tens of thousands of arrays.
|
||||
$folders = isset($state['folderSegments']) ? implode("\n", iterator_to_array($repo->readLines(array_values(array_merge(...$state['folderSegments']))), false)) : '';
|
||||
$repo->putSnapshot($state['snapshot'],
|
||||
['time' => $state['started'], 'parent' => $state['parent'], 'label' => $state['label'],
|
||||
'roots' => array_keys($state['roots']), 'stats' => $state['stats'],
|
||||
'warnings' => self::warnings($state)] + ($emptyDirs === [] ? [] : ['emptyDirs' => $emptyDirs]) + $state['meta'],
|
||||
'warnings' => self::warnings($state)] + ($emptyDirs === [] ? [] : ['emptyDirs' => $emptyDirs])
|
||||
+ ($folders === '' ? [] : ['folderTimes' => base64_encode((string)gzdeflate($folders))]) + $state['meta'],
|
||||
$repo->readLines(array_values(array_merge(...$state['segments']))));
|
||||
$state['phase'] = 'commit';
|
||||
}
|
||||
@@ -250,10 +275,26 @@ final class BackupRun {
|
||||
$stats['unreadable'] = ($stats['unreadable'] ?? 0) + 1;
|
||||
}
|
||||
|
||||
/** @return array{unreadable: list<string>, changing: list<string>} examples for the report (first 50 each) */
|
||||
/**
|
||||
* A file or directory left out because of its name. Remembered like an unreadable one, in a form
|
||||
* that fits the JSON state and is safe to show: control characters and invalid bytes replaced.
|
||||
*/
|
||||
private static function recordBadName(array &$state, array &$stats, string $logical): void {
|
||||
$shown = (string)preg_replace('/[\x00-\x1f\x7f]/', '?', mb_scrub($logical, 'UTF-8'));
|
||||
if (isset($state['badNames'][$shown])) {
|
||||
return;
|
||||
}
|
||||
if (count($state['badNames'] ?? []) < 1000) {
|
||||
$state['badNames'][$shown] = true;
|
||||
}
|
||||
$stats['badNames'] = ($stats['badNames'] ?? 0) + 1;
|
||||
}
|
||||
|
||||
/** @return array{unreadable: list<string>, badNames: list<string>, changing: list<string>} examples for the report (first 50 each) */
|
||||
public static function warnings(array $state): array {
|
||||
return [
|
||||
'unreadable' => array_slice(array_map('strval', array_keys($state['unreadable'] ?? [])), 0, 50),
|
||||
'badNames' => array_slice(array_map('strval', array_keys($state['badNames'] ?? [])), 0, 50),
|
||||
'changing' => array_slice(array_map('strval', array_values($state['changing'] ?? [])), 0, 50),
|
||||
];
|
||||
}
|
||||
|
||||
+17
-1
@@ -17,6 +17,8 @@ namespace OCA\NgBackup\Job;
|
||||
* and be merge-joined with the previous snapshot without holding any list in memory.
|
||||
*/
|
||||
final class TreeWalker {
|
||||
private bool $atFolder = false;
|
||||
|
||||
/**
|
||||
* @param array<string, string> $roots name => absolute directory, walked in this order
|
||||
* @param list<string> $exclude path prefixes ("<root>/<relative>") that are skipped entirely;
|
||||
@@ -27,9 +29,16 @@ final class TreeWalker {
|
||||
private array $exclude = [],
|
||||
/** Called with the logical path of a directory that cannot be listed. */
|
||||
private ?\Closure $onUnreadable = null,
|
||||
/** Also yield directories with something in them, right before the first thing in them. */
|
||||
private bool $withFolders = false,
|
||||
) {
|
||||
}
|
||||
|
||||
/** Whether what was just yielded is a directory with something in it (see $withFolders). */
|
||||
public function atFolderWithContent(): bool {
|
||||
return $this->atFolder;
|
||||
}
|
||||
|
||||
/** @return \Generator<string, string> logical path => absolute path, strictly after $after */
|
||||
public function walk(?string $after = null): \Generator {
|
||||
foreach ($this->roots as $name => $dir) {
|
||||
@@ -75,7 +84,14 @@ final class TreeWalker {
|
||||
if (is_link($childAbs)) {
|
||||
continue; // never follow symlinks out of the data directory
|
||||
}
|
||||
if (is_dir($childAbs)) {
|
||||
$childIsDir = is_dir($childAbs);
|
||||
if (($childIsDir || is_file($childAbs)) && $empty && !$this->isExcluded($childLogical, $childIsDir)
|
||||
&& $this->withFolders && $logical !== '' && ($after === null || self::compare($logical, $after) > 0)) {
|
||||
$this->atFolder = true;
|
||||
yield $logical => $abs;
|
||||
$this->atFolder = false;
|
||||
}
|
||||
if ($childIsDir) {
|
||||
$empty = $empty && $this->isExcluded($childLogical, true);
|
||||
// Skip a directory entirely if everything in it sorts before the cursor.
|
||||
if ($after !== null && self::compare($childLogical, $after) < 0 && !self::isPrefix($childLogical, $after)) {
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Migration;
|
||||
|
||||
use Closure;
|
||||
use OCP\DB\ISchemaWrapper;
|
||||
use OCP\DB\Types;
|
||||
use OCP\Migration\IOutput;
|
||||
use OCP\Migration\SimpleMigrationStep;
|
||||
|
||||
/** Local copy of what the catalog on the location says about a snapshot: its note and pin. */
|
||||
final class Version000007Date20261107000000 extends SimpleMigrationStep {
|
||||
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
|
||||
/** @var ISchemaWrapper $schema */
|
||||
$schema = $schemaClosure();
|
||||
$t = $schema->getTable('ngb_snapshots');
|
||||
$changed = false;
|
||||
if (!$t->hasColumn('note')) {
|
||||
$t->addColumn('note', Types::STRING, ['notnull' => false, 'length' => 1000]);
|
||||
$changed = true;
|
||||
}
|
||||
if (!$t->hasColumn('pinned')) {
|
||||
$t->addColumn('pinned', Types::BOOLEAN, ['notnull' => false, 'default' => false]);
|
||||
$changed = true;
|
||||
}
|
||||
return $changed ? $schema : null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Migration;
|
||||
|
||||
use Closure;
|
||||
use OCA\NgBackup\Service\LocationSchedule;
|
||||
use OCA\NgBackup\Service\Schedule;
|
||||
use OCP\DB\ISchemaWrapper;
|
||||
use OCP\DB\Types;
|
||||
use OCP\IDBConnection;
|
||||
use OCP\Migration\IOutput;
|
||||
use OCP\Migration\SimpleMigrationStep;
|
||||
|
||||
/** A schedule per location. Each existing location gets the daily time that held for all of them. */
|
||||
final class Version000008Date20261107010000 extends SimpleMigrationStep {
|
||||
public function __construct(
|
||||
private IDBConnection $db,
|
||||
private Schedule $schedule,
|
||||
) {
|
||||
}
|
||||
|
||||
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
|
||||
/** @var ISchemaWrapper $schema */
|
||||
$schema = $schemaClosure();
|
||||
$t = $schema->getTable('ngb_targets');
|
||||
if ($t->hasColumn('schedule')) {
|
||||
return null;
|
||||
}
|
||||
$t->addColumn('schedule', Types::STRING, ['notnull' => false, 'length' => 16]);
|
||||
$t->addColumn('schedule_zone', Types::STRING, ['notnull' => false, 'length' => 64]);
|
||||
$t->addColumn('schedule_since', Types::BIGINT, ['notnull' => false, 'unsigned' => true]);
|
||||
return $schema;
|
||||
}
|
||||
|
||||
public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options): void {
|
||||
['time' => $time, 'timezone' => $zone] = $this->schedule->get();
|
||||
$qb = $this->db->getQueryBuilder();
|
||||
$qb->update('ngb_targets')
|
||||
->set('schedule', $qb->createNamedParameter(Schedule::validTime($time) ? LocationSchedule::normalize($time) : ''))
|
||||
->set('schedule_zone', $qb->createNamedParameter($zone))
|
||||
->where($qb->expr()->isNull('schedule'));
|
||||
$qb->executeStatement();
|
||||
}
|
||||
}
|
||||
@@ -46,6 +46,10 @@ final class Notifier implements INotifier {
|
||||
$notification->setParsedSubject($l->t('Backup to %1$s finished with warnings: %2$s unreadable, %3$s changed while being read', [$p['target'], $p['unreadable'], $p['changing']]));
|
||||
$notification->setParsedMessage($l->t('Unreadable files and folders are missing from this backup (check their permissions). Files that kept changing were saved as they were while being read. Examples: %s', [$p['examples'] ?? '']));
|
||||
break;
|
||||
case 'run_bad_names':
|
||||
$notification->setParsedSubject($l->n('Backup to %s finished, but %n item is missing because of its name', 'Backup to %s finished, but %n items are missing because of their names', (int)$p['count'], [$p['target']]));
|
||||
$notification->setParsedMessage($l->t('Names with a backslash or a control character, or that are not valid UTF-8, are not supported, so these files and folders are not in this backup. Rename them; the next backup includes them. Examples: %s', [$p['examples'] ?? '']));
|
||||
break;
|
||||
case 'quota_warning':
|
||||
$notification->setParsedSubject($l->t('Backup location %1$s is at %2$s%% of its limit (%3$s of %4$s)', [$p['target'], $p['percent'], $p['used'], $p['limit']]));
|
||||
$notification->setParsedMessage($l->t('Backups stop when the limit is reached. Raise the limit, or let the retention policy remove older backups.'));
|
||||
@@ -57,13 +61,20 @@ final class Notifier implements INotifier {
|
||||
: $l->t('No successful backup to %1$s since %2$s', [$p['target'], $p['since']]));
|
||||
$notification->setParsedMessage($l->t('Check the backup settings and the server log.'));
|
||||
break;
|
||||
case 'schedule_off':
|
||||
$notification->setParsedSubject($l->t('Backup location %s has no schedule', [$p['target']]));
|
||||
$notification->setParsedMessage(($p['others'] ?? '') === '1'
|
||||
? $l->t('No backups run to this location until it gets a schedule or someone starts one. If that is what you want, you can ignore this reminder.')
|
||||
: $l->t('No location has a schedule, so no backups run at all unless someone starts one. Set a schedule under Backups.'));
|
||||
break;
|
||||
case 'security_event':
|
||||
$who = (string)($p['actor'] ?? '');
|
||||
$notification->setParsedSubject(match ($p['event'] ?? '') {
|
||||
'kit_downloaded' => $l->t('%s downloaded the backup recovery kit', [$who]),
|
||||
'target_removed' => $l->t('%1$s removed backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'target_changed' => $l->t('%1$s changed the connection settings of backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'append_only_off' => $l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'schedule_off' => $l->t('%1$s turned off the schedule of backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'append_only_off' =>$l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'retention_changed' => $l->t('%1$s changed the backup retention to %2$s', [$who, $p['policy'] ?? '']),
|
||||
'snapshots_forgotten' => $l->t('%1$s moved %2$s snapshots of %3$s to the trash; they are deleted permanently after %4$s unless restored', [$who, $p['count'] ?? '', $p['target'] ?? '', $p['until'] ?? '']),
|
||||
'snapshot_untrashed' => $l->t('%1$s restored snapshot %2$s of %3$s from the trash', [$who, $p['snapshot'] ?? '', $p['target'] ?? '']),
|
||||
|
||||
@@ -30,6 +30,10 @@ final class Catalog {
|
||||
public array $trash = [];
|
||||
/** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */
|
||||
public array $userExports = [];
|
||||
/** @var array<string, true> snapshots the retention policy leaves alone */
|
||||
public array $pinned = [];
|
||||
/** @var array<string, string> snapshot id => note, also while the snapshot is in the trash */
|
||||
public array $notes = [];
|
||||
public string $hash = '';
|
||||
|
||||
public function __construct(
|
||||
@@ -82,6 +86,8 @@ final class Catalog {
|
||||
// Catalogs written before user exports existed have no such key: default to empty rather
|
||||
// than adopting the location's current users/ listing (that would skip verification).
|
||||
$this->userExports = array_fill_keys($data['userExports'] ?? [], true);
|
||||
$this->pinned = array_fill_keys($data['pinned'] ?? [], true);
|
||||
$this->notes = $data['notes'] ?? [];
|
||||
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
|
||||
}
|
||||
|
||||
@@ -93,9 +99,12 @@ final class Catalog {
|
||||
if ($this->backend->exists($this->path($next))) {
|
||||
throw new RepositoryException("Catalog generation $next was written by another process at the same time; try again");
|
||||
}
|
||||
// A pin ends when its snapshot leaves the list, a note when the snapshot is gone for good.
|
||||
$this->pinned = array_intersect_key($this->pinned, $this->snapshots);
|
||||
$this->notes = array_intersect_key($this->notes, $this->snapshots + $this->trash);
|
||||
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
|
||||
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash),
|
||||
'userExports' => array_keys($this->userExports)], JSON_THROW_ON_ERROR);
|
||||
'userExports' => array_keys($this->userExports), 'pinned' => array_keys($this->pinned), 'notes' => $this->notes], JSON_THROW_ON_ERROR);
|
||||
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
|
||||
$stream = fopen('php://memory', 'w+b');
|
||||
fwrite($stream, $encrypted);
|
||||
|
||||
@@ -134,6 +134,49 @@ final class Repository {
|
||||
return $this->catalog->gen;
|
||||
}
|
||||
|
||||
/** @return list<string> snapshots the retention policy leaves alone */
|
||||
public function pinnedSnapshots(): array {
|
||||
return array_map('strval', array_keys($this->catalog->pinned));
|
||||
}
|
||||
|
||||
/** @return array<string, string> snapshot id => note */
|
||||
public function snapshotNotes(): array {
|
||||
return $this->catalog->notes;
|
||||
}
|
||||
|
||||
public function setPinned(string $snapshotId, bool $pinned): void {
|
||||
$this->requireSnapshot($snapshotId);
|
||||
if (isset($this->catalog->pinned[$snapshotId]) === $pinned) {
|
||||
return;
|
||||
}
|
||||
if ($pinned) {
|
||||
$this->catalog->pinned[$snapshotId] = true;
|
||||
} else {
|
||||
unset($this->catalog->pinned[$snapshotId]);
|
||||
}
|
||||
$this->catalog->write();
|
||||
}
|
||||
|
||||
/** A note kept in the catalog: unlike the label inside the snapshot, it can change afterwards. */
|
||||
public function setSnapshotNote(string $snapshotId, string $note): void {
|
||||
$this->requireSnapshot($snapshotId);
|
||||
if (($this->catalog->notes[$snapshotId] ?? '') === $note) {
|
||||
return;
|
||||
}
|
||||
if ($note === '') {
|
||||
unset($this->catalog->notes[$snapshotId]);
|
||||
} else {
|
||||
$this->catalog->notes[$snapshotId] = $note;
|
||||
}
|
||||
$this->catalog->write();
|
||||
}
|
||||
|
||||
private function requireSnapshot(string $snapshotId): void {
|
||||
if (!isset($this->catalog->snapshots[$snapshotId])) {
|
||||
throw new \InvalidArgumentException('There is no snapshot ' . $snapshotId . ' on this location');
|
||||
}
|
||||
}
|
||||
|
||||
/** Index files downloaded when this repository was opened (0 when the local cache was complete). */
|
||||
public function indexDownloads(): int {
|
||||
return $this->index->downloaded;
|
||||
@@ -392,9 +435,41 @@ final class Repository {
|
||||
self::restoreEmptyDir($target, $dest, $path, $mtime);
|
||||
}
|
||||
}
|
||||
// Very last, nothing is written in them any more. Also on a run that goes on after a stop, and
|
||||
// the deepest first. The time is not worth stopping for: all the files are there by now.
|
||||
foreach (array_reverse($this->folderTimes($snapshotId)) as [$path, $mtime]) {
|
||||
if (!self::underPrefix($path, $prefix) || ($stripPrefix && $path === rtrim($prefix, '/'))) {
|
||||
continue; // the target itself is not ours to change
|
||||
}
|
||||
if ($heartbeat !== null) {
|
||||
$heartbeat();
|
||||
}
|
||||
$dest = self::destination($target, $path, $prefix, $stripPrefix);
|
||||
@touch(self::directoryInTarget($target, $dest, $path), $mtime);
|
||||
}
|
||||
return $count;
|
||||
}
|
||||
|
||||
/**
|
||||
* The folders with something in them that a snapshot recorded, parents before children (none
|
||||
* before 1.1.0). Damage here is no reason to give up on the files of a restore.
|
||||
*
|
||||
* @return list<array{0: string, 1: int}> path, modification time
|
||||
*/
|
||||
public function folderTimes(string $snapshotId): array {
|
||||
$meta = $this->lastMeta[$snapshotId] ?? $this->snapshotMeta($snapshotId);
|
||||
$packed = is_string($meta['folderTimes'] ?? null) ? base64_decode($meta['folderTimes'], true) : false;
|
||||
$plain = $packed === false ? false : @gzinflate($packed);
|
||||
$folders = [];
|
||||
foreach ($plain === false ? [] : explode("\n", $plain) as $line) {
|
||||
$folder = json_decode($line, true);
|
||||
if (is_array($folder) && is_string($folder[0] ?? null) && self::isSafePath($folder[0]) && is_int($folder[1] ?? null)) {
|
||||
$folders[] = [$folder[0], $folder[1]];
|
||||
}
|
||||
}
|
||||
return $folders;
|
||||
}
|
||||
|
||||
/**
|
||||
* The folders without anything in them that a snapshot recorded (none before 1.0.3).
|
||||
*
|
||||
|
||||
@@ -61,6 +61,16 @@ final class AlertService {
|
||||
'examples' => mb_strimwidth(implode(', ', array_merge($examples['unreadable'], $examples['changing'])), 0, 300, '…')]);
|
||||
}
|
||||
|
||||
/**
|
||||
* A backup that finished but left out files or folders because of their name.
|
||||
*
|
||||
* @param list<string> $examples
|
||||
*/
|
||||
public function runBadNames(Run $run, string $targetName, int $count, array $examples): void {
|
||||
$this->notifyAdmins('run_bad_names', 'run-' . $run->getId(), ['target' => $targetName, 'count' => (string)$count,
|
||||
'examples' => mb_strimwidth(implode(', ', $examples), 0, 300, '…')]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Remember how much is stored on a location (for the settings page) and warn administrators
|
||||
* once a day when it is at 80% of its limit or more.
|
||||
@@ -91,9 +101,36 @@ final class AlertService {
|
||||
return $v === '' ? null : (int)$v;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether it is time to remind administrators of a location without a schedule: once a week
|
||||
* while another location is still backed up, every day when none is. An hour of margin, so
|
||||
* that the moment of the cron run does not push a reminder to the next day.
|
||||
*/
|
||||
public static function offReminderDue(int $last, int $now, bool $othersActive): bool {
|
||||
return $now - $last >= ($othersActive ? 7 * 86400 : 86400) - 3600;
|
||||
}
|
||||
|
||||
public function checkOff(Target $target, bool $othersActive): void {
|
||||
if ((string)$target->getSchedule() !== '') {
|
||||
return;
|
||||
}
|
||||
$key = 'off_notified_' . $target->getId();
|
||||
$now = time();
|
||||
$last = max((int)$target->getScheduleSince(), $this->appConfig->getValueInt(Application::APP_ID, $key, 0));
|
||||
if (!self::offReminderDue($last, $now, $othersActive)) {
|
||||
return;
|
||||
}
|
||||
$this->appConfig->setValueInt(Application::APP_ID, $key, $now);
|
||||
$this->notifyAdmins('schedule_off', 'off-' . $target->getId() . '-' . date('Y-m-d', $now),
|
||||
['target' => $target->getName(), 'others' => $othersActive ? '1' : '0']);
|
||||
}
|
||||
|
||||
/** At most one notification per location per day. */
|
||||
public function checkStale(Target $target): void {
|
||||
$last = $this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
|
||||
if ((string)$target->getSchedule() === '') {
|
||||
return; // no schedule is a choice, as for a location that is only connected now and then
|
||||
}
|
||||
$last =$this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
|
||||
$lastTime = $last?->getFinishedAt() ?? $target->getCreatedAt();
|
||||
if (time() - $lastTime < self::STALE_AFTER || $this->runs->findRunning($target->getId(), BackupService::KIND_FULL) !== []) {
|
||||
return;
|
||||
|
||||
@@ -58,6 +58,18 @@ final class BackupService {
|
||||
$this->openedWith = $opened;
|
||||
}
|
||||
|
||||
/**
|
||||
* The running backups in the order they get their turn: first come, first served. The first one
|
||||
* runs, the others are queued; only one at a time reads the disks.
|
||||
*
|
||||
* @return list<Run>
|
||||
*/
|
||||
public function runsInTurn(): array {
|
||||
$running = $this->runs->findRunning(null, self::KIND_FULL);
|
||||
usort($running, static fn (Run $a, Run $b): int => [$a->getStartedAt(), $a->getId()] <=> [$b->getStartedAt(), $b->getId()]);
|
||||
return $running;
|
||||
}
|
||||
|
||||
public function start(Target $target, string $label = ''): Run {
|
||||
$label = trim($label);
|
||||
if (mb_strlen($label) > self::LABEL_MAX) {
|
||||
@@ -210,6 +222,10 @@ final class BackupService {
|
||||
$state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
|
||||
|
||||
if ($run->getPhase() === 'db') {
|
||||
if (!isset($state['dump'])) {
|
||||
// It may have been queued for hours: the restore point has the time of its data.
|
||||
$state['files']['started'] = gmdate('c');
|
||||
}
|
||||
$dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_'));
|
||||
$dbStats = $state['dbStats'] ?? [];
|
||||
// With system cron the whole dump usually fits in one step (one transaction, fully
|
||||
@@ -253,8 +269,13 @@ final class BackupService {
|
||||
try {
|
||||
$this->alerts->storedBytes($target, $repo->storedBytes());
|
||||
$stats = $state['files']['stats'];
|
||||
$warnings = BackupRun::warnings($state['files']);
|
||||
if (($stats['badNames'] ?? 0) > 0) {
|
||||
$this->logger->warning(sprintf('NG Backup run %d to %s: %d left out because their name is not supported (rename them): %s',
|
||||
$run->getId(), $target->getName(), $stats['badNames'], implode(', ', $warnings['badNames'])));
|
||||
$this->alerts->runBadNames($run, $target->getName(), (int)$stats['badNames'], $warnings['badNames']);
|
||||
}
|
||||
if (($stats['unreadable'] ?? 0) > 0 || ($stats['changing'] ?? 0) > 0) {
|
||||
$warnings = BackupRun::warnings($state['files']);
|
||||
$this->logger->warning(sprintf('NG Backup run %d to %s: %d unreadable (missing from the snapshot): %s; %d changed while being read: %s',
|
||||
$run->getId(), $target->getName(), $stats['unreadable'] ?? 0, implode(', ', $warnings['unreadable']),
|
||||
$stats['changing'] ?? 0, implode(', ', $warnings['changing'])));
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Service;
|
||||
|
||||
/**
|
||||
* When a location is backed up: "" (off), "HH:MM" (daily) or "HH:MM/N" (from HH:MM every N hours,
|
||||
* starting over at HH:MM the next day), in the location's time zone.
|
||||
*/
|
||||
final class LocationSchedule {
|
||||
/** Attempts per moment after a failure, an hour apart, instead of one every few minutes. */
|
||||
public const MAX_ATTEMPTS = 3;
|
||||
public const RETRY_AFTER = 3600;
|
||||
|
||||
/** @throws \InvalidArgumentException */
|
||||
public static function normalize(string $schedule): string {
|
||||
$schedule = trim($schedule);
|
||||
if ($schedule === '' || strtolower($schedule) === 'off') {
|
||||
return '';
|
||||
}
|
||||
if (preg_match('/^(([01]\d|2[0-3]):[0-5]\d)(?:\/(\d{1,2}))?$/', $schedule, $m) !== 1) {
|
||||
throw new \InvalidArgumentException('Use "off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM');
|
||||
}
|
||||
$hours = (int)($m[3] ?? 24);
|
||||
if ($hours < 1 || $hours > 24) {
|
||||
throw new \InvalidArgumentException('Every 1 to 24 hours');
|
||||
}
|
||||
return $hours === 24 ? $m[1] : $m[1] . '/' . $hours;
|
||||
}
|
||||
|
||||
/**
|
||||
* The most recent scheduled moment at or before $now; null without a schedule.
|
||||
*
|
||||
* @param non-empty-string $zone
|
||||
*/
|
||||
public static function lastMoment(string $schedule, string $zone, int $now): ?int {
|
||||
if ($schedule === '') {
|
||||
return null;
|
||||
}
|
||||
$tz = new \DateTimeZone($zone);
|
||||
$today = (new \DateTimeImmutable('@' . $now))->setTimezone($tz);
|
||||
$moments = array_merge(self::moments($schedule, $today->modify('-1 day'), $tz), self::moments($schedule, $today, $tz));
|
||||
$last = null;
|
||||
foreach ($moments as $moment) {
|
||||
if ($moment <= $now && ($last === null || $moment > $last)) {
|
||||
$last = $moment;
|
||||
}
|
||||
}
|
||||
return $last;
|
||||
}
|
||||
|
||||
/** @return list<int> the moments of one day */
|
||||
private static function moments(string $schedule, \DateTimeImmutable $day, \DateTimeZone $tz): array {
|
||||
[$time, $every] = array_pad(explode('/', $schedule), 2, '24');
|
||||
[$hour, $minute] = array_map('intval', explode(':', $time));
|
||||
$moments = [];
|
||||
for ($h = $hour; $h < 24; $h += (int)$every) {
|
||||
$moments[] = (new \DateTimeImmutable($day->format('Y-m-d') . sprintf(' %02d:%02d', $h, $minute), $tz))->getTimestamp();
|
||||
}
|
||||
return $moments;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a backup has to start for $moment. One that was made since then is enough, also when
|
||||
* several moments went by during a long backup. A moment from before the schedule was set does
|
||||
* not count: choosing a time must not start a backup right away.
|
||||
*/
|
||||
public static function isDue(int $moment, int $since, ?int $lastDoneStarted, int $attempts, ?int $lastAttemptFinished, int $now): bool {
|
||||
if ($moment < $since || ($lastDoneStarted !== null && $lastDoneStarted >= $moment) || $attempts >= self::MAX_ATTEMPTS) {
|
||||
return false;
|
||||
}
|
||||
return $attempts === 0 || $now - (int)$lastAttemptFinished >= self::RETRY_AFTER;
|
||||
}
|
||||
}
|
||||
@@ -11,6 +11,7 @@ use OCA\NgBackup\AppInfo\Application;
|
||||
use OCA\NgBackup\Db\RunMapper;
|
||||
use OCA\NgBackup\Db\SnapshotMapper;
|
||||
use OCA\NgBackup\Db\Target;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCP\IAppConfig;
|
||||
use OCP\Lock\ILockingProvider;
|
||||
use OCP\Lock\LockedException;
|
||||
@@ -33,6 +34,54 @@ final class PruneService {
|
||||
) {
|
||||
}
|
||||
|
||||
/** Characters of a note on a snapshot (the column holds 1000) */
|
||||
public const NOTE_MAX = 500;
|
||||
|
||||
/** Pin or unpin a snapshot: the retention policy leaves a pinned one alone. */
|
||||
public function setPinned(Target $target, string $snapshotId, bool $pinned): void {
|
||||
$changed = $this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $pinned): bool {
|
||||
$was = in_array($snapshotId, $repo->pinnedSnapshots(), true);
|
||||
$repo->setPinned($snapshotId, $pinned);
|
||||
return $was !== $pinned;
|
||||
});
|
||||
$this->targets->syncSnapshot($target, $snapshotId, pinned: $pinned);
|
||||
if ($changed) {
|
||||
// Unpinning opens the way to removing it, so every administrator hears about that.
|
||||
$this->alerts->securityEvent($pinned ? 'snapshot_pinned' : 'snapshot_unpinned', ['target' => $target->getName(), 'snapshot' => $snapshotId], !$pinned);
|
||||
}
|
||||
}
|
||||
|
||||
public function setNote(Target $target, string $snapshotId, string $note): void {
|
||||
$note = trim((string)preg_replace('/[\x00-\x1f\x7f]+/', ' ', $note));
|
||||
if (!mb_check_encoding($note, 'UTF-8')) {
|
||||
throw new \InvalidArgumentException('A note must be valid UTF-8');
|
||||
}
|
||||
if (mb_strlen($note) > self::NOTE_MAX) {
|
||||
throw new \InvalidArgumentException('A note has at most ' . self::NOTE_MAX . ' characters');
|
||||
}
|
||||
$this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $note): bool {
|
||||
$repo->setSnapshotNote($snapshotId, $note);
|
||||
return true;
|
||||
});
|
||||
$this->targets->syncSnapshot($target, $snapshotId, note: $note);
|
||||
}
|
||||
|
||||
/**
|
||||
* Write to the catalog next to a running backup or restore, which only takes it for a moment to
|
||||
* commit. Not during a cleanup, which holds a repository opened earlier and writes from that.
|
||||
*
|
||||
* @param callable(Repository): bool $change
|
||||
*/
|
||||
private function changeCatalog(Target $target, callable $change): bool {
|
||||
try {
|
||||
return $this->leases->with(self::lockKey($target), LeaseService::SHARED, 300,
|
||||
fn (): bool => $this->leases->with(self::catalogLockKey($target), LeaseService::EXCLUSIVE, 300,
|
||||
fn (): bool => $change($this->targets->repository($target))));
|
||||
} catch (LockedException) {
|
||||
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' or another change to its restore points is in progress; try again later');
|
||||
}
|
||||
}
|
||||
|
||||
public function untrash(Target $target, string $snapshotId): void {
|
||||
$lease = $this->lockExclusive($target, 300);
|
||||
try {
|
||||
@@ -87,6 +136,7 @@ final class PruneService {
|
||||
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
||||
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
||||
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
||||
$s->setNote($repo->snapshotNotes()[$snapshotId] ?? null);
|
||||
try {
|
||||
$this->snapshots->insert($s);
|
||||
} catch (\Throwable) {
|
||||
@@ -208,7 +258,7 @@ final class PruneService {
|
||||
$meta = $repo->snapshotMeta($id);
|
||||
$times[$id] = (int)strtotime((string)($meta['time'] ?? '@0'));
|
||||
}
|
||||
$keep = $this->policy()->keep($times);
|
||||
$keep = $this->policy()->keepWithPinned($times, $repo->pinnedSnapshots());
|
||||
$forget = array_values(array_diff(array_keys($times), array_keys($keep)));
|
||||
$actor = $scheduled ? 'retention policy' : $this->alerts->actor();
|
||||
$purged = [];
|
||||
|
||||
@@ -155,6 +155,17 @@ final class RestoreService {
|
||||
$dest = $this->ensureFolder($userFolder, $parentRel);
|
||||
$names = $single ? [basename($rel) => $entries['']] : $entries;
|
||||
|
||||
// The folders with something in them that this restore creates get their time back; one that
|
||||
// was already there keeps its own, as its content changes now. Noted before anything is written.
|
||||
/** @var array<array-key, int> $newFolders a folder called "2026" is an integer key */
|
||||
$newFolders = [];
|
||||
foreach ($single ? [] : $repo->folderTimes($snapshotId) as [$dir, $mtime]) {
|
||||
$relative = $dir === $source ? '' : (str_starts_with($dir, $source . '/') ? substr($dir, strlen($source) + 1) : null);
|
||||
if ($relative !== null && ($relative === '' ? !$destExisted : !$dest->nodeExists($relative))) {
|
||||
$newFolders[$relative] = $mtime;
|
||||
}
|
||||
}
|
||||
|
||||
$this->cleaned = [];
|
||||
$restored = 0;
|
||||
foreach ($names as $relPath => $entry) {
|
||||
@@ -190,6 +201,13 @@ final class RestoreService {
|
||||
if ($mode === self::MODE_REPLACE && !$single) {
|
||||
$trashed = $this->trashExtras($dest, '', array_flip(array_keys($names)) + $emptyDirs);
|
||||
}
|
||||
// Last and deepest first: a change in a folder also touches the folders above it.
|
||||
foreach (array_reverse($newFolders, true) as $dir => $mtime) {
|
||||
$this->ensureFolder($dest, (string)$dir)->touch($mtime);
|
||||
if ($progress !== null) {
|
||||
$progress($restored, count($names), (string)$dir); // keeps the lease alive too
|
||||
}
|
||||
}
|
||||
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
|
||||
}
|
||||
|
||||
|
||||
@@ -28,6 +28,19 @@ final class RetentionPolicy {
|
||||
return ['last' => $this->last, 'daily' => $this->daily, 'weekly' => $this->weekly, 'monthly' => $this->monthly];
|
||||
}
|
||||
|
||||
/**
|
||||
* keep() for the snapshots that are not pinned, plus the pinned ones. A pinned snapshot does not
|
||||
* take the place of a daily, weekly or monthly one.
|
||||
*
|
||||
* @param array<string, int> $snapshots id => unix time
|
||||
* @param list<string> $pinned
|
||||
* @return array<string, list<string>>
|
||||
*/
|
||||
public function keepWithPinned(array $snapshots, array $pinned): array {
|
||||
$pins = array_intersect_key($snapshots, array_flip($pinned));
|
||||
return $this->keep(array_diff_key($snapshots, $pins)) + array_fill_keys(array_keys($pins), ['pinned']);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, int> $snapshots id => unix time
|
||||
* @return array<string, list<string>> id => reasons it is kept (ids not in the result are removed)
|
||||
|
||||
@@ -13,10 +13,11 @@ use OCP\IConfig;
|
||||
use OCP\IGroupManager;
|
||||
|
||||
/**
|
||||
* The daily backup time: "HH:MM" in a time zone. Nextcloud runs PHP in UTC, so a time without a
|
||||
* zone meant UTC (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone
|
||||
* is converted once to the zone of the server or of an administrator, at today's offset: it then
|
||||
* shows as it effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
||||
* The time zone times are shown in, and the daily time from before each location had its own
|
||||
* schedule (see LocationSchedule). Nextcloud runs PHP in UTC, so a time without a zone meant UTC
|
||||
* (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone is converted once
|
||||
* to the zone of the server or of an administrator, at today's offset: it then shows as it
|
||||
* effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
||||
*/
|
||||
final class Schedule {
|
||||
public const TIME_KEY = 'schedule_time';
|
||||
@@ -44,27 +45,9 @@ final class Schedule {
|
||||
return ['time' => $time, 'timezone' => $zone];
|
||||
}
|
||||
|
||||
/** @throws \InvalidArgumentException for a malformed time or an unknown zone */
|
||||
public function set(string $time, string $zone): void {
|
||||
if ($time !== '' && !self::validTime($time)) {
|
||||
throw new \InvalidArgumentException('Use HH:MM or empty');
|
||||
}
|
||||
if (!self::validZone($zone)) {
|
||||
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
||||
}
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
|
||||
}
|
||||
|
||||
/** Today's scheduled moment as a Unix time, in the schedule's zone; null without a schedule. */
|
||||
public function todayAt(?int $now = null): ?int {
|
||||
['time' => $time, 'timezone' => $zone] = $this->get();
|
||||
if ($time === '' || !self::validTime($time)) {
|
||||
return null;
|
||||
}
|
||||
$tz = new \DateTimeZone($zone);
|
||||
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone($tz)->format('Y-m-d');
|
||||
return (new \DateTimeImmutable($day . ' ' . $time, $tz))->getTimestamp();
|
||||
/** @return non-empty-string the zone of a location's schedule, or else this one */
|
||||
public function zoneOr(?string $zone): string {
|
||||
return $zone !== null && self::validZone($zone) ? $zone : $this->get()['timezone'];
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -35,6 +35,7 @@ final class TargetService {
|
||||
private \OCA\NgBackup\Db\DbIndexCache $indexCache,
|
||||
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
|
||||
private SnapshotMapper $snapshots,
|
||||
private AlertService $alerts,
|
||||
) {
|
||||
}
|
||||
|
||||
@@ -78,6 +79,11 @@ final class TargetService {
|
||||
$target->setBasePath(trim($basePath, '/'));
|
||||
$target->setRepositoryId($repo->id());
|
||||
$target->setCreatedAt(time());
|
||||
// Like the location before it: a backup scheme is usually one scheme.
|
||||
$first = $this->mapper->findAll()[0] ?? null;
|
||||
$target->setSchedule($first?->getSchedule() ?? '');
|
||||
$target->setScheduleZone($first?->getScheduleZone());
|
||||
$target->setScheduleSince(time());
|
||||
$target = $this->mapper->insert($target);
|
||||
if (!$created) {
|
||||
// Disaster recovery: occ backup:list reads this local cache, not the repository
|
||||
@@ -208,7 +214,25 @@ final class TargetService {
|
||||
return $result;
|
||||
}
|
||||
|
||||
/** Update the local copy of a snapshot's note or pin, if the snapshot is listed here at all. */
|
||||
public function syncSnapshot(Target $target, string $snapshotId, ?string $note = null, ?bool $pinned = null): void {
|
||||
try {
|
||||
$s = $this->snapshots->findOne($target->getId(), $snapshotId);
|
||||
} catch (DoesNotExistException) {
|
||||
return;
|
||||
}
|
||||
if ($note !== null) {
|
||||
$s->setNote($note !== '' ? $note : null);
|
||||
}
|
||||
if ($pinned !== null) {
|
||||
$s->setPinned($pinned);
|
||||
}
|
||||
$this->snapshots->update($s);
|
||||
}
|
||||
|
||||
private function adoptSnapshots(Target $target, Repository $repo): void {
|
||||
$notes = $repo->snapshotNotes();
|
||||
$pinned = array_flip($repo->pinnedSnapshots());
|
||||
foreach ($repo->listSnapshots() as $snapshotId) {
|
||||
try {
|
||||
$this->snapshots->findOne($target->getId(), $snapshotId);
|
||||
@@ -227,6 +251,8 @@ final class TargetService {
|
||||
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
||||
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
||||
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
||||
$s->setNote($notes[$snapshotId] ?? null);
|
||||
$s->setPinned(isset($pinned[$snapshotId]));
|
||||
$this->snapshots->insert($s);
|
||||
} catch (\Throwable $e) {
|
||||
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: snapshot ' . $snapshotId . ' on ' . $target->getName() . ' not listed: ' . $e->getMessage(), ['exception' => $e]);
|
||||
@@ -248,6 +274,26 @@ final class TargetService {
|
||||
}
|
||||
|
||||
/** Size limit for the data stored on this location (null = none). */
|
||||
/**
|
||||
* @param string $schedule see LocationSchedule
|
||||
* @throws \InvalidArgumentException
|
||||
*/
|
||||
public function setSchedule(Target $target, string $schedule, string $zone): Target {
|
||||
if (!Schedule::validZone($zone)) {
|
||||
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
||||
}
|
||||
$new = LocationSchedule::normalize($schedule);
|
||||
$turnedOff = $new === '' && (string)$target->getSchedule() !== '';
|
||||
$target->setSchedule($new);
|
||||
$target->setScheduleZone($zone);
|
||||
$target->setScheduleSince(time());
|
||||
$target = $this->mapper->update($target);
|
||||
if ($turnedOff) {
|
||||
$this->alerts->securityEvent('schedule_off', ['target' => $target->getName()]);
|
||||
}
|
||||
return $target;
|
||||
}
|
||||
|
||||
public function setMaxBytes(Target $target, ?int $maxBytes): Target {
|
||||
$target->setMaxBytes($maxBytes);
|
||||
return $this->mapper->update($target);
|
||||
|
||||
@@ -300,6 +300,22 @@
|
||||
<code><![CDATA[$t]]></code>
|
||||
</UndefinedDocblockClass>
|
||||
</file>
|
||||
<file src="lib/Migration/Version000008Date20261107010000.php">
|
||||
<UndefinedDocblockClass>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
</UndefinedDocblockClass>
|
||||
</file>
|
||||
<file src="lib/Migration/Version000007Date20261107000000.php">
|
||||
<UndefinedDocblockClass>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
</UndefinedDocblockClass>
|
||||
</file>
|
||||
<file src="lib/Migration/Version000005Date20261002040000.php">
|
||||
<UndefinedDocblockClass>
|
||||
<code><![CDATA[$t]]></code>
|
||||
|
||||
+3
-1
@@ -34,8 +34,10 @@ export default {
|
||||
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
|
||||
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
|
||||
tick: () => call('post', '/tick'),
|
||||
setSchedule: (time, timezone) => call('put', '/schedule', { time, timezone }),
|
||||
setTargetSchedule: (id, schedule, timezone) => call('put', `/targets/${id}/schedule`, { schedule, timezone }),
|
||||
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
|
||||
setSnapshotNote: (targetId, snapshotId, note) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/note`, { note }),
|
||||
setSnapshotPinned: (targetId, snapshotId, pinned) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/pin`, { pinned }),
|
||||
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
|
||||
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
|
||||
}
|
||||
|
||||
@@ -5,10 +5,7 @@
|
||||
<template>
|
||||
<NcSettingsSection :name="t('ng_backup', 'Backups')"
|
||||
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
|
||||
<div class="ngb-row">
|
||||
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, {zone}, empty = off)', { zone: browserZone })" />
|
||||
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||
</div>
|
||||
<LocationScheduleRow v-for="target in status.targets" :key="target.id" :target="target" @changed="emit('changed')" @error="error = $event" />
|
||||
<div class="ngb-row">
|
||||
<NcTextField v-model="label" class="ngb-label" :maxlength="100" :label="t('ng_backup', 'Label (optional)')"
|
||||
:placeholder="t('ng_backup', 'e.g. before the upgrade to Nextcloud 35')" />
|
||||
@@ -37,7 +34,7 @@
|
||||
<td>{{ formatDateTime(run.startedAt) }}</td>
|
||||
<td>{{ targetName(run.targetId) }}</td>
|
||||
<td>{{ run.kind }}</td>
|
||||
<td>{{ run.snapshot ? formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}</td>
|
||||
<td>{{ run.snapshot ? (run.snapshot.pinned ? '📌 ' : '') + formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') + (run.snapshot.note ? ' – ' + run.snapshot.note : '') : '' }}</td>
|
||||
<td :class="'ngb-' + run.status">{{ runStatus(run) }}{{ run.error ? ': ' + run.error : '' }}</td>
|
||||
<td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
|
||||
<td>{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td>
|
||||
@@ -60,24 +57,19 @@
|
||||
</template>
|
||||
|
||||
<script setup>
|
||||
import { ref, watch } from 'vue'
|
||||
import { ref } from 'vue'
|
||||
import { n, t } from '@nextcloud/l10n'
|
||||
import NcButton from '@nextcloud/vue/components/NcButton'
|
||||
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||
import LocationScheduleRow from './LocationScheduleRow.vue'
|
||||
import api from '../api.js'
|
||||
import { formatDateTime, formatSize, toZone } from '../format.js'
|
||||
import { formatDateTime, formatSize } from '../format.js'
|
||||
|
||||
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
|
||||
const emit = defineEmits(['changed', 'show-all-runs'])
|
||||
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
|
||||
// zone is shown in this one, so it reads as the moment it runs here.
|
||||
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
||||
const shown = (s) => (s && props.status.scheduleTimezone && props.status.scheduleTimezone !== browserZone) ? toZone(s, props.status.scheduleTimezone, browserZone) : s
|
||||
const schedule = ref(shown(props.status.schedule))
|
||||
const error = ref('')
|
||||
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
|
||||
|
||||
const label = ref('')
|
||||
|
||||
@@ -97,6 +89,9 @@ const runStatus = (run) => {
|
||||
if (run.status !== 'running') {
|
||||
return run.status
|
||||
}
|
||||
if (run.queued) {
|
||||
return t('ng_backup', 'queued, after the backup that is running now')
|
||||
}
|
||||
const detail = run.phase + percent(run)
|
||||
if (run.waitingFor) {
|
||||
return t('ng_backup', 'waiting for background jobs ({detail}, last step {minutes} min ago)', { detail, minutes: Math.floor(run.waitingFor / 60) })
|
||||
@@ -116,21 +111,10 @@ async function start(target) {
|
||||
error.value = e.message
|
||||
}
|
||||
}
|
||||
|
||||
async function saveSchedule() {
|
||||
error.value = ''
|
||||
try {
|
||||
await api.setSchedule(schedule.value.trim(), browserZone)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
error.value = e.message
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
||||
.ngb-time { max-width: 260px; }
|
||||
.ngb-label { max-width: 360px; }
|
||||
.ngb-table-wrap { overflow-x: auto; }
|
||||
.ngb-table { width: 100%; border-collapse: collapse; }
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
<!--
|
||||
- SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
- SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
-->
|
||||
<template>
|
||||
<div class="ngb-row">
|
||||
<strong class="ngb-name">{{ target.name }}</strong>
|
||||
<NcSelect v-model="kind" :options="kinds" label="label" :input-label="t('ng_backup', 'Schedule')" :clearable="false" />
|
||||
<NcTextField v-if="kind.id !== 'off'" v-model="time" class="ngb-time" :label="kind.id === 'daily' ? t('ng_backup', 'At (HH:MM, {zone})', { zone: browserZone }) : t('ng_backup', 'From (HH:MM, {zone})', { zone: browserZone })" />
|
||||
<NcTextField v-if="kind.id === 'hours'" v-model="hours" class="ngb-hours" type="number" min="1" max="23" :label="t('ng_backup', 'Every … hours')" />
|
||||
<NcButton :disabled="busy || wanted === current" @click="save">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script setup>
|
||||
import { computed, ref, watch } from 'vue'
|
||||
import { t } from '@nextcloud/l10n'
|
||||
import NcButton from '@nextcloud/vue/components/NcButton'
|
||||
import NcSelect from '@nextcloud/vue/components/NcSelect'
|
||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||
import api from '../api.js'
|
||||
import { toZone } from '../format.js'
|
||||
|
||||
const props = defineProps({ target: { type: Object, required: true } })
|
||||
const emit = defineEmits(['changed', 'error'])
|
||||
|
||||
const kinds = [
|
||||
{ id: 'off', label: t('ng_backup', 'Off') },
|
||||
{ id: 'daily', label: t('ng_backup', 'Once a day') },
|
||||
{ id: 'hours', label: t('ng_backup', 'Every few hours') },
|
||||
]
|
||||
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
|
||||
// zone is shown in this one, so it reads as the moment it runs here.
|
||||
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
||||
const kind = ref(kinds[0])
|
||||
const time = ref('02:00')
|
||||
const hours = ref('6')
|
||||
const busy = ref(false)
|
||||
|
||||
const current = computed(() => {
|
||||
const [at, every] = props.target.schedule.split('/')
|
||||
return at === '' ? '' : (props.target.scheduleZone !== browserZone ? toZone(at, props.target.scheduleZone, browserZone) : at) + (every ? '/' + every : '')
|
||||
})
|
||||
const wanted = computed(() => {
|
||||
if (kind.value.id === 'off') {
|
||||
return ''
|
||||
}
|
||||
return time.value.trim() + (kind.value.id === 'hours' ? '/' + String(hours.value).trim() : '')
|
||||
})
|
||||
|
||||
watch(current, (schedule) => {
|
||||
const [at, every] = schedule.split('/')
|
||||
kind.value = kinds[at === '' ? 0 : (every ? 2 : 1)]
|
||||
time.value = at === '' ? time.value : at
|
||||
hours.value = every ?? hours.value
|
||||
}, { immediate: true })
|
||||
|
||||
async function save() {
|
||||
busy.value = true
|
||||
emit('error', '')
|
||||
try {
|
||||
await api.setTargetSchedule(props.target.id, wanted.value, browserZone)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
emit('error', e.message)
|
||||
} finally {
|
||||
busy.value = false
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
||||
.ngb-name { min-width: 120px; align-self: center; }
|
||||
.ngb-time { max-width: 220px; }
|
||||
.ngb-hours { max-width: 120px; }
|
||||
</style>
|
||||
@@ -11,6 +11,13 @@
|
||||
</div>
|
||||
|
||||
<template v-if="snapshot">
|
||||
<div class="ngb-row ngb-marks">
|
||||
<NcTextField v-model="note" class="ngb-note" :maxlength="500" :label="t('ng_backup', 'Note (can be changed later)')" />
|
||||
<NcButton :disabled="marking || note.trim() === (snapshot.note ?? '')" @click="saveNote">{{ t('ng_backup', 'Save note') }}</NcButton>
|
||||
<NcCheckboxRadioSwitch :model-value="snapshot.pinned" :disabled="marking" @update:model-value="savePinned">
|
||||
{{ t('ng_backup', 'Pin: the retention policy does not remove this restore point') }}
|
||||
</NcCheckboxRadioSwitch>
|
||||
</div>
|
||||
<div class="ngb-crumbs">
|
||||
<NcButton variant="tertiary" @click="open('data')">data</NcButton>
|
||||
<template v-for="(crumb, i) in crumbs" :key="crumb.path">
|
||||
@@ -49,6 +56,7 @@ import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
|
||||
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||
import NcSelect from '@nextcloud/vue/components/NcSelect'
|
||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||
import { confirmPassword } from '@nextcloud/password-confirmation'
|
||||
import '@nextcloud/password-confirmation/style.css'
|
||||
import api from '../api.js'
|
||||
@@ -74,7 +82,10 @@ const modes = [
|
||||
{ id: 'replace', label: t('ng_backup', 'Into the original place, overwrite and move files that are not in the backup to the trash') },
|
||||
]
|
||||
|
||||
const snapshotLabel = (s) => formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '')
|
||||
const note = ref('')
|
||||
const marking = ref(false)
|
||||
|
||||
const snapshotLabel = (s) => (s.pinned ? '📌 ' : '') + formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '') + (s.note ? ' – ' + s.note : '')
|
||||
const join = (name) => path.value + '/' + name
|
||||
const restorable = (p) => /^data\/[^/]+\/files(\/.*)?$/.test(p)
|
||||
const crumbs = computed(() => {
|
||||
@@ -88,7 +99,29 @@ watch(target, async (tg) => {
|
||||
snapshots.value = list.map(s => ({ ...s, displayLabel: snapshotLabel(s) }))
|
||||
snapshot.value = snapshots.value[0] ?? null
|
||||
}, { immediate: true })
|
||||
watch(snapshot, () => { selected.value = ''; open('data') })
|
||||
watch(snapshot, (s) => { selected.value = ''; note.value = s?.note ?? ''; open('data') })
|
||||
|
||||
// Changes the snapshot in place: choosing another one, which opens its folders again, is not needed
|
||||
async function mark(call, change) {
|
||||
const s = snapshot.value
|
||||
marking.value = true
|
||||
error.value = message.value = ''
|
||||
try {
|
||||
await call(target.value.id, s.id)
|
||||
change(s)
|
||||
s.displayLabel = snapshotLabel(s)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
error.value = e.message
|
||||
} finally {
|
||||
marking.value = false
|
||||
}
|
||||
}
|
||||
const saveNote = () => {
|
||||
const text = note.value.trim()
|
||||
return mark((targetId, id) => api.setSnapshotNote(targetId, id, text), (s) => { s.note = text })
|
||||
}
|
||||
const savePinned = (pinned) => mark((targetId, id) => api.setSnapshotPinned(targetId, id, pinned), (s) => { s.pinned = pinned })
|
||||
|
||||
async function open(p) {
|
||||
if (!snapshot.value) {
|
||||
@@ -125,6 +158,8 @@ async function restore() {
|
||||
<style scoped>
|
||||
.ngb-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 12px; }
|
||||
.ngb-row :deep(.ngb-snapshot) { min-width: min(360px, 100%); }
|
||||
.ngb-marks { align-items: flex-end; }
|
||||
.ngb-note { max-width: 360px; }
|
||||
.ngb-crumbs { display: flex; align-items: center; flex-wrap: wrap; }
|
||||
.ngb-entries li { display: flex; align-items: center; gap: 12px; min-height: 36px; }
|
||||
.ngb-entries li.selected { background: var(--color-primary-element-light); border-radius: var(--border-radius); }
|
||||
|
||||
@@ -82,7 +82,11 @@ class EmptyDirsTest extends TestCase {
|
||||
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
|
||||
}
|
||||
$this->assertSame('commit', $state['phase']);
|
||||
$this->assertSame(4, $state['stats']['unreadable'], 'each reported once, over several steps');
|
||||
$this->assertSame(4, $state['stats']['badNames'], 'each reported once, over several steps');
|
||||
$this->assertArrayNotHasKey('unreadable', $state['stats'], 'not a problem with permissions');
|
||||
$shown = BackupRun::warnings($state)['badNames'];
|
||||
$this->assertContains('data/alice/files/line?break', $shown, 'no control character in a message');
|
||||
$this->assertSame($shown, array_filter($shown, fn (string $name): bool => mb_check_encoding($name, 'UTF-8')));
|
||||
$this->repo()->recordSnapshot($state['snapshot']);
|
||||
|
||||
$this->assertSame(['data/alice/files/Hollow/Nested', 'data/skipped/inner'], array_column($this->repo()->emptyDirs($state['snapshot']), 0));
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||
|
||||
use OCA\NgBackup\Backend\LocalBackend;
|
||||
use OCA\NgBackup\Crypto\KeyRing;
|
||||
use OCA\NgBackup\Job\BackupRun;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** Folders with something in them get their original modification time back on a restore. */
|
||||
class FolderTimesTest extends TestCase {
|
||||
use TempDirTrait;
|
||||
|
||||
private const OLD = 1500000000;
|
||||
private const OLDER = 1400000000;
|
||||
|
||||
private LocalBackend $backend;
|
||||
private KeyRing $keys;
|
||||
private string $src;
|
||||
|
||||
protected function setUp(): void {
|
||||
$this->backend = new LocalBackend($this->tempDir());
|
||||
$this->keys = KeyRing::generate();
|
||||
Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||
$this->src = $this->tempDir();
|
||||
mkdir("$this->src/data/alice/files/Old/Deep", 0700, true);
|
||||
mkdir("$this->src/data/alice/files/2026", 0700, true);
|
||||
file_put_contents("$this->src/data/alice/files/Old/Deep/a.txt", 'a');
|
||||
file_put_contents("$this->src/data/alice/files/Old/b.txt", 'b');
|
||||
file_put_contents("$this->src/data/alice/files/2026/c.txt", 'c');
|
||||
touch("$this->src/data/alice/files/Old/Deep", self::OLDER);
|
||||
touch("$this->src/data/alice/files/Old", self::OLD);
|
||||
touch("$this->src/data/alice/files/2026", self::OLD);
|
||||
}
|
||||
|
||||
private function repo(): Repository {
|
||||
return Repository::openWithKey($this->backend, $this->keys);
|
||||
}
|
||||
|
||||
private function backup(float $seconds = 30.0): string {
|
||||
$state = BackupRun::start(['data' => "$this->src/data"]);
|
||||
for ($i = 0; $i < 500 && $state['phase'] !== 'commit'; $i++) {
|
||||
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, microtime(true) + $seconds)), true);
|
||||
}
|
||||
$this->assertSame('commit', $state['phase']);
|
||||
$this->repo()->recordSnapshot($state['snapshot']);
|
||||
return $state['snapshot'];
|
||||
}
|
||||
|
||||
private function time(string $path): int {
|
||||
clearstatcache();
|
||||
return (int)filemtime($path);
|
||||
}
|
||||
|
||||
public function testEachFolderIsRecordedOnceAlsoInSmallSteps(): void {
|
||||
$expected = ['data', 'data/alice', 'data/alice/files', 'data/alice/files/2026', 'data/alice/files/Old', 'data/alice/files/Old/Deep'];
|
||||
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup()), 0));
|
||||
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup(0.0)), 0), 'a step per entry');
|
||||
}
|
||||
|
||||
public function testNoEmptyFolderIsAmongThem(): void {
|
||||
mkdir("$this->src/data/alice/files/Hollow");
|
||||
$snapshot = $this->backup();
|
||||
$this->assertNotContains('data/alice/files/Hollow', array_column($this->repo()->folderTimes($snapshot), 0));
|
||||
$this->assertContains('data/alice/files/Hollow', array_column($this->repo()->emptyDirs($snapshot), 0));
|
||||
}
|
||||
|
||||
public function testARestoreBringsThemBack(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$this->repo()->restore($snapshot, $target);
|
||||
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
|
||||
$this->assertSame(self::OLDER, $this->time("$target/data/alice/files/Old/Deep"));
|
||||
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/2026"));
|
||||
}
|
||||
|
||||
public function testARestoreThatGoesOnAfterAStopToo(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$this->repo()->restore($snapshot, $target);
|
||||
touch("$target/data/alice/files/Old", 1800000000);
|
||||
$this->repo()->restore($snapshot, $target, keepUnchanged: true);
|
||||
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
|
||||
}
|
||||
|
||||
public function testAPrefixLeavesTheTargetAndOthersAlone(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$this->repo()->restore($snapshot, $target, 'data/alice/files/Old/', true);
|
||||
$this->assertSame(self::OLDER, $this->time("$target/Deep"));
|
||||
$this->assertNotSame(self::OLD, $this->time($target), 'the directory that was chosen is not ours to change');
|
||||
$this->assertDirectoryDoesNotExist("$target/2026");
|
||||
}
|
||||
|
||||
public function testALinkOutOfTheTargetIsRefused(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$outside = $this->tempDir();
|
||||
touch($outside, 1800000000);
|
||||
mkdir("$target/data/alice/files", 0700, true);
|
||||
symlink($outside, "$target/data/alice/files/Old");
|
||||
$this->expectException(\OCA\NgBackup\Repository\RestoreTargetException::class);
|
||||
$this->repo()->restore($snapshot, $target);
|
||||
}
|
||||
|
||||
public function testAFolderWithABadNameIsLeftOutAndWhatIsInItReported(): void {
|
||||
mkdir("$this->src/data/odd\\folder");
|
||||
file_put_contents("$this->src/data/odd\\folder/x", 'x');
|
||||
$state = BackupRun::start(['data' => "$this->src/data"]);
|
||||
for ($i = 0; $i < 50 && $state['phase'] !== 'commit'; $i++) {
|
||||
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
|
||||
}
|
||||
$this->assertSame('commit', $state['phase']);
|
||||
$this->assertSame(1, $state['stats']['badNames']);
|
||||
$this->repo()->recordSnapshot($state['snapshot']);
|
||||
$this->assertNotContains("data/odd\\folder", array_column($this->repo()->folderTimes($state['snapshot']), 0));
|
||||
}
|
||||
|
||||
public function testASnapshotWithoutThemHasNone(): void {
|
||||
$snapshot = $this->repo()->backupDirectory($this->src)['snapshot'];
|
||||
$this->assertSame([], $this->repo()->folderTimes($snapshot));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||
|
||||
use OCA\NgBackup\Backend\LocalBackend;
|
||||
use OCA\NgBackup\Crypto\KeyRing;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** A pin and a note on a snapshot live in the catalog on the location, so a new server sees them too. */
|
||||
class SnapshotMarksTest extends TestCase {
|
||||
use TempDirTrait;
|
||||
|
||||
private LocalBackend $backend;
|
||||
private KeyRing $keys;
|
||||
private string $snapshot;
|
||||
|
||||
protected function setUp(): void {
|
||||
$this->backend = new LocalBackend($this->tempDir());
|
||||
$this->keys = KeyRing::generate();
|
||||
$src = $this->tempDir();
|
||||
file_put_contents("$src/a", 'x');
|
||||
$this->snapshot = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'))->backupDirectory($src)['snapshot'];
|
||||
}
|
||||
|
||||
private function repo(): Repository {
|
||||
return Repository::openWithKey($this->backend, $this->keys);
|
||||
}
|
||||
|
||||
public function testTheyAreThereAfterOpeningAgain(): void {
|
||||
$this->repo()->setPinned($this->snapshot, true);
|
||||
$this->repo()->setSnapshotNote($this->snapshot, 'before the upgrade');
|
||||
$this->assertSame([$this->snapshot], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([$this->snapshot => 'before the upgrade'], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testTheyCanBeTakenOffAgain(): void {
|
||||
$this->repo()->setPinned($this->snapshot, true);
|
||||
$this->repo()->setSnapshotNote($this->snapshot, 'x');
|
||||
$this->repo()->setPinned($this->snapshot, false);
|
||||
$this->repo()->setSnapshotNote($this->snapshot, '');
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testNoChangeWritesNoCatalog(): void {
|
||||
$repo = $this->repo();
|
||||
$repo->setPinned($this->snapshot, true);
|
||||
$generation = $repo->catalogGeneration();
|
||||
$repo->setPinned($this->snapshot, true);
|
||||
$repo->setSnapshotNote($this->snapshot, '');
|
||||
$this->assertSame($generation, $repo->catalogGeneration());
|
||||
}
|
||||
|
||||
public function testASnapshotThatIsNotThereIsRefused(): void {
|
||||
$this->expectException(\InvalidArgumentException::class);
|
||||
$this->repo()->setPinned('0123456789abcdef0123456789abcdef', true);
|
||||
}
|
||||
|
||||
public function testATrashedSnapshotLosesItsPinAndKeepsItsNote(): void {
|
||||
$repo = $this->repo();
|
||||
$repo->setPinned($this->snapshot, true);
|
||||
$repo->setSnapshotNote($this->snapshot, 'kept');
|
||||
$repo->forget($this->snapshot);
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
|
||||
|
||||
$repo = $this->repo();
|
||||
$repo->untrash($this->snapshot);
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots(), 'back in the list, as a normal one');
|
||||
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testThePurgeTakesTheNoteAlong(): void {
|
||||
$repo = $this->repo();
|
||||
$repo->setSnapshotNote($this->snapshot, 'kept');
|
||||
$repo->forget($this->snapshot);
|
||||
$repo->purgeTrash(0);
|
||||
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testACatalogFromBeforeThisVersionReadsAsNone(): void {
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||
|
||||
use OCA\NgBackup\Service\LocationSchedule as S;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
class LocationScheduleTest extends TestCase {
|
||||
private const BERLIN = 'Europe/Berlin';
|
||||
|
||||
private static function at(string $time): int {
|
||||
return (int)strtotime($time);
|
||||
}
|
||||
|
||||
public function testWhatCanBeSaved(): void {
|
||||
$this->assertSame('', S::normalize('off'));
|
||||
$this->assertSame('', S::normalize(' '));
|
||||
$this->assertSame('02:30', S::normalize('02:30'));
|
||||
$this->assertSame('02:30', S::normalize('02:30/24'));
|
||||
$this->assertSame('00:00/6', S::normalize('00:00/6'));
|
||||
foreach (['2:30', '24:00', '02:30/0', '02:30/25', 'daily', '02:30/x'] as $bad) {
|
||||
try {
|
||||
S::normalize($bad);
|
||||
$this->fail("$bad accepted");
|
||||
} catch (\InvalidArgumentException) {
|
||||
$this->addToAssertionCount(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public function testOffHasNoMoment(): void {
|
||||
$this->assertNull(S::lastMoment('', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
|
||||
}
|
||||
|
||||
public function testDailyIsInTheZoneOfTheLocation(): void {
|
||||
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
|
||||
$this->assertSame(self::at('2026-11-09 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 01:00 UTC')), 'not yet today');
|
||||
}
|
||||
|
||||
public function testEveryNHoursStartsOverTheNextDay(): void {
|
||||
$zone = 'UTC';
|
||||
$this->assertSame(self::at('2026-11-10 12:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 13:00 UTC')));
|
||||
$this->assertSame(self::at('2026-11-09 22:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 01:59 UTC')), '02, 07, 12, 17, 22, then 02 again');
|
||||
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 02:00 UTC')), 'at the moment itself');
|
||||
}
|
||||
|
||||
public function testAClockChangeInTheNightStillGivesOneMoment(): void {
|
||||
$this->assertSame(self::at('2026-03-29 01:30 UTC'), S::lastMoment('02:30', self::BERLIN, self::at('2026-03-29 12:00 UTC')), '02:30 does not exist that night');
|
||||
$this->assertNotNull(S::lastMoment('02:30', self::BERLIN, self::at('2026-10-25 12:00 UTC')));
|
||||
}
|
||||
|
||||
public function testChoosingATimeDoesNotStartABackup(): void {
|
||||
$moment = self::at('2026-11-10 02:00 UTC');
|
||||
$this->assertFalse(S::isDue($moment, $moment + 1, null, 0, null, $moment + 3600));
|
||||
$this->assertTrue(S::isDue($moment, $moment - 1, null, 0, null, $moment + 3600));
|
||||
$this->assertTrue(S::isDue($moment, 0, null, 0, null, $moment), 'a location from before schedules per location');
|
||||
}
|
||||
|
||||
public function testABackupSinceTheMomentIsEnough(): void {
|
||||
$moment = 1000000;
|
||||
$this->assertFalse(S::isDue($moment, 0, $moment, 1, $moment + 10, $moment + 20));
|
||||
$this->assertTrue(S::isDue($moment, 0, $moment - 1, 0, null, $moment + 20), 'it started before this moment: one more');
|
||||
}
|
||||
|
||||
public function testAfterAFailureOnlyAFewAttemptsAnHourApart(): void {
|
||||
$moment = 1000000;
|
||||
$finished = $moment + 60;
|
||||
$this->assertFalse(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER - 1));
|
||||
$this->assertTrue(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER));
|
||||
$this->assertFalse(S::isDue($moment, 0, null, S::MAX_ATTEMPTS, $finished, $finished + 86400));
|
||||
}
|
||||
}
|
||||
@@ -25,6 +25,17 @@ class RetentionPolicyTest extends TestCase {
|
||||
$this->assertCount(3, $months);
|
||||
}
|
||||
|
||||
public function testAPinnedSnapshotStaysAndDoesNotCount(): void {
|
||||
$policy = new RetentionPolicy(1, 0, 0, 0);
|
||||
$snaps = ['old' => 100, 'pinned' => 300, 'new' => 200];
|
||||
$this->assertSame(['pinned'], array_keys($policy->keep($snaps)), 'without the pin it would take the place of "last"');
|
||||
$keep = $policy->keepWithPinned($snaps, ['pinned', 'gone-already']);
|
||||
$this->assertSame(['pinned'], $keep['pinned']);
|
||||
$this->assertSame(['last'], $keep['new'], 'the newest one that is not pinned');
|
||||
$this->assertArrayNotHasKey('old', $keep);
|
||||
$this->assertArrayNotHasKey('gone-already', $keep);
|
||||
}
|
||||
|
||||
public function testKeepsEverythingWhenFewSnapshots(): void {
|
||||
$keep = (new RetentionPolicy())->keep(['a' => 100, 'b' => 200]);
|
||||
$this->assertSame(['a', 'b'], array_values(array_intersect(['a', 'b'], array_keys($keep))));
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||
|
||||
use OCA\NgBackup\Service\AlertService;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** A location without a schedule: a weekly reminder while another is backed up, else a daily one. */
|
||||
class ScheduleOffReminderTest extends TestCase {
|
||||
public function testOnceAWeekWhileAnotherLocationIsActive(): void {
|
||||
$this->assertFalse(AlertService::offReminderDue(0, 6 * 86400, true));
|
||||
$this->assertTrue(AlertService::offReminderDue(0, 7 * 86400, true));
|
||||
}
|
||||
|
||||
public function testEveryDayWhenNoLocationIsActive(): void {
|
||||
$this->assertFalse(AlertService::offReminderDue(0, 3600, false));
|
||||
$this->assertTrue(AlertService::offReminderDue(0, 86400, false));
|
||||
$this->assertTrue(AlertService::offReminderDue(0, 86400 - 60, false), 'a cron run a minute earlier than yesterday');
|
||||
}
|
||||
}
|
||||
@@ -57,15 +57,4 @@ class ScheduleTest extends TestCase {
|
||||
$this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get());
|
||||
}
|
||||
|
||||
public function testTodayAtIsTheMomentInTheSchedulesZone(): void {
|
||||
$this->values = [Schedule::TIME_KEY => '03:00', Schedule::ZONE_KEY => 'Europe/Berlin'];
|
||||
$at = $this->schedule()->todayAt(strtotime('2026-10-22 12:00 UTC'));
|
||||
$this->assertSame(strtotime('2026-10-22 01:00 UTC'), $at, '03:00 in Berlin (summer time) is 01:00 UTC');
|
||||
}
|
||||
|
||||
public function testAnUnknownZoneOrABadTimeIsRefused(): void {
|
||||
$s = $this->schedule();
|
||||
$this->expectException(\InvalidArgumentException::class);
|
||||
$s->set('03:00', 'Mars/Olympus');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -23,12 +23,15 @@ ok "key, kit code and confirmation"
|
||||
occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add"
|
||||
ok "location added"
|
||||
|
||||
# Test data: a user with files.
|
||||
# Test data: a user with files. No skeleton files on the first login, below, in the middle of the checks.
|
||||
occ config:system:set skeletondirectory --value='' >/dev/null
|
||||
occ config:system:set templatedirectory --value='' >/dev/null
|
||||
$X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
|
||||
# large.bin (40 MB) fills more than one pack: a restore to a directory reads ahead over several ranges
|
||||
$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs/Empty/Nested $DATA/$U/files/Hollow && touch -d @1700000000 $DATA/$U/files/Hollow && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin; head -c 40000000 /dev/urandom > $DATA/$U/files/large.bin"
|
||||
$X sh -c "touch -d @1600000000 $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs"
|
||||
occ files:scan "$U" >/dev/null
|
||||
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||
|
||||
occ backup:run ci -l first >/dev/null || fail "first backup"
|
||||
SECOND=$(occ backup:run ci -l second)
|
||||
@@ -44,6 +47,14 @@ occ files:scan "$U" >/dev/null
|
||||
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m new-folder >/dev/null || fail "restore new-folder"
|
||||
$X sh -c "ls $DATA/$U/files | grep -q '^Restored '" || fail "no Restored folder"
|
||||
$X sh -c "test -d $DATA/$U/files/Restored\ */Docs/Empty/Nested" || fail "the empty folder is not in the Restored folder"
|
||||
# Folders with files: on disk and in what Nextcloud shows, the parent as well as the child.
|
||||
RDIR=$($X sh -c "cd $DATA/$U/files && ls -d Restored\ *")
|
||||
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs/Sub")" = 1600000000 ] || fail "a folder with files did not get its time back"
|
||||
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs")" = 1600000000 ] || fail "the parent folder did not get its time back"
|
||||
DAV=$($X curl -sf -u "$U:Roundtrip-pw-2026" -X PROPFIND -H 'Depth: 1' "http://localhost/remote.php/dav/files/$U/$(php -r 'echo rawurlencode($argv[1]);' "$RDIR")/Docs/") || fail "PROPFIND"
|
||||
[ "$(echo "$DAV" | grep -o 'Sun, 13 Sep 2020 12:26:40 GMT' | wc -l)" = 2 ] || fail "Nextcloud does not show the time of the folders: $(echo "$DAV" | grep -o '<d:getlastmodified>[^<]*' | sort | uniq -c)"
|
||||
occ backup:restore ci "$SNAP" "data/$U/files/Docs/Sub" -m merge >/dev/null || fail "restore into a folder that is there"
|
||||
[ "$($X stat -c %Y "$DATA/$U/files/Docs/Sub")" != 1600000000 ] || fail "a folder that was already there got the old time"
|
||||
occ backup:browse ci "$SNAP" "data/$U/files" | grep Hollow >/dev/null || fail "an empty folder is not shown when browsing the snapshot"
|
||||
occ backup:restore ci "$SNAP" "data/$U/files/Hollow" -m merge >/dev/null || fail "restore of a folder that was empty"
|
||||
$X sh -c "test -d $DATA/$U/files/Hollow" || fail "the empty folder did not come back by itself"
|
||||
@@ -52,8 +63,8 @@ occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m replace >/dev/null || fail
|
||||
$X sh -c "test -d $DATA/$U/files/Docs/Empty/Nested" || fail "replace did not bring back the empty folder, or moved it to the trash"
|
||||
ok "empty folders: restored in each mode, by themselves and shown when browsing"
|
||||
$X sh -c "rm -rf '$DATA/$U/files/Restored '*" ; occ files:scan "$U" >/dev/null
|
||||
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore"
|
||||
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore: $(diff <(echo "$SUM_BEFORE") <(echo "$SUM_AFTER") | cut -c1-120)"
|
||||
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
||||
ok "restore new-folder and replace: files identical, extra file in the trash"
|
||||
|
||||
@@ -67,7 +78,7 @@ ok "restore of one file to a directory; the temporary of a stopped restore remov
|
||||
|
||||
# All of the user's files to a directory: over S3 and SFTP the packs are read ahead
|
||||
occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-dir >/dev/null || fail "restore of a folder to a directory"
|
||||
SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||
SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||
[ "$SUM_BEFORE" = "$SUM_DIR" ] || fail "the files restored to a directory differ"
|
||||
$X sh -c "ls -A /tmp/ngb-rt-dir | grep -q ngb-restore-spool" && fail "the read-ahead spool was left in the directory"
|
||||
$X sh -c "test -d /tmp/ngb-rt-dir/data/$U/files/Docs/Empty/Nested" || fail "the empty folder is not in the directory"
|
||||
@@ -101,13 +112,71 @@ ok "without system cron: warning in backup:status and the API, backup finished b
|
||||
# The Bin reads in parts, as the page does (what is in the parts: CatalogTest)
|
||||
api "$API/targets/$TID/trash" | php -r '$p = json_decode(stream_get_contents(STDIN), true); exit(is_int($p["total"] ?? null) && is_array($p["items"] ?? null) && count($p["items"]) <= $p["total"] ? 0 : 1);' || fail "the Bin API does not give a total and items"
|
||||
ok "the Bin API reads in parts, with a total"
|
||||
# Locations are backed up one after the other: the second one started is queued until the first is done.
|
||||
for L in 2 3; do
|
||||
$X sh -c "mkdir -p /tmp/ngb-rt-target$L"
|
||||
occ backup:target:add ci$L local -a null::null -o datadir=/tmp/ngb-rt-target$L --path=repo >/dev/null || fail "target add ci$L"
|
||||
done
|
||||
IDS=$(api "$API/status" | php -r 'foreach (json_decode(stream_get_contents(STDIN), true)["targets"] as $t) { if ($t["name"] === "ci2" || $t["name"] === "ci3") { echo $t["id"], " "; } }')
|
||||
for ID in $IDS; do api -X POST -d "{\"targetId\": $ID, \"label\": \"turn\"}" "$API/backups" >/dev/null || fail "start a backup to location $ID"; done
|
||||
# Never two runs to one location: a second start gets the run that is there.
|
||||
ID=${IDS%% *}
|
||||
RUNS=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||
api -X POST -d "{\"targetId\": $ID, \"label\": \"again\"}" "$API/backups" >/dev/null || fail "second start"
|
||||
AGAIN=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||
[ "$RUNS" = 2 ] && [ "$AGAIN" = 2 ] || fail "a second start to one location: $RUNS, then $AGAIN running"
|
||||
occ backup:status | grep "queued" >/dev/null || fail "the second backup is not queued"
|
||||
QUEUED=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["queued"]));')
|
||||
[ "$QUEUED" = 1 ] || fail "$QUEUED backups queued instead of 1"
|
||||
for i in $(seq 1 60); do
|
||||
api -X POST "$API/tick" >/dev/null || fail "tick"
|
||||
RUNNING=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||
[ "$RUNNING" != 0 ] || break
|
||||
done
|
||||
[ "$RUNNING" = 0 ] || fail "the queued backups did not finish"
|
||||
occ backup:list ci3 | grep " turn " >/dev/null || fail "the queued backup made no snapshot"
|
||||
ok "locations in turn: the second backup queued, and done after the first"
|
||||
occ background:cron >/dev/null
|
||||
$X php -f /var/www/html/cron.php >/dev/null 2>&1 || true
|
||||
occ backup:status | grep "system cron" >/dev/null && fail "backup:status still warns after cron ran"
|
||||
sleep 8 # APCu app config cache of the web server
|
||||
api "$API/status" | php -r 'exit(json_decode(stream_get_contents(STDIN), true)["cron"]["ok"] ? 0 : 1);' || fail "the status API still reports a missing cron after cron ran"
|
||||
ok "with a system cron that ran: no warning in backup:status or the API"
|
||||
|
||||
# A schedule per location. Choosing a time starts nothing; the moment itself does, and only there.
|
||||
JOB=$(occ background-job:list --class='OCA\NgBackup\BackgroundJob\BackupJob' --output=json | php -r 'echo json_decode(stream_get_contents(STDIN), true)[0]["id"];')
|
||||
job() { occ background-job:execute "$JOB" --force-execute >/dev/null; }
|
||||
occ backup:target:schedule ci off >/dev/null
|
||||
occ backup:target:schedule ci3 off >/dev/null
|
||||
occ backup:target:schedule ci2 00:00 --timezone=UTC >/dev/null || fail "set a schedule"
|
||||
occ backup:target:schedule ci2 | grep "00:00 (UTC)" >/dev/null || fail "schedule not shown"
|
||||
job
|
||||
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 0 ] || fail "choosing a time started a backup"
|
||||
occ backup:target:schedule ci2 "$($X date -u -d '+1 minute' +%H:%M)/6" --timezone=UTC >/dev/null || fail "set every 6 hours"
|
||||
sleep 65
|
||||
job
|
||||
occ backup:list ci2 | grep " scheduled " >/dev/null || fail "the scheduled moment started no backup"
|
||||
[ "$(occ backup:list ci3 | grep -c ' scheduled ')" = 0 ] || fail "a location that is off was backed up"
|
||||
job
|
||||
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 1 ] || fail "one moment started more than one backup"
|
||||
occ backup:target:schedule ci2 nonsense >/dev/null 2>&1 && fail "a bad schedule was accepted"
|
||||
ok "schedule per location: nothing on choosing a time, one backup at the moment, none when off"
|
||||
|
||||
# What administrators are told, as Nextcloud shows it.
|
||||
notes() { api "http://localhost/ocs/v2.php/apps/notifications/api/v2/notifications?format=json"; }
|
||||
occ backup:target:schedule ci2 off >/dev/null
|
||||
$X sh -c "touch '$DATA/ngb-rt-bad\\name.txt'"
|
||||
occ backup:run ci3 -l names | grep "their name is not supported" >/dev/null || fail "occ does not report the name"
|
||||
$X sh -c "rm '$DATA/ngb-rt-bad\\name.txt'"
|
||||
NOTES=$(notes) || fail "read the notifications"
|
||||
echo "$NOTES" | grep "turned off the schedule of backup location ci2" >/dev/null || fail "no notification about the schedule that was turned off"
|
||||
echo "$NOTES" | grep "is missing because of its name" >/dev/null || fail "no notification about the name"
|
||||
echo "$NOTES" | grep "Rename them" >/dev/null || fail "the notification about the name does not ask to rename"
|
||||
job
|
||||
notes | grep "has no schedule" >/dev/null && fail "a reminder right after turning it off"
|
||||
for L in 2 3; do occ backup:target:remove ci$L >/dev/null || fail "target remove ci$L"; done
|
||||
occ user:delete "$ADM" >/dev/null
|
||||
ok "notifications: schedule turned off, name not supported, no early reminder"
|
||||
|
||||
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
|
||||
occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify"
|
||||
@@ -120,6 +189,17 @@ ok "database dump and restore"
|
||||
|
||||
# Retention, trash and catalog.
|
||||
occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null
|
||||
# A pinned restore point stays, and a note can change afterwards.
|
||||
FIRST=$(occ backup:list ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
|
||||
occ backup:pin ci "$FIRST" >/dev/null || fail "pin"
|
||||
occ backup:note ci "$FIRST" "a note made later" >/dev/null || fail "note"
|
||||
occ backup:list ci | grep " first " | grep " yes " | grep "a note made later" >/dev/null || fail "pin and note not listed"
|
||||
occ backup:prune ci --dry-run | grep "would forget 1 " >/dev/null || fail "a pinned snapshot would be removed"
|
||||
occ backup:pin ci "$FIRST" --unpin >/dev/null || fail "unpin"
|
||||
occ backup:prune ci --dry-run | grep "would forget 2 " >/dev/null || fail "an unpinned snapshot would stay"
|
||||
occ backup:note ci "$FIRST" >/dev/null || fail "remove note"
|
||||
occ backup:list ci | grep -q "a note made later" && fail "note not removed"
|
||||
ok "pin and note on a restore point"
|
||||
occ backup:prune ci | grep "forgot 2" >/dev/null || fail "prune" # first and second; "page" (above) is the last
|
||||
occ backup:trash ci | grep " first " >/dev/null || fail "trash"
|
||||
TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
|
||||
|
||||
Reference in New Issue
Block a user