Compare commits
38 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c6fdaad487 | |||
| 6ae1b2277f | |||
| 2a72acb70b | |||
| fb8e8d0694 | |||
| 042fdae686 | |||
| 99f1655d04 | |||
| 373d7e0424 | |||
| c5024d3ddc | |||
| 345b1d545e | |||
| 98067f7746 | |||
| cd951af1a1 | |||
| cf9cf50286 | |||
| 2b21596b3a | |||
| 953312bf44 | |||
| 569077fa69 | |||
| 51a4a67bc4 | |||
| a8b5e85c9b | |||
| 36f2427e9a | |||
| e58b86fc48 | |||
| 193f65940b | |||
| 6540a9314a | |||
| f81c5e13e9 | |||
| 1b621b7149 | |||
| 1736005648 | |||
| c7a651f269 | |||
| 8897d8074f | |||
| 0b9fa1ae92 | |||
| ee406c481e | |||
| dd2e1af997 | |||
| d96229f370 | |||
| b14b16c66c | |||
| ce40053cec | |||
| 46e0186f35 | |||
| d5eb7abbff | |||
| 4c973d5cd8 | |||
| e9592cb6c6 | |||
| b86b07a49c | |||
| ecae6e9cdd |
@@ -11,6 +11,90 @@ Each heading starts with that `<version>`, once: the app store shows a release t
|
|||||||
whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
|
whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
|
||||||
shown under 1.0.0. The git tag of a release candidate goes after the date.
|
shown under 1.0.0. The git tag of a release candidate goes after the date.
|
||||||
|
|
||||||
|
## [1.1.0] - 2026-11-09
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- Pin a restore point, on the restore page or with `occ backup:pin`: the retention policy leaves it
|
||||||
|
alone, for example the one from before an upgrade. Unpinning notifies all administrators.
|
||||||
|
- A note on a restore point that you can change afterwards, on the restore page or with
|
||||||
|
`occ backup:note`. Pins and notes are stored encrypted on the location, so a new server sees them.
|
||||||
|
An older version of NG Backup can still read the location, but drops them when it writes to it.
|
||||||
|
- A schedule per location: off, once a day, or every N hours from a given time, on the page under
|
||||||
|
Backups or with `occ backup:target:schedule`. After the update every location keeps the time that
|
||||||
|
held for all of them; a new location starts with the schedule of the first one. A schedule is
|
||||||
|
shown and changed in the time zone it was saved in, which the page names next to the time.
|
||||||
|
- All folders get their original modification time back on a restore, not only the empty ones, from
|
||||||
|
the first backup made with this version. An older version of NG Backup can still read new restore
|
||||||
|
points. A restore through Nextcloud into a folder that is already there leaves its time alone.
|
||||||
|
- Heavy background jobs (Memories, Recognize, Preview Generator, full text search, the antivirus
|
||||||
|
background scan) pause while a backup runs, at most 6 hours in a row, then they get an hour. No app
|
||||||
|
is turned off, and a job that is running goes on. On by default; turn it off under Backups or with
|
||||||
|
`occ backup:pause-jobs off`.
|
||||||
|
- Disaster recovery without the recovery kit: on a new server without a backup key,
|
||||||
|
`occ backup:target:add` finds the backups on the location and takes their key from there with one
|
||||||
|
of the old server's passphrases. Backups then wait until a new recovery kit is confirmed. The
|
||||||
|
deletion delay is now also stored on the location, sealed with the key, from its first backup
|
||||||
|
with this version; a location backed up only by an older version gives the default of 7 days.
|
||||||
|
When the settings on the location cannot be read, the longest delay (365 days) is taken, so
|
||||||
|
nothing leaves the Bin sooner than on the old server; the command says so.
|
||||||
|
- A passphrase change that cannot reach a location (offline at the time) now reaches it at the
|
||||||
|
location's next backup, so a new server can open it with the new passphrase.
|
||||||
|
- `occ backup:target:add --repair-config`: when the config file of a location is gone, the recovery
|
||||||
|
kit writes it again (for locations backed up by this version or newer). Before, the message said
|
||||||
|
this was possible, but it was not.
|
||||||
|
- The key page warns against creating a new key when you are restoring a server. A key made there
|
||||||
|
by mistake can still make way for the key of the backups, as long as it was never used (no
|
||||||
|
location, no backup): `occ backup:key:import --replace-unused-key` with the kit, or
|
||||||
|
`occ backup:target:add --replace-unused-key` with a passphrase of the old server. A key that
|
||||||
|
protected a location is never replaced, and neither is a key from before this version. All
|
||||||
|
administrators are notified of a replacement.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- The list of backups stays within the page: a long label, note or error wraps onto more lines
|
||||||
|
instead of making the table scroll sideways.
|
||||||
|
- Reading files for a backup is much faster on disks where every request waits, such as network and
|
||||||
|
cloud volumes: files are read in pieces of 4 MiB instead of 8 KiB. On a test server the disk
|
||||||
|
delivered 27 MiB/s instead of 5 MiB/s.
|
||||||
|
- Locations are backed up one after the other: a backup that has to wait shows as queued. A restore
|
||||||
|
point has the time its backup really began, not the time it was queued.
|
||||||
|
- Choosing a time no longer starts a backup right away when that time has already passed today.
|
||||||
|
- A location without a schedule no longer gets the notification that it has had no recent backup.
|
||||||
|
Instead, all administrators are told when a schedule is turned off, and reminded once a week while
|
||||||
|
another location is still backed up, or every day when none is.
|
||||||
|
- A file or folder that is left out because of its name now gets a notification of its own, in the
|
||||||
|
log and in `occ backup:run`. It says the item is missing from the backup, gives example paths and
|
||||||
|
asks to rename it; before, it was reported as unreadable, with advice about permissions.
|
||||||
|
|
||||||
|
## [1.0.3] - 2026-11-06
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- Empty folders are backed up and restored (#27), in `occ backup:restore`, `occ backup:restore:full`
|
||||||
|
and the restore page, where they are shown too. They come back from the first backup made with
|
||||||
|
this version; older restore points do not contain them. An older version of NG Backup can still
|
||||||
|
read new restore points, and then only leaves the empty folders out.
|
||||||
|
- A full disk during a restore no longer looks like a damaged backup. The restore now stops with a
|
||||||
|
message about the disk, also with `--salvage`, and does not point at `--salvage` any more. The same
|
||||||
|
goes for other trouble writing to the restore directory, such as permissions.
|
||||||
|
- A file or folder whose name a snapshot cannot hold (a backslash, a control character, not UTF-8) is
|
||||||
|
left out and reported as unreadable. Before, a backslash or control character could make the whole
|
||||||
|
restore point unreadable, and a name that is not UTF-8 made the backup fail.
|
||||||
|
- A restore that goes on after a stop also removes the temporary files in folders where every file
|
||||||
|
is already in place, and checks that those folders are inside the restore directory before it
|
||||||
|
leaves their files alone.
|
||||||
|
- After a restore that was killed and then ran again, a backup no longer has to wait up to two hours
|
||||||
|
for the locks of the killed run.
|
||||||
|
- A restore only removes read-ahead folders of its own (`.ngb-restore-spool-` with 8 hex digits).
|
||||||
|
- A restore keeps its lock on the location during a long row of files that are already in place.
|
||||||
|
- A database restore that fails no longer leaves behind tables it created, which could stop the
|
||||||
|
installation of that app later. When only NG Backup's own settings cannot be put back after the
|
||||||
|
database was restored, `--salvage` no longer says the database is still the fresh one.
|
||||||
|
- A connection that breaks while opening a pack on the location (seen with SMB) is retried once and
|
||||||
|
is handled by `--salvage`, instead of stopping the restore with the error of the storage.
|
||||||
|
- Without a system cron, several running backups take turns while the page is open.
|
||||||
|
- The Bin tab shows how far it is while it reads the removed restore points, which can take a while.
|
||||||
|
It also shows when it cannot read a location, instead of an empty bin, and refreshes when a run
|
||||||
|
ends.
|
||||||
|
|
||||||
## [1.0.2] - 2026-11-06
|
## [1.0.2] - 2026-11-06
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
@@ -55,6 +139,9 @@ shown under 1.0.0. The git tag of a release candidate goes after the date.
|
|||||||
|
|
||||||
### Known issues
|
### Known issues
|
||||||
- Empty folders are not backed up or restored; to be fixed in 1.0.3 (#27).
|
- Empty folders are not backed up or restored; to be fixed in 1.0.3 (#27).
|
||||||
|
- A full disk during a restore looks like a damaged backup ("Size mismatch"), and with `--salvage`
|
||||||
|
the restore goes on and lists the remaining files. The backup itself is fine: make room and run
|
||||||
|
the restore again. To be fixed in 1.0.3.
|
||||||
|
|
||||||
## [1.0.1] - 2026-11-01
|
## [1.0.1] - 2026-11-01
|
||||||
|
|
||||||
|
|||||||
@@ -437,7 +437,47 @@ in README "Known limitations":
|
|||||||
met vertraging is nog niet gemeten); Bin-tabblad; schatting `target:add`; nieuwe screenshots
|
met vertraging is nog niet gemeten); Bin-tabblad; schatting `target:add`; nieuwe screenshots
|
||||||
(1–3 vervangen, 4-bin erbij; in `info.xml` bij de release naar tag v1.0.2 laten wijzen en
|
(1–3 vervangen, 4-bin erbij; in `info.xml` bij de release naar tag v1.0.2 laten wijzen en
|
||||||
4-bin toevoegen). Getest: unit (208), psalm, integratie NC35 sqlite/pgsql, S3/SFTP/FTP/SMB,
|
4-bin toevoegen). Getest: unit (208), psalm, integratie NC35 sqlite/pgsql, S3/SFTP/FTP/SMB,
|
||||||
volledige restore met `--salvage`. Nog te doen: DR-test, release.
|
volledige restore met `--salvage`. **1.0.2 uitgebracht 2026-11-06** (tag `v1.0.2`, PR #28).
|
||||||
|
- **Naar 1.0.3, uit de reviews van 1.0.2 (achteraf gedraaid, 2026-11-06):**
|
||||||
|
1. Lege mappen (#27).
|
||||||
|
2. Een volle schijf onder `--salvage` telt als beschadigde backup en de restore gaat door; zonder
|
||||||
|
`--salvage` wijst de melding er ten onrechte naar. `fwrite` controleren, schrijffouten stoppen de
|
||||||
|
restore (`Repository::restoreEntries`; ook `RestoreService::writeFile` nakijken). Beide reviews.
|
||||||
|
3. Hervatten: controle op de doelmap vóór het overslaan, en tijdelijke bestanden ook opruimen in
|
||||||
|
een map waar alles al klopt.
|
||||||
|
4. Alleen `.ngb-restore-spool-<hex>` opruimen.
|
||||||
|
5. Salvage-test: tijdelijke bestanden recursief zoeken; `admin` en `frtest` apart controleren.
|
||||||
|
6. Nog nakijken: lease verversen bij overgeslagen bestanden; fout ná de database-commit
|
||||||
|
(`restoreOwnAppConfig`) en tabellen die na een rollback blijven staan; backups zonder cron
|
||||||
|
via de pagina (eerlijke beurten, stappen die de webtimeout halen); Bin verversen en fouten
|
||||||
|
tonen; SMB-uitzondering bij het openen van een pack onder `--salvage`.
|
||||||
|
Niet doen: bij hervatten de inhoud vergelijken (grootte en mtime is ook het criterium van de backup).
|
||||||
|
**Stand op branch `1.0.3` (2026-11-06):** punten 1-6 gebouwd. Lege mappen staan in het metarecord
|
||||||
|
van de snapshot, niet tussen de bestanden: 1.0.2 en ouder lezen zo'n snapshot gewoon en slaan alleen
|
||||||
|
de lege mappen over. Geen webtimeout-wijziging zonder cron: alleen in de code nagelopen, niet getest
|
||||||
|
(Nextcloud zet de PHP-tijdslimiet ruim, een gestopte stap wordt overgedaan); wel eerlijke beurten.
|
||||||
|
Bin leest in delen met teller en balk (besluit 2026-11-06, ook met oog op meerdere backups per
|
||||||
|
dag). Getest: unit (volle schijf op echte tmpfs), integratie NC35 sqlite/pgsql/mysql, sftp, salvage,
|
||||||
|
salvage-files; Bin in een echte browser met vertraagde verzoeken (1 van 7 t/m 7). Nog: reviews
|
||||||
|
vóór de PR, README-regels bij de versiecommit.
|
||||||
|
**Simulaties (2026-11-06, NC 34.0.4, 2 GB, lokale SFTP, `~/dev-local/ngb-sim/sim.sh`):** normaal
|
||||||
|
(pgsql), stop en hervat (pgsql → MariaDB, restore met kill -9 gestopt), volle schijf met `--salvage`:
|
||||||
|
alle drie geslaagd, alle 3.101 gebruikersbestanden en 7 lege mappen gelijk, share, agenda en WebDAV
|
||||||
|
werken. Vond één fout, opgelost: na een kill bleven vergrendelingen tot 2 uur staan, waardoor de
|
||||||
|
backup van stap 11 werd geweigerd. `files:scan` "Updated" zijn in de gemeten run alleen mappen
|
||||||
|
(mtime van mappen wordt niet teruggezet, dus nieuwe mtime en etag; bestanden ongemoeid). "Removed":
|
||||||
|
o.a. `cache` van een gebruiker, die de backup uitsluit. Waarschijnlijk verklaart dit ook de 50.080
|
||||||
|
"updated" op nc01, niet op nc01 zelf vastgesteld.
|
||||||
|
**Naar 1.1.0 of 1.2.0 (besluit 2026-11-06):** alle mappen krijgen bij een restore hun oorspronkelijke
|
||||||
|
tijd terug. Hoort zo, vooral bij oudere verzamelingen: de maptijd wordt gebruikt om in te schatten
|
||||||
|
wanneer een map is aangemaakt. De backup moet dan de tijd van alle mappen vastleggen en de restore
|
||||||
|
die aan het eind terugzetten; werkt pas vanaf de eerste backup met die versie. Op de roadmap onder 1.1.0.
|
||||||
|
**1.0.3 uitgebracht (2026-11-06)**, ook op nc01 geïnstalleerd.
|
||||||
|
- **Branches (besluit 2026-11-06):** werk voor een versie op een branch met die naam (`1.0.3`);
|
||||||
|
`polish` alleen voor kleine cosmetische zaken, per geval bekijken of en wanneer die gemerged
|
||||||
|
worden. SMB staat uit op sub5 en sub6; sub6 is het testaccount, eerst aanzetten voor een SMB-test.
|
||||||
|
Na 1.0.3 gaat het werk naar `1.1.0`. Een spoedfix die niet kan wachten gaat op `1.0.4` vanaf `main`,
|
||||||
|
krijgt een eigen merge en release; daarna `main` in `1.1.0` mergen, zodat daar met de fix getest wordt.
|
||||||
- **1.0.1** (uitgebracht 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
|
- **1.0.1** (uitgebracht 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
|
||||||
1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november,
|
1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november,
|
||||||
niet aangekondigd en los van de release. Voor 1.0.2 ook: nieuwe screenshots (het labelveld bij
|
niet aangekondigd en los van de release. Voor 1.0.2 ook: nieuwe screenshots (het labelveld bij
|
||||||
@@ -532,11 +572,33 @@ in README "Known limitations":
|
|||||||
- **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3,
|
- **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3,
|
||||||
4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan
|
4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan
|
||||||
een paar bestanden.
|
een paar bestanden.
|
||||||
- **1.1.0:**
|
- **1.1.0** (geen harde datum: een week was een voorzet, langer mag; de releasefrequentie mag omlaag,
|
||||||
|
besluit 2026-11-06):
|
||||||
|
**Betere melding voor namen die een snapshot niet kan bevatten** (backslash, stuurteken, geen UTF-8):
|
||||||
|
een eigen melding in plaats van "unreadable" met het advies over rechten. Zegt dat deze bestanden of
|
||||||
|
mappen in de backup ontbreken, geeft voorbeelden van de paden en vraagt ze te hernoemen; de volgende
|
||||||
|
backup neemt ze mee. Alleen als Nextcloud-melding: de app mailt niet zelf, of meldingen gemaild worden
|
||||||
|
is de keuze van de beheerder (besluit 2026-11-06).
|
||||||
|
**Stand op branch `1.1.0` (2026-11-07):** klaar: betere melding, punten 2, 4, 5, 6 en 8, en de maptijden
|
||||||
|
(`files:scan` na een volledige restore: 0 "Updated" in de simulatie; opslag gepakt in het metarecord).
|
||||||
|
Punt 2 reserveert de jobs in `oc_jobs` met één tijdstempel, alleen wat Nextcloud op dat moment zou kunnen
|
||||||
|
starten; 6 uur aan, dan 1 uur vrij; bij een crash geeft Nextcloud de reservering na 12 uur zelf vrij;
|
||||||
|
de logregels staan op niveau info, dus standaard niet zichtbaar. Pin en notitie staan
|
||||||
|
in de catalogus, met een kopie in de database voor de lijsten. Een moment van vóór het instellen van
|
||||||
|
een schema start niets. Geen schema: geen melding "geen recente backup", wel een melding aan alle
|
||||||
|
beheerders bij het uitzetten en een herinnering, wekelijks zolang een andere locatie een schema heeft,
|
||||||
|
anders dagelijks (besluit 2026-11-07). Nog niet getest: de update vanaf 1.0.3, de pagina in een browser.
|
||||||
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
|
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
|
||||||
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
||||||
`target:add` en sleutelimport (het gevoeligste pad), daarom niet in rc1. Daarna de
|
`target:add` en sleutelimport (het gevoeligste pad), daarom niet in rc1. Daarna de
|
||||||
DR-test op PVE02 herhalen zonder kit.
|
DR-test op PVE02 herhalen zonder kit.
|
||||||
|
**Gebouwd (2026-11-07):** `backup:target:add` op een server zonder sleutel kijkt eerst of de
|
||||||
|
locatie een repository heeft, vraagt dan een wachtwoordzin (of `--passphrase-file`), opent de
|
||||||
|
slots uit `config`, controleert de sleutel tegen de `check` en slaat hem pas dan op; geen
|
||||||
|
kitbevestiging, dus backups wachten op een nieuwe kit. Nieuw bestand `settings` op de locatie,
|
||||||
|
versleuteld met de hoofdsleutel en eenmalig geschreven (bij `target:add` en bij het begin van
|
||||||
|
een backup): de verwijdertermijn, anders de standaard 7 dagen. Getest: unit-tests en
|
||||||
|
`full-restore-roundtrip.sh 35-apache pgsql no-kit`. Nog open: de DR-test op PVE02 zonder kit.
|
||||||
2. Instelling "zware achtergrondjobs pauzeren tijdens een backup" (standaard aan, uit te zetten):
|
2. Instelling "zware achtergrondjobs pauzeren tijdens een backup" (standaard aan, uit te zetten):
|
||||||
Memories, files_antivirus BackgroundScanner, Recognize, Preview Generator, fulltextsearch.
|
Memories, files_antivirus BackgroundScanner, Recognize, Preview Generator, fulltextsearch.
|
||||||
Via de reservering/last_run van die jobs (geen apps uitzetten), crashbestendig, maximaal
|
Via de reservering/last_run van die jobs (geen apps uitzetten), crashbestendig, maximaal
|
||||||
@@ -544,6 +606,14 @@ in README "Known limitations":
|
|||||||
3. Snelheid: lezen en uploaden laten overlappen, meerdere bestanden tegelijk lezen, eventueel
|
3. Snelheid: lezen en uploaden laten overlappen, meerdere bestanden tegelijk lezen, eventueel
|
||||||
een eigen worker (`occ background-job:worker`). Gemeten op nc01: 55% wachten op de schijf,
|
een eigen worker (`occ background-job:worker`). Gemeten op nc01: 55% wachten op de schijf,
|
||||||
33% op de SMB-upload, ~7% CPU. Vergt extra PHP-processen; botst mogelijk met uitgangspunt 1.
|
33% op de SMB-upload, ~7% CPU. Vergt extra PHP-processen; botst mogelijk met uitgangspunt 1.
|
||||||
|
**Gemeten 2026-11-07 op nc01 (alleen lezen):** de schijf (sdb, 2 TB, ROTA=1) haalt met `dd`
|
||||||
|
21 MiB/s met 1 stroom, 37 met 4, 55 met 8. PHP las 8 KiB per verzoek: 5,2 MiB/s. Met
|
||||||
|
`stream_set_read_buffer(0)` en blokken van 4 MiB: 26,8 MiB/s. Hulpprocessen die vooruitlezen
|
||||||
|
gaven daarbovenop niets (20 MiB/s, ze concurreren). Gebouwd: grote leesverzoeken (commit
|
||||||
|
"Read files ... 4 MiB"). Niet gebouwd: meerdere uploads tegelijk; SMB bundelt al per 1 MB
|
||||||
|
(icewind/smb), dus winst over een lange lijn vraagt meerdere packs tegelijk.
|
||||||
|
**Besluit 2026-11-08: meerdere uploads tegelijk in 1.2.0**, met een instelling (standaard 2)
|
||||||
|
zodat de beheerder het afstemt op zijn server en de omvang van zijn backup.
|
||||||
4. **Schema per locatie** (besluit 2026-10-19): in de instellingen van een locatie "uit",
|
4. **Schema per locatie** (besluit 2026-10-19): in de instellingen van een locatie "uit",
|
||||||
"dagelijks om HH:MM" of "elke N uur (vanaf HH:MM)". Nu is er één `schedule_time` voor alle
|
"dagelijks om HH:MM" of "elke N uur (vanaf HH:MM)". Nu is er één `schedule_time` voor alle
|
||||||
locaties, één keer per dag. Bij de update wordt de huidige tijd het schema van elke
|
locaties, één keer per dag. Bij de update wordt de huidige tijd het schema van elke
|
||||||
@@ -564,11 +634,10 @@ in README "Known limitations":
|
|||||||
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
||||||
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
||||||
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
||||||
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
|
9. **Feedbackvraag weghalen in 1.2.0** (besluit 2026-11-07, niet in 1.1.0): de regel "We would like
|
||||||
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
||||||
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
|
dan staan. Er komt weinig feedback bij zo'n 150 downloads per versie; geen bericht is goed
|
||||||
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
|
bericht. Het formulier (`experience.yml`) in de issues blijft.
|
||||||
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
|
|
||||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ installations and hosted Nextcloud. (Choosing SMB as a storage backend is the on
|
|||||||
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
||||||
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
||||||
|
|
||||||
**Status: 1.0.2, stable.** Disaster recovery has been tested on real installations (see "Tested
|
**Status: 1.1.0, stable.** Disaster recovery has been tested on real installations (see "Tested
|
||||||
disaster recoveries" below). See "Known limitations" below, `PLAN.md` for the design and
|
disaster recoveries" below). See "Known limitations" below, `PLAN.md` for the design and
|
||||||
`docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
|
`docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
|
||||||
`docs/DISASTER-RECOVERY.md`.
|
`docs/DISASTER-RECOVERY.md`.
|
||||||
@@ -87,8 +87,9 @@ All backups of an installation are encrypted with one master key.
|
|||||||
- The **recovery kit** is your own copy of that encrypted key. Still keep it off the server and
|
- The **recovery kit** is your own copy of that encrypted key. Still keep it off the server and
|
||||||
off the location: an attacker who gets write access to the location can damage or delete its
|
off the location: an attacker who gets write access to the location can damage or delete its
|
||||||
`config` file on purpose, and without the kit that makes every backup there unreadable.
|
`config` file on purpose, and without the kit that makes every backup there unreadable.
|
||||||
- For a **restore on a new server** (disaster recovery) you currently need the **recovery kit**
|
- For a **restore on a new server** (disaster recovery) **one passphrase** is enough: the new
|
||||||
and **one passphrase**.
|
server takes the key from the location. With the **recovery kit** you can restore even when the
|
||||||
|
location's `config` file is damaged or gone.
|
||||||
- Planned: an option to not store the key on the location at all. Then the kit is the only copy,
|
- Planned: an option to not store the key on the location at all. Then the kit is the only copy,
|
||||||
and losing it means losing every backup; that option will come with an explicit warning.
|
and losing it means losing every backup; that option will come with an explicit warning.
|
||||||
- Up to **three passphrases** ("slots") can open the key, so one person leaving or dying does not
|
- Up to **three passphrases** ("slots") can open the key, so one person leaving or dying does not
|
||||||
@@ -169,6 +170,8 @@ Full restores onto a fresh server, following only [the guide](docs/DISASTER-RECO
|
|||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| Production, about 1 TB, 150,000 files (Nextcloud All-in-One) | Hetzner Storage Box (SFTP), Germany → Canada | PostgreSQL → PostgreSQL | 150,622 files restored in about 25 h (13-14 MiB/s), sample identical |
|
| Production, about 1 TB, 150,000 files (Nextcloud All-in-One) | Hetzner Storage Box (SFTP), Germany → Canada | PostgreSQL → PostgreSQL | 150,622 files restored in about 25 h (13-14 MiB/s), sample identical |
|
||||||
| The same, 1.0.2 | Hetzner Storage Box (SMB), Germany → Canada | PostgreSQL → PostgreSQL | 150,755 files restored in about 29 h (7-13 MiB/s), going on after a lost connection; sample identical, empty folders not restored ([#27](https://github.com/dillardblom/ng_backup/issues/27)) |
|
| The same, 1.0.2 | Hetzner Storage Box (SMB), Germany → Canada | PostgreSQL → PostgreSQL | 150,755 files restored in about 29 h (7-13 MiB/s), going on after a lost connection; sample identical, empty folders not restored ([#27](https://github.com/dillardblom/ng_backup/issues/27)) |
|
||||||
|
| 1.0.3, small simulations of 2 GB: as planned, with the restore killed halfway, and with a disk that runs full | Local SFTP server | PostgreSQL → PostgreSQL and MariaDB | all 3,101 user files and all empty folders identical; users, a share and a calendar back |
|
||||||
|
| 1.1.0, the same three simulations | Local SFTP server | PostgreSQL → PostgreSQL and MariaDB | all 3,101 user files identical in content, size and time, all empty folders back; users, a share and a calendar back |
|
||||||
| 138 GB simulation: 20 users, calendars, contacts, mail, shares, team folders with an ACL, versions, trash | OVH S3, Warsaw → Canada | MariaDB → PostgreSQL | all 15,772 files and every database count identical |
|
| 138 GB simulation: 20 users, calendars, contacts, mail, shares, team folders with an ACL, versions, trash | OVH S3, Warsaw → Canada | MariaDB → PostgreSQL | all 15,772 files and every database count identical |
|
||||||
| The same | OVH S3 | MariaDB → MariaDB | all 15,772 files and every database count identical |
|
| The same | OVH S3 | MariaDB → MariaDB | all 15,772 files and every database count identical |
|
||||||
| The same | Hetzner Storage Box (SMB), Germany → Canada | MariaDB → SQLite | all 15,772 files and every database count identical (27.6 h, about 1.5 MB/s) |
|
| The same | Hetzner Storage Box (SMB), Germany → Canada | MariaDB → SQLite | all 15,772 files and every database count identical (27.6 h, about 1.5 MB/s) |
|
||||||
@@ -265,21 +268,9 @@ What is planned, without dates. Ideas and votes are welcome in the
|
|||||||
[issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
|
[issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
|
||||||
a 👍 on its issue helps decide what comes first.
|
a 👍 on its issue helps decide what comes first.
|
||||||
|
|
||||||
**1.0.3**
|
**1.2.0**
|
||||||
- Empty folders in the backup and the restore
|
- Faster backups over a long line: several uploads at the same time, with a setting
|
||||||
([#27](https://github.com/dillardblom/ng_backup/issues/27)).
|
(2 by default) to suit the server and the size of the backup.
|
||||||
|
|
||||||
**1.1.0**
|
|
||||||
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
|
||||||
your passphrases.
|
|
||||||
- A schedule per location, also "every N hours".
|
|
||||||
- Locations backed up one after the other instead of at the same time.
|
|
||||||
- A note on a restore point that you can change afterwards.
|
|
||||||
- Pinning a restore point, so the retention policy never removes it (for example the one from
|
|
||||||
before an upgrade).
|
|
||||||
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
|
||||||
Generator) while a backup runs.
|
|
||||||
- Faster backups: reading and uploading at the same time.
|
|
||||||
|
|
||||||
**Later**
|
**Later**
|
||||||
- A full restore onto the existing installation, without first installing a fresh Nextcloud, in a
|
- A full restore onto the existing installation, without first installing a fresh Nextcloud, in a
|
||||||
@@ -293,9 +284,11 @@ a 👍 on its issue helps decide what comes first.
|
|||||||
|
|
||||||
On the backlog; know them before you rely on NG Backup:
|
On the backlog; know them before you rely on NG Backup:
|
||||||
|
|
||||||
- **Empty folders do not come back.** A backup only records files, so a folder without files is
|
- **Empty folders only come back from backups made with 1.0.3 or newer.** Older restore points do
|
||||||
not restored. To be fixed in 1.0.3
|
not contain them.
|
||||||
([#27](https://github.com/dillardblom/ng_backup/issues/27)).
|
- **Folders with files in them only get their original modification time back from backups made with
|
||||||
|
1.1.0 or newer.** From older restore points they show the time of the restore; the files keep theirs.
|
||||||
|
A restore through Nextcloud into a folder that is already there leaves that folder's time alone.
|
||||||
- **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
|
- **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
|
||||||
`config.php`) is not supported: a backup refuses to start there. Folders mounted through the
|
`config.php`) is not supported: a backup refuses to start there. Folders mounted through the
|
||||||
External storage app are not in the data directory and are **not included**, without a warning.
|
External storage app are not in the data directory and are **not included**, without a warning.
|
||||||
@@ -330,10 +323,13 @@ On the backlog; know them before you rely on NG Backup:
|
|||||||
| `backup:key:init`, `backup:key:kit`, `backup:key:confirm` | create the key, recovery kit, confirmation |
|
| `backup:key:init`, `backup:key:kit`, `backup:key:confirm` | create the key, recovery kit, confirmation |
|
||||||
| `backup:key:slots`, `backup:key:slot:add/replace/remove` | passphrase slots |
|
| `backup:key:slots`, `backup:key:slot:add/replace/remove` | passphrase slots |
|
||||||
| `backup:target:backends/add/edit/list/remove/append-only` | backup locations |
|
| `backup:target:backends/add/edit/list/remove/append-only` | backup locations |
|
||||||
|
| `backup:target:schedule` | when a location is backed up: off, daily, or every N hours |
|
||||||
| `backup:run`, `backup:status`, `backup:list` | back up, status (runs of the last 7 days; `--all` for all), restore points |
|
| `backup:run`, `backup:status`, `backup:list` | back up, status (runs of the last 7 days; `--all` for all), restore points |
|
||||||
| `backup:cancel <run>` | stop a running backup |
|
| `backup:cancel <run>` | stop a running backup |
|
||||||
| `backup:browse`, `backup:restore` | browse and restore |
|
| `backup:browse`, `backup:restore` | browse and restore |
|
||||||
| `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash |
|
| `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash |
|
||||||
|
| `backup:pin`, `backup:note` | keep a restore point, a note you can change later |
|
||||||
|
| `backup:pause-jobs` | whether heavy background jobs of other apps pause during a backup |
|
||||||
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
||||||
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
||||||
| `backup:key:import` | import a recovery kit on a fresh installation |
|
| `backup:key:import` | import a recovery kit on a fresh installation |
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ welcome: use the "Share your experience" link on the admin page.
|
|||||||
|
|
||||||
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
||||||
you need help with restoring a backup after a crash.]]></description>
|
you need help with restoring a backup after a crash.]]></description>
|
||||||
<version>1.0.2</version>
|
<version>1.1.0</version>
|
||||||
<licence>agpl</licence>
|
<licence>agpl</licence>
|
||||||
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
||||||
<namespace>NgBackup</namespace>
|
<namespace>NgBackup</namespace>
|
||||||
@@ -56,10 +56,10 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<website>https://github.com/dillardblom/ng_backup</website>
|
<website>https://github.com/dillardblom/ng_backup</website>
|
||||||
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
||||||
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/1-backups.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/1-backups.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/2-restore.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/2-restore.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/3-settings.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/3-settings.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.2/screenshots/4-bin.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/4-bin.png</screenshot>
|
||||||
<dependencies>
|
<dependencies>
|
||||||
<php min-version="8.1"/>
|
<php min-version="8.1"/>
|
||||||
<lib>sodium</lib>
|
<lib>sodium</lib>
|
||||||
@@ -83,11 +83,15 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<command>OCA\NgBackup\Command\ListSnapshots</command>
|
<command>OCA\NgBackup\Command\ListSnapshots</command>
|
||||||
<command>OCA\NgBackup\Command\Browse</command>
|
<command>OCA\NgBackup\Command\Browse</command>
|
||||||
<command>OCA\NgBackup\Command\Restore</command>
|
<command>OCA\NgBackup\Command\Restore</command>
|
||||||
|
<command>OCA\NgBackup\Command\PauseJobs</command>
|
||||||
|
<command>OCA\NgBackup\Command\Pin</command>
|
||||||
|
<command>OCA\NgBackup\Command\Note</command>
|
||||||
<command>OCA\NgBackup\Command\Prune</command>
|
<command>OCA\NgBackup\Command\Prune</command>
|
||||||
<command>OCA\NgBackup\Command\Retention</command>
|
<command>OCA\NgBackup\Command\Retention</command>
|
||||||
<command>OCA\NgBackup\Command\TargetAppendOnly</command>
|
<command>OCA\NgBackup\Command\TargetAppendOnly</command>
|
||||||
<command>OCA\NgBackup\Command\TargetTrustCurrent</command>
|
<command>OCA\NgBackup\Command\TargetTrustCurrent</command>
|
||||||
<command>OCA\NgBackup\Command\TargetLimit</command>
|
<command>OCA\NgBackup\Command\TargetLimit</command>
|
||||||
|
<command>OCA\NgBackup\Command\TargetSchedule</command>
|
||||||
<command>OCA\NgBackup\Command\Trash</command>
|
<command>OCA\NgBackup\Command\Trash</command>
|
||||||
<command>OCA\NgBackup\Command\TrashRestore</command>
|
<command>OCA\NgBackup\Command\TrashRestore</command>
|
||||||
<command>OCA\NgBackup\Command\KeySlots</command>
|
<command>OCA\NgBackup\Command\KeySlots</command>
|
||||||
|
|||||||
@@ -25,8 +25,11 @@ return [
|
|||||||
['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'],
|
['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'],
|
||||||
['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'],
|
['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'],
|
||||||
['name' => 'AdminApi#tick', 'url' => '/api/tick', 'verb' => 'POST'],
|
['name' => 'AdminApi#tick', 'url' => '/api/tick', 'verb' => 'POST'],
|
||||||
['name' => 'AdminApi#setSchedule', 'url' => '/api/schedule', 'verb' => 'PUT'],
|
['name' => 'AdminApi#setPauseHeavyJobs', 'url' => '/api/settings/pause-heavy-jobs', 'verb' => 'PUT'],
|
||||||
|
['name' => 'AdminApi#setTargetSchedule', 'url' => '/api/targets/{id}/schedule', 'verb' => 'PUT'],
|
||||||
['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'],
|
['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'],
|
||||||
|
['name' => 'AdminApi#setSnapshotNote', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/note', 'verb' => 'PUT'],
|
||||||
|
['name' => 'AdminApi#setSnapshotPinned', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/pin', 'verb' => 'PUT'],
|
||||||
['name' => 'AdminApi#browse', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/browse', 'verb' => 'GET'],
|
['name' => 'AdminApi#browse', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/browse', 'verb' => 'GET'],
|
||||||
['name' => 'AdminApi#startRestore', 'url' => '/api/restores', 'verb' => 'POST'],
|
['name' => 'AdminApi#startRestore', 'url' => '/api/restores', 'verb' => 'POST'],
|
||||||
],
|
],
|
||||||
|
|||||||
@@ -5,7 +5,8 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
|||||||
|
|
||||||
## What you need
|
## What you need
|
||||||
|
|
||||||
- [ ] The **recovery kit** file (`ng_backup-kit.json`) and **one passphrase** of it.
|
- [ ] **One passphrase** of the old server, and if you have it, the **recovery kit** file
|
||||||
|
(`ng_backup-kit.json`).
|
||||||
- [ ] The **connection details of the backup location**: host, user, password or keys, bucket, folder.
|
- [ ] The **connection details of the backup location**: host, user, password or keys, bucket, folder.
|
||||||
- [ ] A new server with the **same Nextcloud major version** as the old one (for example 34),
|
- [ ] A new server with the **same Nextcloud major version** as the old one (for example 34),
|
||||||
a database (PostgreSQL, MySQL/MariaDB or SQLite; it may differ from the old one) and room for
|
a database (PostgreSQL, MySQL/MariaDB or SQLite; it may differ from the old one) and room for
|
||||||
@@ -15,8 +16,10 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
|||||||
- [ ] A list of the apps that are **not in the app store** (your own builds, apps copied in by
|
- [ ] A list of the apps that are **not in the app store** (your own builds, apps copied in by
|
||||||
hand). Keep their packages somewhere safe beforehand.
|
hand). Keep their packages somewhere safe beforehand.
|
||||||
|
|
||||||
No kit? If the location still holds its `config` file, the key is there too, but this version
|
No kit? Since version 1.1.0 the passphrase is enough: the key is also stored on the location (in
|
||||||
can only import it from a kit. Keep the kit off the server, next to the passphrase.
|
its `config` file), locked with the same passphrases. Skip step 4. The kit is still the safer
|
||||||
|
copy: it helps when that file on the location is damaged. Keep it off the server, next to the
|
||||||
|
passphrase.
|
||||||
|
|
||||||
## Steps
|
## Steps
|
||||||
|
|
||||||
@@ -40,7 +43,8 @@ This keeps the new server quiet (no mail syncs, no notifications, no backups) un
|
|||||||
|
|
||||||
**3. Install NG Backup** (the same version or newer) and enable it.
|
**3. Install NG Backup** (the same version or newer) and enable it.
|
||||||
|
|
||||||
**4. Import the backup key.** Copy the kit to the new server so the web server user can read it.
|
**4. Import the backup key** (with a kit; without one, go to step 5). Copy the kit to the new
|
||||||
|
server so the web server user can read it.
|
||||||
|
|
||||||
From your own computer:
|
From your own computer:
|
||||||
|
|
||||||
@@ -67,13 +71,29 @@ the index and shows how far it is, for example
|
|||||||
|
|
||||||
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=... -o root=... --path=...
|
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=... -o root=... --path=...
|
||||||
|
|
||||||
If it says "No backup key yet", the key is not imported. Run step 4:
|
Without a kit (step 4 skipped), it finds the backups and asks for a passphrase of the old server;
|
||||||
|
you type it yourself. It then shows the key fingerprint and the deletion delay of the old server.
|
||||||
|
A wrong passphrase changes nothing: run the command again. Backups made before version 1.1.0 did
|
||||||
|
not store the deletion delay on the location; it is then set to 7 days.
|
||||||
|
|
||||||
occ backup:key:import /tmp/ng_backup-kit.json
|
If it says that the location's **config file is gone**, the key is only in the recovery kit:
|
||||||
|
import it (step 4), then add the location with `--repair-config` at the end of the same command.
|
||||||
|
That writes the config again from the kit; it works for locations backed up by version 1.1.0 or
|
||||||
|
newer.
|
||||||
|
|
||||||
Do **not** run `backup:key:init`, even though the message suggests it: that makes a brand new key,
|
If it says "No backup key yet, and no backups at this location", the connection details or
|
||||||
and your backups were encrypted with the old one. NG Backup then refuses the location ("created
|
`--path` differ from the old server's. Check them; do **not** run `backup:key:init`: that makes a
|
||||||
with a different key").
|
brand new key, and your backups were encrypted with the old one. NG Backup then refuses the
|
||||||
|
location ("created with a different key").
|
||||||
|
|
||||||
|
**Made a new key by mistake** (with `backup:key:init` or on the page) before adding the location?
|
||||||
|
As long as that key was never used (no location added with it, no backup), it can make way for the
|
||||||
|
key of your backups. NG Backup says so when it refuses the location. Add `--replace-unused-key` to
|
||||||
|
`occ backup:key:import` (with the kit) or to `occ backup:target:add` (without the kit; it asks for
|
||||||
|
a passphrase of the old server). The passphrases and kit of the unused key then stop working. A key
|
||||||
|
that was used for a location is never replaced: its own backups depend on it. A key made with a
|
||||||
|
version before 1.1.0 always counts as used (those versions did not keep track), also when it never
|
||||||
|
protected a location; recover onto a fresh Nextcloud installation instead.
|
||||||
|
|
||||||
**6. List the restore points** and pick one, normally the newest:
|
**6. List the restore points** and pick one, normally the newest:
|
||||||
|
|
||||||
@@ -117,7 +137,18 @@ first. If none is better, `--salvage` restores whatever can be read:
|
|||||||
|
|
||||||
occ backup:restore:full offsite <snapshot> --install-apps --salvage
|
occ backup:restore:full offsite <snapshot> --install-apps --salvage
|
||||||
|
|
||||||
|
If a restore already stopped halfway, run it as in step 8: maintenance mode off, with `--force` and
|
||||||
|
without `--install-apps`:
|
||||||
|
|
||||||
|
occ maintenance:mode --off && occ backup:restore:full offsite <snapshot> --salvage --force
|
||||||
|
|
||||||
- Files that cannot be read are left out and listed at the end; everything else is restored.
|
- Files that cannot be read are left out and listed at the end; everything else is restored.
|
||||||
|
- A full disk is not damage: the restore stops, also with `--salvage`, and says so. Make room and
|
||||||
|
run the restore again as in step 8.
|
||||||
|
- **On version 1.0.2, check the disk first if the reason is "Size mismatch".** There a full disk
|
||||||
|
looks like damaged data: the message points at `--salvage`, and with `--salvage` the restore goes
|
||||||
|
on and lists the remaining files. The backup itself is fine. Update NG Backup, make room and run
|
||||||
|
the restore again as in step 8.
|
||||||
- If the database cannot be restored (its dump is damaged, or it does not fit this
|
- If the database cannot be restored (its dump is damaged, or it does not fit this
|
||||||
installation), the database stays as the fresh installation made it, and the data directory
|
installation), the database stays as the fresh installation made it, and the data directory
|
||||||
and config of the backup go to a folder of their own in the data directory,
|
and config of the backup go to a folder of their own in the data directory,
|
||||||
@@ -140,9 +171,12 @@ The command then ends with exit code 2 instead of 0. The same option exists for
|
|||||||
- From Nextcloud All-in-One to a regular server: also check the cache settings (`memcache.*`,
|
- From Nextcloud All-in-One to a regular server: also check the cache settings (`memcache.*`,
|
||||||
`redis`), which point at AIO's containers.
|
`redis`), which point at AIO's containers.
|
||||||
- Log in, open a few files, a calendar and a contact.
|
- Log in, open a few files, a calendar and a contact.
|
||||||
- Empty folders do not come back: a backup only records files (to be fixed in 1.0.3, see
|
- `occ files:scan` can report many folders as "Updated". That is expected: a restore does not yet put
|
||||||
[issue #27](https://github.com/dillardblom/ng_backup/issues/27)). `occ files:scan` then removes
|
back the modification time of folders with files in them, so they show the time of the restore.
|
||||||
them from the file cache, so it reports them as removed.
|
The files themselves keep their time and content.
|
||||||
|
- Empty folders only come back from backups made with version 1.0.3 or newer. With an older
|
||||||
|
restore point they are gone: `occ files:scan` then removes them from the file cache, so it reports
|
||||||
|
them as removed.
|
||||||
- Start background jobs again (cron), as in step 2 in reverse.
|
- Start background jobs again (cron), as in step 2 in reverse.
|
||||||
|
|
||||||
**10. Point your domain at the new server** and check that clients reconnect.
|
**10. Point your domain at the new server** and check that clients reconnect.
|
||||||
@@ -150,6 +184,10 @@ The command then ends with exit code 2 instead of 0. The same option exists for
|
|||||||
**11. Make a backup of the new server** to the same location (`occ backup:run offsite`). The old
|
**11. Make a backup of the new server** to the same location (`occ backup:run offsite`). The old
|
||||||
server must stay off: two servers must never write to the same location.
|
server must stay off: two servers must never write to the same location.
|
||||||
|
|
||||||
|
Restored without a kit? Then backups wait until you have a new one: download it on the backup
|
||||||
|
page (or `occ backup:key:kit --output=...`), store it off the server and confirm it
|
||||||
|
(`occ backup:key:confirm`).
|
||||||
|
|
||||||
## Never
|
## Never
|
||||||
|
|
||||||
- Never let the old server write to the location again if it comes back. Switch its backups off
|
- Never let the old server write to the location again if it comes back. Switch its backups off
|
||||||
|
|||||||
@@ -79,7 +79,9 @@ occ backup:list
|
|||||||
Later runs upload only changed data. An unchanged run uploads (almost) nothing; the output shows
|
Later runs upload only changed data. An unchanged run uploads (almost) nothing; the output shows
|
||||||
the number of uploaded bytes, so check that value when you test incremental behaviour.
|
the number of uploaded bytes, so check that value when you test incremental behaviour.
|
||||||
|
|
||||||
Schedule `occ backup:run` from system cron (or use the Administration settings, if you prefer).
|
Set when each location is backed up in the Administration settings, or with
|
||||||
|
`occ backup:target:schedule offsite 02:00` (once a day), `02:00/6` (every 6 hours from 02:00) or `off`.
|
||||||
|
Locations are backed up one after the other.
|
||||||
|
|
||||||
## 4. Check the backup regularly
|
## 4. Check the backup regularly
|
||||||
|
|
||||||
@@ -155,6 +157,10 @@ occ backup:list offsite # the snapshots must be lis
|
|||||||
occ backup:restore:full offsite <snapshot>
|
occ backup:restore:full offsite <snapshot>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Without the kit, leave out `backup:key:import`: `backup:target:add` finds the backups and takes
|
||||||
|
the key from the location with a passphrase (`--passphrase-file=/path/passphrase`, or it asks).
|
||||||
|
Backups then wait until a new kit is downloaded and confirmed.
|
||||||
|
|
||||||
The command refuses on an installation that already has more than one user, and asks before it
|
The command refuses on an installation that already has more than one user, and asks before it
|
||||||
starts; `--force` skips both (also needed to rerun it after it stopped halfway).
|
starts; `--force` skips both (also needed to rerun it after it stopped halfway).
|
||||||
|
|
||||||
@@ -173,8 +179,13 @@ occ backup:prune offsite --dry-run # show first
|
|||||||
occ backup:prune offsite # apply the policy; forgotten snapshots go to the trash first
|
occ backup:prune offsite # apply the policy; forgotten snapshots go to the trash first
|
||||||
occ backup:trash # list the trash
|
occ backup:trash # list the trash
|
||||||
occ backup:trash:restore offsite <id>
|
occ backup:trash:restore offsite <id>
|
||||||
|
occ backup:pin offsite <id> # keep this restore point; --unpin to undo
|
||||||
|
occ backup:note offsite <id> "text" # a note you can change later; without text it is removed
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The retention policy leaves a pinned restore point alone, and a pinned one does not take the place of
|
||||||
|
a daily, weekly or monthly one. Pin and note are stored on the location, so a new server sees them too.
|
||||||
|
|
||||||
Snapshots are kept according to the policy, and their data is deleted only after the deletion delay
|
Snapshots are kept according to the policy, and their data is deleted only after the deletion delay
|
||||||
(set with `backup:key:init --delete-delay-days`). `occ backup:prune offsite --dry-run` shows what would be
|
(set with `backup:key:init --delete-delay-days`). `occ backup:prune offsite --dry-run` shows what would be
|
||||||
removed without changing anything.
|
removed without changing anything.
|
||||||
|
|||||||
@@ -67,6 +67,9 @@ final class LocalBackend implements IBackend {
|
|||||||
if ($fh === false) {
|
if ($fh === false) {
|
||||||
throw new BackendException('Not found: ' . $path);
|
throw new BackendException('Not found: ' . $path);
|
||||||
}
|
}
|
||||||
|
// Packs are read in large pieces: one request each, not 8 KiB at a time (see Repository::openForReading()).
|
||||||
|
stream_set_read_buffer($fh, 0);
|
||||||
|
stream_set_chunk_size($fh, 4 * 1048576);
|
||||||
return $fh;
|
return $fh;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -124,7 +124,13 @@ final class StorageBackend implements IBackend, IReadsAhead {
|
|||||||
// null: not supported or failing for now; false: not opened, which need not mean
|
// null: not supported or failing for now; false: not opened, which need not mean
|
||||||
// missing. Either way the storage's own stream decides.
|
// missing. Either way the storage's own stream decides.
|
||||||
}
|
}
|
||||||
|
try {
|
||||||
$fh = $this->storage->fopen($this->abs($path), 'r');
|
$fh = $this->storage->fopen($this->abs($path), 'r');
|
||||||
|
} catch (\Exception $e) {
|
||||||
|
// Some storages throw for what is not a missing file (SMB: a connection that broke).
|
||||||
|
// As a BackendException, the caller reads once more and --salvage can go on.
|
||||||
|
throw new BackendException('Cannot read ' . $path . ': ' . $e->getMessage(), 0, $e);
|
||||||
|
}
|
||||||
if ($fh === false) {
|
if ($fh === false) {
|
||||||
throw new BackendException('Not found: ' . $path);
|
throw new BackendException('Not found: ' . $path);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ final class BackupJob extends TimedJob {
|
|||||||
private IAppConfig $appConfig,
|
private IAppConfig $appConfig,
|
||||||
private LoggerInterface $logger,
|
private LoggerInterface $logger,
|
||||||
private \OCA\NgBackup\Service\Schedule $schedule,
|
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||||
|
private \OCA\NgBackup\Service\HeavyJobPause $heavyJobs,
|
||||||
) {
|
) {
|
||||||
parent::__construct($time);
|
parent::__construct($time);
|
||||||
// Well below the usual 5-minute cron interval: the last run is stamped a moment after cron
|
// Well below the usual 5-minute cron interval: the last run is stamped a moment after cron
|
||||||
@@ -43,25 +44,28 @@ final class BackupJob extends TimedJob {
|
|||||||
$this->setTimeSensitivity(IJob::TIME_SENSITIVE);
|
$this->setTimeSensitivity(IJob::TIME_SENSITIVE);
|
||||||
}
|
}
|
||||||
|
|
||||||
private const MAX_DAILY_ATTEMPTS = 3;
|
|
||||||
|
|
||||||
protected function run($argument): void {
|
protected function run($argument): void {
|
||||||
$budget = PHP_SAPI === 'cli' ? 240.0 : 20.0;
|
$budget = PHP_SAPI === 'cli' ? 240.0 : 20.0;
|
||||||
$deadline = microtime(true) + $budget;
|
$deadline = microtime(true) + $budget;
|
||||||
$this->startScheduled();
|
$this->startScheduled();
|
||||||
foreach ($this->runs->findRunning(null, BackupService::KIND_FULL) as $run) {
|
$this->heavyJobs->sync($this->backups->runsInTurn() !== []);
|
||||||
|
// One after the other: runs that read the same disks at the same time are not faster together.
|
||||||
|
foreach ($this->backups->runsInTurn() as $run) {
|
||||||
while (microtime(true) < $deadline && $run->getStatus() === 'running') {
|
while (microtime(true) < $deadline && $run->getStatus() === 'running') {
|
||||||
$next = $this->backups->step($run, min($deadline, microtime(true) + $budget));
|
$next = $this->backups->step($run, min($deadline, microtime(true) + $budget));
|
||||||
if ($next === $run) {
|
if ($next === $run) {
|
||||||
break; // step() hands back the same object when another process holds the run's lease
|
break 2; // step() hands back the same object when another process holds the run's lease
|
||||||
}
|
}
|
||||||
$run = $next;
|
$run = $next;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
$this->failStaleRestores();
|
$this->failStaleRestores();
|
||||||
$this->weeklyPrune();
|
$this->weeklyPrune();
|
||||||
foreach ($this->targets->list() as $t) {
|
$targets = $this->targets->list();
|
||||||
|
foreach ($targets as $t) {
|
||||||
$this->alerts->checkStale($t);
|
$this->alerts->checkStale($t);
|
||||||
|
$othersActive = array_filter($targets, static fn ($o) => $o->getId() !== $t->getId() && (string)$o->getSchedule() !== '') !== [];
|
||||||
|
$this->alerts->checkOff($t, $othersActive);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -102,21 +106,18 @@ final class BackupJob extends TimedJob {
|
|||||||
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
|
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Daily schedule: "HH:MM" in the schedule's time zone (see Schedule); empty = no schedule. */
|
/** Start the backups the schedules of the locations ask for; they run in turn (see LocationSchedule). */
|
||||||
private function startScheduled(): void {
|
private function startScheduled(): void {
|
||||||
$todayAt = $this->schedule->todayAt();
|
$now = time();
|
||||||
if ($todayAt === null || time() < $todayAt) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
foreach ($this->targets->list() as $t) {
|
foreach ($this->targets->list() as $t) {
|
||||||
$last = $this->runs->lastFinished($t->getId(), BackupService::KIND_FULL);
|
$moment = \OCA\NgBackup\Service\LocationSchedule::lastMoment((string)$t->getSchedule(), $this->schedule->zoneOr($t->getScheduleZone()), $now);
|
||||||
if ($this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== [] || ($last !== null && $last->getStartedAt() >= $todayAt)) {
|
if ($moment === null || $this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== []) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
// After a failure: retry at most MAX_DAILY_ATTEMPTS times a day, an hour apart, instead
|
$attempts = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $moment);
|
||||||
// of starting (and notifying about) a new full run every five minutes all day.
|
if (!\OCA\NgBackup\Service\LocationSchedule::isDue($moment, (int)$t->getScheduleSince(),
|
||||||
$today = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $todayAt);
|
$this->dataStarted($this->runs->lastFinished($t->getId(), BackupService::KIND_FULL)),
|
||||||
if (count($today) >= self::MAX_DAILY_ATTEMPTS || ($today !== [] && time() - (int)$today[0]->getFinishedAt() < 3600)) {
|
count($attempts), ($attempts[0] ?? null)?->getFinishedAt(), $now)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
@@ -126,4 +127,15 @@ final class BackupJob extends TimedJob {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** When a backup began reading its data: a queued one can start well after it was asked for. */
|
||||||
|
private function dataStarted(?\OCA\NgBackup\Db\Run $run): ?int {
|
||||||
|
if ($run === null) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
$state = json_decode($run->getState(), true);
|
||||||
|
$started = is_array($state) ? ($state['files']['started'] ?? null) : null;
|
||||||
|
$at = is_string($started) ? strtotime($started) : false;
|
||||||
|
return $at !== false ? $at : $run->getStartedAt();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ declare(strict_types=1);
|
|||||||
namespace OCA\NgBackup\Command;
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
use OCA\NgBackup\Service\KeyService;
|
use OCA\NgBackup\Service\KeyService;
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
use Symfony\Component\Console\Command\Command;
|
use Symfony\Component\Console\Command\Command;
|
||||||
use Symfony\Component\Console\Input\InputArgument;
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
use Symfony\Component\Console\Input\InputInterface;
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
@@ -18,6 +19,7 @@ use Symfony\Component\Console\Question\Question;
|
|||||||
final class KeyImport extends Command {
|
final class KeyImport extends Command {
|
||||||
public function __construct(
|
public function __construct(
|
||||||
private KeyService $keys,
|
private KeyService $keys,
|
||||||
|
private TargetService $targets,
|
||||||
) {
|
) {
|
||||||
parent::__construct();
|
parent::__construct();
|
||||||
}
|
}
|
||||||
@@ -27,11 +29,19 @@ final class KeyImport extends Command {
|
|||||||
->setDescription('Disaster recovery: adopt the backup key from a recovery kit on a fresh installation')
|
->setDescription('Disaster recovery: adopt the backup key from a recovery kit on a fresh installation')
|
||||||
->addArgument('kit-file', InputArgument::REQUIRED, 'The recovery kit file (occ backup:key:kit on the original installation)')
|
->addArgument('kit-file', InputArgument::REQUIRED, 'The recovery kit file (occ backup:key:kit on the original installation)')
|
||||||
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Read the passphrase from this file instead of asking')
|
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Read the passphrase from this file instead of asking')
|
||||||
|
->addOption('replace-unused-key', null, InputOption::VALUE_NONE, 'A new key was made here by mistake: replace it by the kit\'s key, if it was never used (no location, no backup)')
|
||||||
->setHelp(<<<'HELP'
|
->setHelp(<<<'HELP'
|
||||||
For a fresh installation only (occ backup:key:init must not have been run here yet). Reads one
|
For a fresh installation only (occ backup:key:init must not have been run here yet). Reads one
|
||||||
of the kit's passphrase slots to recover the original master key, so occ backup:target:add opens
|
of the kit's passphrase slots to recover the original master key, so occ backup:target:add opens
|
||||||
the existing repository on the location instead of creating a new, empty one.
|
the existing repository on the location instead of creating a new, empty one.
|
||||||
|
|
||||||
|
A new key was made here by mistake before the location was added? As long as it was never used
|
||||||
|
(no location, no backup), --replace-unused-key replaces it by the kit's key. A key that was used
|
||||||
|
is never replaced.
|
||||||
|
|
||||||
|
No kit? Skip this command: occ backup:target:add then takes the key from the location with one
|
||||||
|
of the old server's passphrases. The kit still helps when the location's config file is damaged.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
occ backup:key:import /root/ng_backup-recovery-kit.json
|
occ backup:key:import /root/ng_backup-recovery-kit.json
|
||||||
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=...
|
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=...
|
||||||
@@ -40,10 +50,24 @@ HELP);
|
|||||||
}
|
}
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
if ($this->keys->isInitialized()) {
|
$replace = (bool)$input->getOption('replace-unused-key');
|
||||||
$output->writeln('<error>A backup key already exists here (fingerprint ' . $this->keys->fingerprint() . '). occ backup:key:import is only for a fresh installation.</error>');
|
if ($replace && !$this->keys->isInitialized()) {
|
||||||
|
$output->writeln('<error>There is no backup key here to replace: leave out --replace-unused-key.</error>');
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
|
if ($this->keys->isInitialized()) {
|
||||||
|
if (!$this->targets->keyIsUnused()) {
|
||||||
|
$output->writeln('<error>A backup key already exists here (fingerprint ' . $this->keys->fingerprint() . ') and it has been used for a backup location: it is never replaced.</error>');
|
||||||
|
$output->writeln('occ backup:key:import is for a server that is being restored.');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if (!$replace) {
|
||||||
|
$output->writeln('<error>A backup key already exists here (fingerprint ' . $this->keys->fingerprint() . '), but it was never used: no location, no backup.</error>');
|
||||||
|
$output->writeln('Made by mistake on a server that is being restored? Run this command again with --replace-unused-key:');
|
||||||
|
$output->writeln('the kit\'s key then takes its place, and the passphrases and recovery kit of the unused key stop working.');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
$kitFile = (string)$input->getArgument('kit-file');
|
$kitFile = (string)$input->getArgument('kit-file');
|
||||||
$raw = @file_get_contents($kitFile);
|
$raw = @file_get_contents($kitFile);
|
||||||
if ($raw === false) {
|
if ($raw === false) {
|
||||||
@@ -70,12 +94,16 @@ HELP);
|
|||||||
$pass = (string)$helper->ask($input, $output, $q);
|
$pass = (string)$helper->ask($input, $output, $q);
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
|
if ($replace) {
|
||||||
|
$this->targets->importKitReplacingUnusedKey($kit, $pass);
|
||||||
|
} else {
|
||||||
$this->keys->importRecoveryKit($kit, $pass);
|
$this->keys->importRecoveryKit($kit, $pass);
|
||||||
|
}
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
$output->writeln('Backup key imported, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
$output->writeln(($replace ? 'The unused key was replaced. ' : '') . 'Backup key imported, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
||||||
// Read from the kit file, which is not authenticated beyond the wrapped key: show it so a
|
// Read from the kit file, which is not authenticated beyond the wrapped key: show it so a
|
||||||
// wrong or altered value stands out.
|
// wrong or altered value stands out.
|
||||||
$output->writeln('Deletion delay for forgotten snapshots: <info>' . $this->keys->deleteDelayDays() . '</info> days (from the kit).');
|
$output->writeln('Deletion delay for forgotten snapshots: <info>' . $this->keys->deleteDelayDays() . '</info> days (from the kit).');
|
||||||
|
|||||||
@@ -40,14 +40,14 @@ final class ListSnapshots extends Command {
|
|||||||
foreach ($targets as $t) {
|
foreach ($targets as $t) {
|
||||||
foreach ($this->snapshots->findForTarget($t->getId()) as $s) {
|
foreach ($this->snapshots->findForTarget($t->getId()) as $s) {
|
||||||
$rows[] = [$t->getName(), $s->getSnapshotId(), Format::time($s->getCreatedAt(), $zone), $s->getKind(),
|
$rows[] = [$t->getName(), $s->getSnapshotId(), Format::time($s->getCreatedAt(), $zone), $s->getKind(),
|
||||||
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getLabel() ?? ''];
|
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getPinned() ? 'yes' : '', $s->getLabel() ?? '', $s->getNote() ?? ''];
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if ($rows === []) {
|
if ($rows === []) {
|
||||||
$output->writeln('No snapshots yet (occ backup:run).');
|
$output->writeln('No snapshots yet (occ backup:run).');
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'label'])->setRows($rows)->render();
|
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'pinned', 'label', 'note'])->setRows($rows)->render();
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\PruneService;
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class Note extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private PruneService $prune,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:note')
|
||||||
|
->setDescription('Set or remove the note on a snapshot')
|
||||||
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
|
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||||
|
->addArgument('note', InputArgument::OPTIONAL, 'The note, at most ' . PruneService::NOTE_MAX . ' characters; without it the note is removed', '');
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$note = (string)$input->getArgument('note');
|
||||||
|
try {
|
||||||
|
$this->prune->setNote($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), $note);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln(trim($note) === '' ? 'Note removed.' : 'Note saved.');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\HeavyJobPause;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class PauseJobs extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private HeavyJobPause $pause,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:pause-jobs')
|
||||||
|
->setDescription('Show or set whether heavy background jobs (Memories, Recognize, Preview Generator, ...) pause while a backup runs')
|
||||||
|
->addArgument('setting', InputArgument::OPTIONAL, '"on" or "off"; omit to show the current one');
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$setting = $input->getArgument('setting');
|
||||||
|
if ($setting !== null) {
|
||||||
|
if ($setting !== 'on' && $setting !== 'off') {
|
||||||
|
$output->writeln('<error>Use "on" or "off".</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$this->pause->setEnabled($setting === 'on');
|
||||||
|
}
|
||||||
|
$status = $this->pause->status();
|
||||||
|
$output->writeln('Pausing heavy background jobs during a backup: ' . ($status['enabled'] ? 'on' : 'off'));
|
||||||
|
if ($status['jobs'] !== []) {
|
||||||
|
$output->writeln('Paused now: ' . implode(', ', $status['jobs']));
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\PruneService;
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Input\InputOption;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class Pin extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private PruneService $prune,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:pin')
|
||||||
|
->setDescription('Keep a snapshot: the retention policy does not remove a pinned one')
|
||||||
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
|
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||||
|
->addOption('unpin', null, InputOption::VALUE_NONE, 'Leave the snapshot to the retention policy again');
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$unpin = (bool)$input->getOption('unpin');
|
||||||
|
try {
|
||||||
|
$this->prune->setPinned($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), !$unpin);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln($unpin ? 'Snapshot unpinned; the retention policy applies to it again.' : 'Snapshot pinned; the retention policy leaves it alone.');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -79,6 +79,9 @@ HELP);
|
|||||||
if (($f['unreadable'] ?? 0) > 0) {
|
if (($f['unreadable'] ?? 0) > 0) {
|
||||||
$output->writeln(sprintf('<comment> warning: %d unreadable files or folders are missing from this backup (see the server log)</comment>', $f['unreadable']));
|
$output->writeln(sprintf('<comment> warning: %d unreadable files or folders are missing from this backup (see the server log)</comment>', $f['unreadable']));
|
||||||
}
|
}
|
||||||
|
if (($f['badNames'] ?? 0) > 0) {
|
||||||
|
$output->writeln(sprintf('<comment> warning: %d files or folders are missing from this backup because their name is not supported; rename them and the next backup includes them (see the server log)</comment>', $f['badNames']));
|
||||||
|
}
|
||||||
if (($f['changing'] ?? 0) > 0) {
|
if (($f['changing'] ?? 0) > 0) {
|
||||||
$output->writeln(sprintf('<comment> warning: %d files kept changing and were saved as read (see the server log)</comment>', $f['changing']));
|
$output->writeln(sprintf('<comment> warning: %d files kept changing and were saved as read (see the server log)</comment>', $f['changing']));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ final class Status extends Command {
|
|||||||
private VerifyService $verify,
|
private VerifyService $verify,
|
||||||
private Schedule $schedule,
|
private Schedule $schedule,
|
||||||
private CronHealth $cron,
|
private CronHealth $cron,
|
||||||
|
private \OCA\NgBackup\Service\BackupService $backups,
|
||||||
) {
|
) {
|
||||||
parent::__construct();
|
parent::__construct();
|
||||||
}
|
}
|
||||||
@@ -61,9 +62,11 @@ final class Status extends Command {
|
|||||||
$rows = [];
|
$rows = [];
|
||||||
$all = (bool)$input->getOption('all');
|
$all = (bool)$input->getOption('all');
|
||||||
$shown = $this->runs->findShown($all ? null : time() - self::RECENT_DAYS * 86400);
|
$shown = $this->runs->findShown($all ? null : time() - self::RECENT_DAYS * 86400);
|
||||||
|
$queued = array_map(static fn (\OCA\NgBackup\Db\Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1));
|
||||||
foreach ($shown as $r) {
|
foreach ($shown as $r) {
|
||||||
$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
|
$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
|
||||||
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus(),
|
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(),
|
||||||
|
in_array($r->getId(), $queued, true) ? 'queued' : ($r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus()),
|
||||||
Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
|
Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
|
||||||
$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
|
$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
|
||||||
isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
|
isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
|
||||||
|
|||||||
@@ -13,7 +13,9 @@ use Symfony\Component\Console\Command\Command;
|
|||||||
use Symfony\Component\Console\Input\InputArgument;
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
use Symfony\Component\Console\Input\InputInterface;
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
use Symfony\Component\Console\Input\InputOption;
|
use Symfony\Component\Console\Input\InputOption;
|
||||||
|
use Symfony\Component\Console\Helper\QuestionHelper;
|
||||||
use Symfony\Component\Console\Output\OutputInterface;
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
use Symfony\Component\Console\Question\Question;
|
||||||
|
|
||||||
final class TargetAdd extends Command {
|
final class TargetAdd extends Command {
|
||||||
public function __construct(
|
public function __construct(
|
||||||
@@ -31,10 +33,25 @@ final class TargetAdd extends Command {
|
|||||||
->addOption('auth', 'a', InputOption::VALUE_REQUIRED, 'Auth mechanism id (see backup:target:backends)')
|
->addOption('auth', 'a', InputOption::VALUE_REQUIRED, 'Auth mechanism id (see backup:target:backends)')
|
||||||
->addOption('option', 'o', InputOption::VALUE_REQUIRED | InputOption::VALUE_IS_ARRAY, 'Backend/auth option key=value (repeatable)')
|
->addOption('option', 'o', InputOption::VALUE_REQUIRED | InputOption::VALUE_IS_ARRAY, 'Backend/auth option key=value (repeatable)')
|
||||||
->addOption('path', 'p', InputOption::VALUE_REQUIRED, 'Folder on the location for this repository', 'ng_backup')
|
->addOption('path', 'p', InputOption::VALUE_REQUIRED, 'Folder on the location for this repository', 'ng_backup')
|
||||||
|
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Disaster recovery without the recovery kit: read the passphrase from this file instead of asking')
|
||||||
|
->addOption('repair-config', null, InputOption::VALUE_NONE, 'The location\'s config file is gone: write it again with the key of this server or of the imported recovery kit (locations backed up by 1.1.0 or newer)')
|
||||||
|
->addOption('replace-unused-key', null, InputOption::VALUE_NONE, 'Disaster recovery after a new key was made here by mistake: replace that key, if it was never used, by the key of the backups at this location (asks for a passphrase of the old server)')
|
||||||
->setHelp(<<<'HELP'
|
->setHelp(<<<'HELP'
|
||||||
The location is reached with files_external's own connection code, but it is never added as an
|
The location is reached with files_external's own connection code, but it is never added as an
|
||||||
external storage: it does not appear in Files for anyone. Credentials are stored encrypted.
|
external storage: it does not appear in Files for anyone. Credentials are stored encrypted.
|
||||||
|
|
||||||
|
Disaster recovery without the recovery kit: on a server without a backup key, a location that
|
||||||
|
holds backups gives its key with one of the old server's passphrases. Run this command first
|
||||||
|
(no occ backup:key:import), then occ backup:list and occ backup:restore:full.
|
||||||
|
|
||||||
|
A new key was made here by mistake (occ backup:key:init or the web page) before the location
|
||||||
|
was added? As long as that key was never used (no location, no backup), --replace-unused-key
|
||||||
|
replaces it by the key of the backups. A key that was used is never replaced.
|
||||||
|
|
||||||
|
With the recovery kit: occ backup:key:import first; this command then asks for nothing more.
|
||||||
|
If the location's config file is gone, only the kit has the key: import it, then add the
|
||||||
|
location with --repair-config.
|
||||||
|
|
||||||
Examples:
|
Examples:
|
||||||
occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket=backups -o hostname=s3.example.com -o region=eu-central-1 -o use_ssl=true -o use_path_style=true -o key=AKIA... -o secret=... --path=nextcloud
|
occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket=backups -o hostname=s3.example.com -o region=eu-central-1 -o use_ssl=true -o use_path_style=true -o key=AKIA... -o secret=... --path=nextcloud
|
||||||
|
|
||||||
@@ -43,13 +60,6 @@ HELP);
|
|||||||
}
|
}
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
if (!$this->keys->isInitialized()) {
|
|
||||||
$output->writeln('<error>No backup key yet.</error>');
|
|
||||||
$output->writeln('Restoring from a location that already has backups (disaster recovery)? Import the key from the recovery kit first:');
|
|
||||||
$output->writeln(' occ backup:key:import /path/to/ng_backup-kit.json');
|
|
||||||
$output->writeln('Only for a new location without backups: create a new key with occ backup:key:init. A new key cannot read existing backups.');
|
|
||||||
return 1;
|
|
||||||
}
|
|
||||||
$options = [];
|
$options = [];
|
||||||
foreach ($input->getOption('option') as $pair) {
|
foreach ($input->getOption('option') as $pair) {
|
||||||
if (!str_contains($pair, '=')) {
|
if (!str_contains($pair, '=')) {
|
||||||
@@ -63,24 +73,125 @@ HELP);
|
|||||||
$output->writeln('<error>--auth is required (see occ backup:target:backends)</error>');
|
$output->writeln('<error>--auth is required (see occ backup:target:backends)</error>');
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
|
$name = (string)$input->getArgument('name');
|
||||||
|
$backend = (string)$input->getArgument('backend');
|
||||||
|
$auth = (string)$input->getOption('auth');
|
||||||
|
$path = (string)$input->getOption('path');
|
||||||
|
$passphrase = null;
|
||||||
|
$repair = (bool)$input->getOption('repair-config');
|
||||||
|
$replace = (bool)$input->getOption('replace-unused-key');
|
||||||
|
if ($replace) {
|
||||||
|
if ($repair) {
|
||||||
|
$output->writeln('<error>--replace-unused-key needs the config at the location; with the config gone only the kit helps: occ backup:key:import <kit file> --replace-unused-key, then --repair-config.</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if (!$this->keys->isInitialized()) {
|
||||||
|
$output->writeln('<error>There is no backup key here to replace: leave out --replace-unused-key.</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if (!$this->targets->keyIsUnused()) {
|
||||||
|
$output->writeln('<error>The backup key of this server (fingerprint ' . $this->keys->fingerprint() . ') has been used for a backup location: it is never replaced.</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln('The backup key of this server (fingerprint ' . $this->keys->fingerprint() . ') was never used. It is replaced by the key of the backups at');
|
||||||
|
$output->writeln('this location, once a passphrase of the old server opens it. Its own passphrases and recovery kit then stop working.');
|
||||||
|
$passphrase = $this->passphrase($input, $output);
|
||||||
|
if ($passphrase === null) {
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
} elseif (!$this->keys->isInitialized()) {
|
||||||
|
try {
|
||||||
|
$state = $this->targets->locationState($backend, $auth, $options, $path);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($state === TargetService::LOCATION_EMPTY) {
|
||||||
|
$output->writeln('<error>No backup key yet, and no backups at this location.</error>');
|
||||||
|
$output->writeln('Restoring a server (disaster recovery)? Check the connection details and --path: they must be the same as on the old server.');
|
||||||
|
$output->writeln('Only for a new location: create a new key first with occ backup:key:init. A new key cannot read existing backups.');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($state === TargetService::LOCATION_DATA_WITHOUT_CONFIG) {
|
||||||
|
$output->writeln('<error>This location holds backups, but its config file is gone, and the key was stored in it.</error>');
|
||||||
|
$output->writeln('Now only the recovery kit has the key. Import it, then run this command again with --repair-config:');
|
||||||
|
$output->writeln(' occ backup:key:import /path/to/ng_backup-kit.json');
|
||||||
|
$output->writeln('Do not run occ backup:key:init: a new key cannot read these backups.');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($repair) {
|
||||||
|
$output->writeln('<error>The config at this location is there: leave out --repair-config.</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$output->writeln('No backup key on this server yet, and this location holds backups. Their key is stored there too,');
|
||||||
|
$output->writeln('locked with the passphrases of the old server: one of them unlocks it (the recovery kit is not needed).');
|
||||||
|
$passphrase = $this->passphrase($input, $output);
|
||||||
|
if ($passphrase === null) {
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
// Adopting an existing repository reads its whole index first, which takes a while on a
|
// Adopting an existing repository reads its whole index first, which takes a while on a
|
||||||
// large backup or a slow line; show how far it is instead of looking stuck.
|
// large backup or a slow line; show how far it is instead of looking stuck.
|
||||||
$output->writeln('Connecting to the location; with existing backups this reads their index first ...');
|
$output->writeln('Connecting to the location; with existing backups this reads their index first ...');
|
||||||
try {
|
try {
|
||||||
$r = $this->targets->add($input->getArgument('name'), $input->getArgument('backend'), $input->getOption('auth'), $options,
|
$r = $passphrase === null
|
||||||
(string)$input->getOption('path'), self::progress($output));
|
? $this->targets->add($name, $backend, $auth, $options, $path, self::progress($output), $repair)
|
||||||
|
: $this->targets->addWithPassphrase($name, $backend, $auth, $options, $path, $passphrase, self::progress($output), $replace);
|
||||||
|
} catch (\OCA\NgBackup\Crypto\CryptoException $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
if (!$this->keys->isInitialized() || $replace) {
|
||||||
|
$output->writeln('Nothing was changed. Run the command again to try another passphrase, or use the recovery kit (occ backup:key:import).');
|
||||||
|
}
|
||||||
|
return 1;
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
if ($output->isDecorated()) {
|
if ($output->isDecorated()) {
|
||||||
$output->write("\r\033[K"); // clear an unfinished progress line
|
$output->write("\r\033[K"); // clear an unfinished progress line
|
||||||
}
|
}
|
||||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
if (str_contains($e->getMessage(), 'different key') && !$replace) {
|
||||||
|
$output->writeln('The backup key here (fingerprint ' . $this->keys->fingerprint() . ') is not the one these backups were made with.');
|
||||||
|
$output->writeln('Restoring a server? The recovery kit must be the old server\'s: its fingerprint is in the kit file. Nothing was changed.');
|
||||||
|
}
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
|
if ($repair) {
|
||||||
|
$output->writeln('The config of the location was written again from the recovery kit.');
|
||||||
|
}
|
||||||
$output->writeln(sprintf('Location <info>%s</info> added (id %d), repository %s %s.', $r['target']->getName(), $r['target']->getId(),
|
$output->writeln(sprintf('Location <info>%s</info> added (id %d), repository %s %s.', $r['target']->getName(), $r['target']->getId(),
|
||||||
$r['target']->getRepositoryId(), $r['created'] ? 'created' : 'found and adopted'));
|
$r['target']->getRepositoryId(), $r['created'] ? 'created' : 'found and adopted'));
|
||||||
|
if ($passphrase !== null) {
|
||||||
|
$output->writeln(($replace ? 'The unused key was replaced. ' : '') . 'Backup key taken from the location, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
||||||
|
$days = 'Deletion delay for forgotten snapshots: <info>' . $this->keys->deleteDelayDays() . '</info> days';
|
||||||
|
$output->writeln(match (true) {
|
||||||
|
($r['deleteDelayDays'] ?? null) !== null => $days . ' (from the location).',
|
||||||
|
$r['settingsUnreadable'] ?? false => '<comment>' . $days . ' (the longest: the settings on the location could not be read or hold no valid delay, so nothing leaves the Bin sooner than on the old server; removed restore points take space for that long).</comment>',
|
||||||
|
default => $days . ' (the default: the location has none stored, its backups were made before NG Backup 1.1.0).',
|
||||||
|
});
|
||||||
|
$output->writeln('Next: <info>occ backup:list</info> and <info>occ backup:restore:full</info>. Before backups run again, download a new recovery kit and confirm it (<info>occ backup:key:kit</info>, <info>occ backup:key:confirm</info>).');
|
||||||
|
}
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private function passphrase(InputInterface $input, OutputInterface $output): ?string {
|
||||||
|
$file = $input->getOption('passphrase-file');
|
||||||
|
if ($file !== null) {
|
||||||
|
$raw = @file_get_contents((string)$file);
|
||||||
|
if ($raw === false) {
|
||||||
|
$output->writeln('<error>Cannot read the passphrase file</error>');
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return rtrim($raw, "\r\n");
|
||||||
|
}
|
||||||
|
if (!$input->isInteractive()) {
|
||||||
|
$output->writeln('<error>Give the passphrase with --passphrase-file, or run the command without --no-interaction to type it.</error>');
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
$q = (new Question('Passphrase of the old server (one of its slots): '))->setHidden(true)->setHiddenFallback(false);
|
||||||
|
/** @var QuestionHelper $helper */
|
||||||
|
$helper = $this->getHelper('question');
|
||||||
|
return (string)$helper->ask($input, $output, $q);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Progress of reading the index: on a terminal one line that is updated in place, otherwise
|
* Progress of reading the index: on a terminal one line that is updated in place, otherwise
|
||||||
* (a log, a pipe) a line every 30 seconds. The time left is estimated from the rate so far.
|
* (a log, a pipe) a line every 30 seconds. The time left is estimated from the rate so far.
|
||||||
|
|||||||
@@ -31,7 +31,8 @@ final class TargetList extends Base {
|
|||||||
$rows = [];
|
$rows = [];
|
||||||
foreach ($this->targets->list() as $t) {
|
foreach ($this->targets->list() as $t) {
|
||||||
$row = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
$row = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
||||||
'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt())];
|
'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt()),
|
||||||
|
'schedule' => (string)$t->getSchedule() !== '' ? $t->getSchedule() : 'off'];
|
||||||
if ($input->getOption('test')) {
|
if ($input->getOption('test')) {
|
||||||
try {
|
try {
|
||||||
$repo = $this->targets->repository($t);
|
$repo = $this->targets->repository($t);
|
||||||
|
|||||||
@@ -0,0 +1,54 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\Schedule;
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Input\InputOption;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class TargetSchedule extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
private Schedule $schedule,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:target:schedule')
|
||||||
|
->setDescription('Show or set when a location is backed up')
|
||||||
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
|
->addArgument('schedule', InputArgument::OPTIONAL, '"off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM; omit to show the current one')
|
||||||
|
->addOption('timezone', null, InputOption::VALUE_REQUIRED, 'Time zone of the time, e.g. Europe/Amsterdam');
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$t = $this->targets->get($input->getArgument('target'));
|
||||||
|
$schedule = $input->getArgument('schedule');
|
||||||
|
if ($schedule === null && $input->getOption('timezone') !== null) {
|
||||||
|
$output->writeln('<error>--timezone goes with a schedule, e.g. backup:target:schedule ' . $t->getName() . ' 02:30 --timezone=Europe/Amsterdam</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
$zone = $input->getOption('timezone') ?? $this->schedule->zoneOr($t->getScheduleZone());
|
||||||
|
if ($schedule !== null) {
|
||||||
|
try {
|
||||||
|
$t = $this->targets->setSchedule($t, $schedule, (string)$zone);
|
||||||
|
} catch (\InvalidArgumentException $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$current = (string)$t->getSchedule();
|
||||||
|
$output->writeln($t->getName() . ': ' . ($current === '' ? 'off' : $current . ' (' . $zone . ')'));
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -48,6 +48,7 @@ final class AdminApiController extends Controller {
|
|||||||
private \OCA\NgBackup\Service\KeySlotService $slots,
|
private \OCA\NgBackup\Service\KeySlotService $slots,
|
||||||
private \OCA\NgBackup\Service\PruneService $prune,
|
private \OCA\NgBackup\Service\PruneService $prune,
|
||||||
private \OCA\NgBackup\Service\Schedule $schedule,
|
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||||
|
private \OCA\NgBackup\Service\HeavyJobPause $heavyJobs,
|
||||||
private \OCA\NgBackup\Service\CronHealth $cron,
|
private \OCA\NgBackup\Service\CronHealth $cron,
|
||||||
) {
|
) {
|
||||||
parent::__construct(Application::APP_ID, $request);
|
parent::__construct(Application::APP_ID, $request);
|
||||||
@@ -59,10 +60,12 @@ final class AdminApiController extends Controller {
|
|||||||
foreach ($this->targets->list() as $t) {
|
foreach ($this->targets->list() as $t) {
|
||||||
$targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
$targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
||||||
'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt(), 'appendOnly' => (bool)$t->getAppendOnly(),
|
'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt(), 'appendOnly' => (bool)$t->getAppendOnly(),
|
||||||
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t)];
|
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t),
|
||||||
|
'schedule' => (string)$t->getSchedule(), 'scheduleZone' => $this->schedule->zoneOr($t->getScheduleZone())];
|
||||||
}
|
}
|
||||||
$shown = $this->runs->findShown($allRuns ? null : time() - \OCA\NgBackup\Command\Status::RECENT_DAYS * 86400);
|
$shown = $this->runs->findShown($allRuns ? null : time() - \OCA\NgBackup\Command\Status::RECENT_DAYS * 86400);
|
||||||
$runs = array_map(fn (Run $r) => $this->runJson($r), $shown);
|
$queued = array_flip(array_map(static fn (Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1)));
|
||||||
|
$runs = array_map(fn (Run $r) => $this->runJson($r) + ['queued' => isset($queued[$r->getId()])], $shown);
|
||||||
return new JSONResponse([
|
return new JSONResponse([
|
||||||
'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(),
|
'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(),
|
||||||
'confirmation' => $this->keys->recoveryKitConfirmation(), 'everConfirmed' => $this->keys->isInitialized() && $this->keys->everConfirmed(),
|
'confirmation' => $this->keys->recoveryKitConfirmation(), 'everConfirmed' => $this->keys->isInitialized() && $this->keys->everConfirmed(),
|
||||||
@@ -74,14 +77,20 @@ final class AdminApiController extends Controller {
|
|||||||
'runs' => $runs,
|
'runs' => $runs,
|
||||||
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
|
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
|
||||||
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
|
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
|
||||||
'schedule' => $this->schedule->get()['time'],
|
|
||||||
'scheduleTimezone' => $this->schedule->get()['timezone'],
|
|
||||||
'cron' => $this->cron->check(),
|
'cron' => $this->cron->check(),
|
||||||
|
'heavyJobs' => $this->heavyJobs->status(),
|
||||||
]);
|
]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function setPauseHeavyJobs(bool $enabled): JSONResponse {
|
||||||
|
$this->heavyJobs->setEnabled($enabled);
|
||||||
|
return new JSONResponse(['enabled' => $enabled]);
|
||||||
|
}
|
||||||
|
|
||||||
/** Seconds of backup work one tick() does: well within a normal web request. */
|
/** Seconds of backup work one tick() does: well within a normal web request. */
|
||||||
public const TICK_SECONDS = 10.0;
|
public const TICK_SECONDS = 10.0;
|
||||||
|
/** Seconds one request for the Bin reads, before the page shows how far it is and asks for more. */
|
||||||
|
private const TRASH_PAGE_SECONDS = 5.0;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Move running backups forward from the open admin page, when background jobs do not run on a
|
* Move running backups forward from the open admin page, when background jobs do not run on a
|
||||||
@@ -95,13 +104,17 @@ final class AdminApiController extends Controller {
|
|||||||
}
|
}
|
||||||
$deadline = microtime(true) + self::TICK_SECONDS;
|
$deadline = microtime(true) + self::TICK_SECONDS;
|
||||||
$stepped = 0;
|
$stepped = 0;
|
||||||
foreach ($this->runs->findRunning(null, BackupService::KIND_FULL) as $run) {
|
$this->heavyJobs->sync($this->backups->runsInTurn() !== []);
|
||||||
|
foreach ($this->backups->runsInTurn() as $run) {
|
||||||
if (microtime(true) >= $deadline) {
|
if (microtime(true) >= $deadline) {
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
// step() returns at once when the background job (or another tab) holds the run.
|
// step() returns at once when the background job (or another tab) holds the run.
|
||||||
$this->backups->step($run, $deadline);
|
$next = $this->backups->step($run, $deadline);
|
||||||
$stepped++;
|
$stepped++;
|
||||||
|
if ($next->getStatus() === Run::RUNNING) {
|
||||||
|
break; // not finished: the next one waits for its turn
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return new JSONResponse(['stepped' => $stepped]);
|
return new JSONResponse(['stepped' => $stepped]);
|
||||||
}
|
}
|
||||||
@@ -174,15 +187,18 @@ final class AdminApiController extends Controller {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public function trash(int $targetId): JSONResponse {
|
public function trash(int $targetId, string $after = ''): JSONResponse {
|
||||||
try {
|
try {
|
||||||
$delay = $this->keys->deleteDelayDays() * 86400;
|
$delay = $this->keys->deleteDelayDays() * 86400;
|
||||||
|
$repo = $this->targets->repository($this->targets->get((string)$targetId));
|
||||||
|
// In parts, so that the page can show how far it is (a read per removed restore point).
|
||||||
|
$page = $repo->trashPage($after, microtime(true) + self::TRASH_PAGE_SECONDS);
|
||||||
$out = [];
|
$out = [];
|
||||||
foreach ($this->targets->repository($this->targets->get((string)$targetId))->trash() as $id => $info) {
|
foreach ($page['items'] as $id => $info) {
|
||||||
$out[] = ['id' => $id, 'label' => $info['label'], 'time' => $info['time'], 'removedAt' => $info['forgottenAt'],
|
$out[] = ['id' => $id, 'label' => $info['label'], 'time' => $info['time'], 'removedAt' => $info['forgottenAt'],
|
||||||
'by' => $info['by'], 'deleteAfter' => $info['forgottenAt'] + $delay];
|
'by' => $info['by'], 'deleteAfter' => $info['forgottenAt'] + $delay];
|
||||||
}
|
}
|
||||||
return new JSONResponse($out);
|
return new JSONResponse(['total' => $page['total'], 'items' => $out]);
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||||
}
|
}
|
||||||
@@ -270,25 +286,43 @@ final class AdminApiController extends Controller {
|
|||||||
return new JSONResponse($this->runJson($run));
|
return new JSONResponse($this->runJson($run));
|
||||||
}
|
}
|
||||||
|
|
||||||
/** @param string $timezone the administrator's zone (the browser's); empty: keep the current one */
|
/** @param string $timezone the administrator's zone (the browser's); empty: the default one */
|
||||||
public function setSchedule(string $time, string $timezone = ''): JSONResponse {
|
public function setTargetSchedule(int $id, string $schedule, string $timezone = ''): JSONResponse {
|
||||||
try {
|
try {
|
||||||
$this->schedule->set($time, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
$t = $this->targets->setSchedule($this->targets->get((string)$id), $schedule, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
||||||
} catch (\InvalidArgumentException $e) {
|
} catch (\Throwable $e) {
|
||||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||||
}
|
}
|
||||||
return new JSONResponse(['schedule' => $time, 'scheduleTimezone' => $this->schedule->get()['timezone']]);
|
return new JSONResponse(['schedule' => $t->getSchedule(), 'scheduleZone' => $t->getScheduleZone()]);
|
||||||
}
|
}
|
||||||
|
|
||||||
public function snapshots(int $targetId): JSONResponse {
|
public function snapshots(int $targetId): JSONResponse {
|
||||||
$out = [];
|
$out = [];
|
||||||
foreach ($this->snapshots->findForTarget($targetId) as $s) {
|
foreach ($this->snapshots->findForTarget($targetId) as $s) {
|
||||||
$out[] = ['id' => $s->getSnapshotId(), 'kind' => $s->getKind(), 'label' => $s->getLabel(), 'createdAt' => $s->getCreatedAt(),
|
$out[] = ['id' => $s->getSnapshotId(), 'kind' => $s->getKind(), 'label' => $s->getLabel(), 'createdAt' => $s->getCreatedAt(),
|
||||||
'files' => $s->getFiles(), 'bytes' => $s->getBytes()];
|
'files' => $s->getFiles(), 'bytes' => $s->getBytes(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
|
||||||
}
|
}
|
||||||
return new JSONResponse($out);
|
return new JSONResponse($out);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function setSnapshotNote(int $targetId, string $snapshotId, string $note = ''): JSONResponse {
|
||||||
|
try {
|
||||||
|
$this->prune->setNote($this->targets->get((string)$targetId), $snapshotId, $note);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||||
|
}
|
||||||
|
return new JSONResponse(['ok' => true]);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function setSnapshotPinned(int $targetId, string $snapshotId, bool $pinned = true): JSONResponse {
|
||||||
|
try {
|
||||||
|
$this->prune->setPinned($this->targets->get((string)$targetId), $snapshotId, $pinned);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||||
|
}
|
||||||
|
return new JSONResponse(['ok' => true]);
|
||||||
|
}
|
||||||
|
|
||||||
public function browse(int $targetId, string $snapshotId, string $path = ''): JSONResponse {
|
public function browse(int $targetId, string $snapshotId, string $path = ''): JSONResponse {
|
||||||
try {
|
try {
|
||||||
return new JSONResponse($this->restore->browse($this->targets->get((string)$targetId), $snapshotId, $path));
|
return new JSONResponse($this->restore->browse($this->targets->get((string)$targetId), $snapshotId, $path));
|
||||||
@@ -333,7 +367,7 @@ final class AdminApiController extends Controller {
|
|||||||
if ($r->getSnapshotId() !== null) {
|
if ($r->getSnapshotId() !== null) {
|
||||||
try {
|
try {
|
||||||
$s = $this->snapshots->findOne($r->getTargetId(), $r->getSnapshotId());
|
$s = $this->snapshots->findOne($r->getTargetId(), $r->getSnapshotId());
|
||||||
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel()];
|
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
|
||||||
} catch (\Throwable) {
|
} catch (\Throwable) {
|
||||||
// Snapshot already pruned or trashed: the run keeps its id, just not the date.
|
// Snapshot already pruned or trashed: the run keeps its id, just not the date.
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -87,14 +87,20 @@ final class KeyRing {
|
|||||||
foreach ($field['slots'] as $slot) {
|
foreach ($field['slots'] as $slot) {
|
||||||
try {
|
try {
|
||||||
return self::unwrap($slot['key'], $passphrase);
|
return self::unwrap($slot['key'], $passphrase);
|
||||||
} catch (CryptoException) {
|
} catch (CryptoException|\SodiumException|\TypeError|\ValueError) {
|
||||||
|
// a wrong passphrase for this slot, or a damaged slot: the next one may still open
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
throw new CryptoException('Wrong passphrase or damaged key');
|
throw new CryptoException('Wrong passphrase or damaged key');
|
||||||
}
|
}
|
||||||
|
|
||||||
/** @param array{v:int, salt:string, ops:int, mem:int, nonce:string, key:string} $wrapped */
|
/** @param array<string, mixed> $wrapped from a kit file or a location's config: checked here, not trusted */
|
||||||
public static function unwrap(array $wrapped, #[\SensitiveParameter] string $passphrase): self {
|
public static function unwrap(array $wrapped, #[\SensitiveParameter] string $passphrase): self {
|
||||||
|
if (!isset($wrapped['salt'], $wrapped['nonce'], $wrapped['key'], $wrapped['ops'], $wrapped['mem'])
|
||||||
|
|| !is_string($wrapped['salt']) || !is_string($wrapped['nonce']) || !is_string($wrapped['key'])
|
||||||
|
|| !is_int($wrapped['ops']) || !is_int($wrapped['mem'])) {
|
||||||
|
throw new CryptoException('Damaged key file: a part of the key is missing');
|
||||||
|
}
|
||||||
$b64 = static fn (string $s): string => sodium_base642bin($s, SODIUM_BASE64_VARIANT_ORIGINAL);
|
$b64 = static fn (string $s): string => sodium_base642bin($s, SODIUM_BASE64_VARIANT_ORIGINAL);
|
||||||
// ops/mem come from unauthenticated JSON (kit file, repository config): bound them, so a
|
// ops/mem come from unauthenticated JSON (kit file, repository config): bound them, so a
|
||||||
// damaged or crafted file cannot make the KDF use unbounded memory or time.
|
// damaged or crafted file cannot make the KDF use unbounded memory or time.
|
||||||
@@ -102,10 +108,22 @@ final class KeyRing {
|
|||||||
|| $wrapped['mem'] < SODIUM_CRYPTO_PWHASH_MEMLIMIT_INTERACTIVE || $wrapped['mem'] > SODIUM_CRYPTO_PWHASH_MEMLIMIT_SENSITIVE) {
|
|| $wrapped['mem'] < SODIUM_CRYPTO_PWHASH_MEMLIMIT_INTERACTIVE || $wrapped['mem'] > SODIUM_CRYPTO_PWHASH_MEMLIMIT_SENSITIVE) {
|
||||||
throw new CryptoException('Unsupported key derivation parameters (damaged or foreign key file)');
|
throw new CryptoException('Unsupported key derivation parameters (damaged or foreign key file)');
|
||||||
}
|
}
|
||||||
$kek = sodium_crypto_pwhash(SODIUM_CRYPTO_SECRETBOX_KEYBYTES, $passphrase, $b64($wrapped['salt']),
|
$kek = null;
|
||||||
|
try {
|
||||||
|
$salt = $b64($wrapped['salt']);
|
||||||
|
$sealed = $b64($wrapped['key']);
|
||||||
|
$nonce = $b64($wrapped['nonce']);
|
||||||
|
$kek = sodium_crypto_pwhash(SODIUM_CRYPTO_SECRETBOX_KEYBYTES, $passphrase, $salt,
|
||||||
$wrapped['ops'], $wrapped['mem'], SODIUM_CRYPTO_PWHASH_ALG_ARGON2ID13);
|
$wrapped['ops'], $wrapped['mem'], SODIUM_CRYPTO_PWHASH_ALG_ARGON2ID13);
|
||||||
$master = sodium_crypto_secretbox_open($b64($wrapped['key']), $b64($wrapped['nonce']), $kek);
|
$master = sodium_crypto_secretbox_open($sealed, $nonce, $kek);
|
||||||
|
} catch (\SodiumException|\ValueError $e) {
|
||||||
|
// invalid base64 or a salt/nonce of the wrong length: damaged, so the caller shows its recovery options
|
||||||
|
throw new CryptoException('Damaged key file: ' . $e->getMessage(), 0, $e);
|
||||||
|
} finally {
|
||||||
|
if ($kek !== null) {
|
||||||
sodium_memzero($kek);
|
sodium_memzero($kek);
|
||||||
|
}
|
||||||
|
}
|
||||||
if ($master === false) {
|
if ($master === false) {
|
||||||
throw new CryptoException('Wrong passphrase or damaged key');
|
throw new CryptoException('Wrong passphrase or damaged key');
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -157,17 +157,27 @@ final class DbDumper {
|
|||||||
}
|
}
|
||||||
// Tables the backup knows the structure of are created first (DDL is not transactional
|
// Tables the backup knows the structure of are created first (DDL is not transactional
|
||||||
// everywhere); tables it does not know are left out, as asked.
|
// everywhere); tables it does not know are left out, as asked.
|
||||||
|
try {
|
||||||
foreach ($missing as $table) {
|
foreach ($missing as $table) {
|
||||||
if (isset($manifest['tables'][$table]['schema'])) {
|
if (isset($manifest['tables'][$table]['schema'])) {
|
||||||
|
$report['created'][] = $table; // first: creating it can stop halfway (its indexes)
|
||||||
$this->db->createSchemaManager()->createTable(self::tableFromStructure($table, $manifest['tables'][$table]['schema'], $this->tableOptions()));
|
$this->db->createSchemaManager()->createTable(self::tableFromStructure($table, $manifest['tables'][$table]['schema'], $this->tableOptions()));
|
||||||
$report['created'][] = $table;
|
|
||||||
} else {
|
} else {
|
||||||
$report['missing'][] = $table;
|
$report['missing'][] = $table;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$this->dropCreatedTables($report);
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
$total = 0;
|
$total = 0;
|
||||||
$foreignKeys = [];
|
$foreignKeys = [];
|
||||||
|
try {
|
||||||
$this->db->beginTransaction();
|
$this->db->beginTransaction();
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$this->dropCreatedTables($report);
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
try {
|
try {
|
||||||
$foreignKeys = $this->suspendForeignKeys(); // first: on SQLite it also takes the write lock
|
$foreignKeys = $this->suspendForeignKeys(); // first: on SQLite it also takes the write lock
|
||||||
foreach ($manifest['tables'] as $table => $info) {
|
foreach ($manifest['tables'] as $table => $info) {
|
||||||
@@ -220,9 +230,19 @@ final class DbDumper {
|
|||||||
$this->restoreSequences($manifest['sequences'] ?? [], $report['missing'], array_merge($report['restored'], $report['skipped']));
|
$this->restoreSequences($manifest['sequences'] ?? [], $report['missing'], array_merge($report['restored'], $report['skipped']));
|
||||||
$this->resumeForeignKeys($foreignKeys);
|
$this->resumeForeignKeys($foreignKeys);
|
||||||
$this->db->commit();
|
$this->db->commit();
|
||||||
// Foreign keys of the tables created above, now their rows are in. Not inside the
|
} catch (\Throwable $e) {
|
||||||
// transaction: MySQL commits on DDL. A failure is reported, not fatal: the data is
|
$this->db->rollBack();
|
||||||
// restored either way.
|
// PostgreSQL rolls the dropped foreign keys back with the transaction; MySQL's session
|
||||||
|
// switch must be turned on again by hand.
|
||||||
|
if (in_array('SET FOREIGN_KEY_CHECKS = 1', $foreignKeys, true)) {
|
||||||
|
$this->db->executeStatement('SET FOREIGN_KEY_CHECKS = 1');
|
||||||
|
}
|
||||||
|
$this->dropCreatedTables($report);
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
|
// Committed. Foreign keys of the tables created above, now their rows are in. Not inside the
|
||||||
|
// transaction: MySQL commits on DDL. A failure is reported, not fatal: the data is restored
|
||||||
|
// either way, and the created tables stay.
|
||||||
foreach ($report['created'] as $table) {
|
foreach ($report['created'] as $table) {
|
||||||
$structure = $manifest['tables'][$table]['schema'] ?? [];
|
$structure = $manifest['tables'][$table]['schema'] ?? [];
|
||||||
foreach ($structure['foreignKeys'] ?? [] as $fk) {
|
foreach ($structure['foreignKeys'] ?? [] as $fk) {
|
||||||
@@ -235,18 +255,26 @@ final class DbDumper {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch (\Throwable $e) {
|
|
||||||
$this->db->rollBack();
|
|
||||||
// PostgreSQL rolls the dropped foreign keys back with the transaction; MySQL's session
|
|
||||||
// switch must be turned on again by hand.
|
|
||||||
if (in_array('SET FOREIGN_KEY_CHECKS = 1', $foreignKeys, true)) {
|
|
||||||
$this->db->executeStatement('SET FOREIGN_KEY_CHECKS = 1');
|
|
||||||
}
|
|
||||||
throw $e;
|
|
||||||
}
|
|
||||||
return $total;
|
return $total;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The restore did not go through: a rollback does not undo CREATE TABLE everywhere. Left behind,
|
||||||
|
* the empty tables of an app would stop that app's installation later ("table already exists").
|
||||||
|
*
|
||||||
|
* @param array{created: list<string>, ...} $report
|
||||||
|
*/
|
||||||
|
private function dropCreatedTables(array &$report): void {
|
||||||
|
foreach ($report['created'] as $table) {
|
||||||
|
try {
|
||||||
|
$this->db->createSchemaManager()->dropTable($table);
|
||||||
|
} catch (\Throwable) {
|
||||||
|
// the error that brought us here is the one to report; a run again uses the table
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$report['created'] = [];
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The table prefix of the instance the dump was made on (from its appconfig table), or null
|
* The table prefix of the instance the dump was made on (from its appconfig table), or null
|
||||||
* when the dump has none.
|
* when the dump has none.
|
||||||
|
|||||||
@@ -24,6 +24,10 @@ use OCP\AppFramework\Db\Entity;
|
|||||||
* @method void setFiles(?int $v)
|
* @method void setFiles(?int $v)
|
||||||
* @method int|null getBytes()
|
* @method int|null getBytes()
|
||||||
* @method void setBytes(?int $v)
|
* @method void setBytes(?int $v)
|
||||||
|
* @method string|null getNote()
|
||||||
|
* @method void setNote(?string $v)
|
||||||
|
* @method bool|null getPinned()
|
||||||
|
* @method void setPinned(?bool $v)
|
||||||
*/
|
*/
|
||||||
final class Snapshot extends Entity {
|
final class Snapshot extends Entity {
|
||||||
protected int $targetId = 0;
|
protected int $targetId = 0;
|
||||||
@@ -33,8 +37,11 @@ final class Snapshot extends Entity {
|
|||||||
protected int $createdAt = 0;
|
protected int $createdAt = 0;
|
||||||
protected ?int $files = null;
|
protected ?int $files = null;
|
||||||
protected ?int $bytes = null;
|
protected ?int $bytes = null;
|
||||||
|
protected ?string $note = null;
|
||||||
|
protected ?bool $pinned = null;
|
||||||
|
|
||||||
public function __construct() {
|
public function __construct() {
|
||||||
|
$this->addType('pinned', 'boolean');
|
||||||
$this->addType('targetId', 'integer');
|
$this->addType('targetId', 'integer');
|
||||||
$this->addType('createdAt', 'integer');
|
$this->addType('createdAt', 'integer');
|
||||||
$this->addType('files', 'integer');
|
$this->addType('files', 'integer');
|
||||||
|
|||||||
@@ -28,6 +28,12 @@ use OCP\AppFramework\Db\Entity;
|
|||||||
* @method void setMaxBytes(?int $v)
|
* @method void setMaxBytes(?int $v)
|
||||||
* @method int getCreatedAt()
|
* @method int getCreatedAt()
|
||||||
* @method void setCreatedAt(int $v)
|
* @method void setCreatedAt(int $v)
|
||||||
|
* @method string|null getSchedule()
|
||||||
|
* @method void setSchedule(?string $v)
|
||||||
|
* @method string|null getScheduleZone()
|
||||||
|
* @method void setScheduleZone(?string $v)
|
||||||
|
* @method int|null getScheduleSince()
|
||||||
|
* @method void setScheduleSince(?int $v)
|
||||||
*/
|
*/
|
||||||
final class Target extends Entity {
|
final class Target extends Entity {
|
||||||
protected string $name = '';
|
protected string $name = '';
|
||||||
@@ -39,8 +45,12 @@ final class Target extends Entity {
|
|||||||
protected int $createdAt = 0;
|
protected int $createdAt = 0;
|
||||||
protected ?bool $appendOnly = null;
|
protected ?bool $appendOnly = null;
|
||||||
protected ?int $maxBytes = null;
|
protected ?int $maxBytes = null;
|
||||||
|
protected ?string $schedule = null;
|
||||||
|
protected ?string $scheduleZone = null;
|
||||||
|
protected ?int $scheduleSince = null;
|
||||||
|
|
||||||
public function __construct() {
|
public function __construct() {
|
||||||
|
$this->addType('scheduleSince', 'integer');
|
||||||
$this->addType('createdAt', 'integer');
|
$this->addType('createdAt', 'integer');
|
||||||
$this->addType('appendOnly', 'boolean');
|
$this->addType('appendOnly', 'boolean');
|
||||||
$this->addType('maxBytes', 'integer');
|
$this->addType('maxBytes', 'integer');
|
||||||
|
|||||||
@@ -52,12 +52,51 @@ final class BackupRun {
|
|||||||
$stats = $state['stats'];
|
$stats = $state['stats'];
|
||||||
$walker = new TreeWalker($state['roots'], $state['exclude'], function (string $logical) use (&$state, &$stats): void {
|
$walker = new TreeWalker($state['roots'], $state['exclude'], function (string $logical) use (&$state, &$stats): void {
|
||||||
self::recordUnreadable($state, $stats, $logical);
|
self::recordUnreadable($state, $stats, $logical);
|
||||||
});
|
}, true);
|
||||||
$parent = new ParentCursor($repo, $state['parent'], $state['parentPos']);
|
$parent = new ParentCursor($repo, $state['parent'], $state['parentPos']);
|
||||||
$lines = [];
|
$lines = [];
|
||||||
|
$emptyDirs = [];
|
||||||
|
$folders = [];
|
||||||
$finishedWalk = true;
|
$finishedWalk = true;
|
||||||
|
|
||||||
foreach ($walker->walk($state['after']) as $logical => $path) {
|
foreach ($walker->walk($state['after']) as $logical => $path) {
|
||||||
|
$safe = mb_check_encoding($logical, 'UTF-8') && Repository::isSafePath($logical);
|
||||||
|
if ($walker->atFolderWithContent()) {
|
||||||
|
// Only its time: what is in it is reported by itself, and a bad name in the path of
|
||||||
|
// each of those is what gets reported. The cursor stays for such a name, see below.
|
||||||
|
$mtime = $safe ? @filemtime($path) : false;
|
||||||
|
if ($safe && $mtime !== false) {
|
||||||
|
$folders[] = json_encode([$logical, $mtime], JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
|
||||||
|
$state['after'] = $logical;
|
||||||
|
if (microtime(true) >= $deadline) {
|
||||||
|
$finishedWalk = false;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (!$safe) {
|
||||||
|
// A name a snapshot cannot hold (a backslash, a control character, not UTF-8). One
|
||||||
|
// such path must not make a backup fail, or the whole snapshot unreadable later.
|
||||||
|
// The cursor stays where it is: the state is JSON, which cannot hold such a name. A
|
||||||
|
// next step meets it again, and it is reported once.
|
||||||
|
self::recordBadName($state, $stats, $logical);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (is_dir($path)) {
|
||||||
|
$mtime = @filemtime($path);
|
||||||
|
if ($mtime === false) {
|
||||||
|
continue; // removed while walking
|
||||||
|
}
|
||||||
|
$emptyDirs[] = json_encode([$logical, $mtime], JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
|
||||||
|
$stats['emptyDirs'] = ($stats['emptyDirs'] ?? 0) + 1;
|
||||||
|
$state['after'] = $logical;
|
||||||
|
if (microtime(true) >= $deadline) {
|
||||||
|
$finishedWalk = false;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
$version = Repository::fileVersion($path);
|
$version = Repository::fileVersion($path);
|
||||||
if ($version === 'missing') {
|
if ($version === 'missing') {
|
||||||
continue; // deleted while walking
|
continue; // deleted while walking
|
||||||
@@ -81,7 +120,7 @@ final class BackupRun {
|
|||||||
if ($cur === null || $cur['path'] !== $logical) {
|
if ($cur === null || $cur['path'] !== $logical) {
|
||||||
$cur = ['path' => $logical, 'offset' => 0, 'blobs' => [], 'version' => $version];
|
$cur = ['path' => $logical, 'offset' => 0, 'blobs' => [], 'version' => $version];
|
||||||
}
|
}
|
||||||
$fh = @fopen($path, 'rb');
|
$fh = Repository::openForReading($path);
|
||||||
if ($fh === false) {
|
if ($fh === false) {
|
||||||
self::recordUnreadable($state, $stats, $logical);
|
self::recordUnreadable($state, $stats, $logical);
|
||||||
$state['cur'] = null;
|
$state['cur'] = null;
|
||||||
@@ -136,6 +175,16 @@ final class BackupRun {
|
|||||||
$writer->write(implode("\n", $lines) . "\n");
|
$writer->write(implode("\n", $lines) . "\n");
|
||||||
$state['segments'][] = $writer->finish();
|
$state['segments'][] = $writer->finish();
|
||||||
}
|
}
|
||||||
|
if ($emptyDirs !== []) {
|
||||||
|
$writer = $repo->blobWriter($stats);
|
||||||
|
$writer->write(implode("\n", $emptyDirs) . "\n");
|
||||||
|
$state['dirSegments'][] = $writer->finish();
|
||||||
|
}
|
||||||
|
if ($folders !== []) {
|
||||||
|
$writer = $repo->blobWriter($stats);
|
||||||
|
$writer->write(implode("\n", $folders) . "\n");
|
||||||
|
$state['folderSegments'][] = $writer->finish();
|
||||||
|
}
|
||||||
$repo->flushPacks();
|
$repo->flushPacks();
|
||||||
$state['parentPos'] = $parent->position();
|
$state['parentPos'] = $parent->position();
|
||||||
$state['stats'] = $stats;
|
$state['stats'] = $stats;
|
||||||
@@ -150,10 +199,22 @@ final class BackupRun {
|
|||||||
if ($state['phase'] === 'finish') {
|
if ($state['phase'] === 'finish') {
|
||||||
// Every segment ends with a newline, so they read as one stream: fetched together,
|
// Every segment ends with a newline, so they read as one stream: fetched together,
|
||||||
// several at a time where the location allows it.
|
// several at a time where the location allows it.
|
||||||
|
// Empty folders go in the meta record, not between the files: a version that does not
|
||||||
|
// know them reads the snapshot as before, and only restores no empty folders.
|
||||||
|
$emptyDirs = [];
|
||||||
|
foreach (isset($state['dirSegments']) ? $repo->readLines(array_values(array_merge(...$state['dirSegments']))) : [] as $line) {
|
||||||
|
if ($line !== '') {
|
||||||
|
$emptyDirs[] = json_decode($line, true, 8, JSON_THROW_ON_ERROR);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The times of the other folders too, packed: every look at a snapshot's meta record reads
|
||||||
|
// this, and a string of a few hundred KiB costs that far less than tens of thousands of arrays.
|
||||||
|
$folders = isset($state['folderSegments']) ? implode("\n", iterator_to_array($repo->readLines(array_values(array_merge(...$state['folderSegments']))), false)) : '';
|
||||||
$repo->putSnapshot($state['snapshot'],
|
$repo->putSnapshot($state['snapshot'],
|
||||||
['time' => $state['started'], 'parent' => $state['parent'], 'label' => $state['label'],
|
['time' => $state['started'], 'parent' => $state['parent'], 'label' => $state['label'],
|
||||||
'roots' => array_keys($state['roots']), 'stats' => $state['stats'],
|
'roots' => array_keys($state['roots']), 'stats' => $state['stats'],
|
||||||
'warnings' => self::warnings($state)] + $state['meta'],
|
'warnings' => self::warnings($state)] + ($emptyDirs === [] ? [] : ['emptyDirs' => $emptyDirs])
|
||||||
|
+ ($folders === '' ? [] : ['folderTimes' => base64_encode((string)gzdeflate($folders))]) + $state['meta'],
|
||||||
$repo->readLines(array_values(array_merge(...$state['segments']))));
|
$repo->readLines(array_values(array_merge(...$state['segments']))));
|
||||||
$state['phase'] = 'commit';
|
$state['phase'] = 'commit';
|
||||||
}
|
}
|
||||||
@@ -214,10 +275,26 @@ final class BackupRun {
|
|||||||
$stats['unreadable'] = ($stats['unreadable'] ?? 0) + 1;
|
$stats['unreadable'] = ($stats['unreadable'] ?? 0) + 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** @return array{unreadable: list<string>, changing: list<string>} examples for the report (first 50 each) */
|
/**
|
||||||
|
* A file or directory left out because of its name. Remembered like an unreadable one, in a form
|
||||||
|
* that fits the JSON state and is safe to show: control characters and invalid bytes replaced.
|
||||||
|
*/
|
||||||
|
private static function recordBadName(array &$state, array &$stats, string $logical): void {
|
||||||
|
$shown = (string)preg_replace('/[\x00-\x1f\x7f]/', '?', mb_scrub($logical, 'UTF-8'));
|
||||||
|
if (isset($state['badNames'][$shown])) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (count($state['badNames'] ?? []) < 1000) {
|
||||||
|
$state['badNames'][$shown] = true;
|
||||||
|
}
|
||||||
|
$stats['badNames'] = ($stats['badNames'] ?? 0) + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array{unreadable: list<string>, badNames: list<string>, changing: list<string>} examples for the report (first 50 each) */
|
||||||
public static function warnings(array $state): array {
|
public static function warnings(array $state): array {
|
||||||
return [
|
return [
|
||||||
'unreadable' => array_slice(array_map('strval', array_keys($state['unreadable'] ?? [])), 0, 50),
|
'unreadable' => array_slice(array_map('strval', array_keys($state['unreadable'] ?? [])), 0, 50),
|
||||||
|
'badNames' => array_slice(array_map('strval', array_keys($state['badNames'] ?? [])), 0, 50),
|
||||||
'changing' => array_slice(array_map('strval', array_values($state['changing'] ?? [])), 0, 50),
|
'changing' => array_slice(array_map('strval', array_values($state['changing'] ?? [])), 0, 50),
|
||||||
];
|
];
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -9,12 +9,16 @@ namespace OCA\NgBackup\Job;
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Walks one or more source roots in a stable depth-first order (entries of a directory sorted
|
* Walks one or more source roots in a stable depth-first order (entries of a directory sorted
|
||||||
* by name, children right after their directory), yielding "<root>/<relative path>" for files.
|
* by name, children right after their directory), yielding "<root>/<relative path>" for files
|
||||||
|
* and for directories with nothing in them to walk (the caller tells them apart with is_dir()).
|
||||||
|
* A root itself without a name is never yielded.
|
||||||
*
|
*
|
||||||
* The same order is used for comparing paths (compare()), so a walk can resume after a cursor
|
* The same order is used for comparing paths (compare()), so a walk can resume after a cursor
|
||||||
* and be merge-joined with the previous snapshot without holding any list in memory.
|
* and be merge-joined with the previous snapshot without holding any list in memory.
|
||||||
*/
|
*/
|
||||||
final class TreeWalker {
|
final class TreeWalker {
|
||||||
|
private bool $atFolder = false;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @param array<string, string> $roots name => absolute directory, walked in this order
|
* @param array<string, string> $roots name => absolute directory, walked in this order
|
||||||
* @param list<string> $exclude path prefixes ("<root>/<relative>") that are skipped entirely;
|
* @param list<string> $exclude path prefixes ("<root>/<relative>") that are skipped entirely;
|
||||||
@@ -25,9 +29,16 @@ final class TreeWalker {
|
|||||||
private array $exclude = [],
|
private array $exclude = [],
|
||||||
/** Called with the logical path of a directory that cannot be listed. */
|
/** Called with the logical path of a directory that cannot be listed. */
|
||||||
private ?\Closure $onUnreadable = null,
|
private ?\Closure $onUnreadable = null,
|
||||||
|
/** Also yield directories with something in them, right before the first thing in them. */
|
||||||
|
private bool $withFolders = false,
|
||||||
) {
|
) {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Whether what was just yielded is a directory with something in it (see $withFolders). */
|
||||||
|
public function atFolderWithContent(): bool {
|
||||||
|
return $this->atFolder;
|
||||||
|
}
|
||||||
|
|
||||||
/** @return \Generator<string, string> logical path => absolute path, strictly after $after */
|
/** @return \Generator<string, string> logical path => absolute path, strictly after $after */
|
||||||
public function walk(?string $after = null): \Generator {
|
public function walk(?string $after = null): \Generator {
|
||||||
foreach ($this->roots as $name => $dir) {
|
foreach ($this->roots as $name => $dir) {
|
||||||
@@ -66,27 +77,41 @@ final class TreeWalker {
|
|||||||
}
|
}
|
||||||
$names = array_values(array_diff($names, ['.', '..']));
|
$names = array_values(array_diff($names, ['.', '..']));
|
||||||
sort($names, SORT_STRING);
|
sort($names, SORT_STRING);
|
||||||
|
$empty = true; // no file or directory below it that is walked, whatever the cursor skips
|
||||||
foreach ($names as $name) {
|
foreach ($names as $name) {
|
||||||
$childLogical = $logical === '' ? $name : $logical . '/' . $name;
|
$childLogical = $logical === '' ? $name : $logical . '/' . $name;
|
||||||
$childAbs = $abs . '/' . $name;
|
$childAbs = $abs . '/' . $name;
|
||||||
if (is_link($childAbs)) {
|
if (is_link($childAbs)) {
|
||||||
continue; // never follow symlinks out of the data directory
|
continue; // never follow symlinks out of the data directory
|
||||||
}
|
}
|
||||||
if (is_dir($childAbs)) {
|
$childIsDir = is_dir($childAbs);
|
||||||
|
if (($childIsDir || is_file($childAbs)) && $empty && !$this->isExcluded($childLogical, $childIsDir)
|
||||||
|
&& $this->withFolders && $logical !== '' && ($after === null || self::compare($logical, $after) > 0)) {
|
||||||
|
$this->atFolder = true;
|
||||||
|
yield $logical => $abs;
|
||||||
|
$this->atFolder = false;
|
||||||
|
}
|
||||||
|
if ($childIsDir) {
|
||||||
|
$empty = $empty && $this->isExcluded($childLogical, true);
|
||||||
// Skip a directory entirely if everything in it sorts before the cursor.
|
// Skip a directory entirely if everything in it sorts before the cursor.
|
||||||
if ($after !== null && self::compare($childLogical, $after) < 0 && !self::isPrefix($childLogical, $after)) {
|
if ($after !== null && self::compare($childLogical, $after) < 0 && !self::isPrefix($childLogical, $after)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
yield from $this->walkDir($childAbs, $childLogical, $after);
|
yield from $this->walkDir($childAbs, $childLogical, $after);
|
||||||
} elseif (is_file($childAbs)) {
|
} elseif (is_file($childAbs)) {
|
||||||
|
$excluded = $this->isExcluded($childLogical, false);
|
||||||
|
$empty = $empty && $excluded;
|
||||||
if ($after !== null && self::compare($childLogical, $after) <= 0) {
|
if ($after !== null && self::compare($childLogical, $after) <= 0) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if (!$this->isExcluded($childLogical, false)) {
|
if (!$excluded) {
|
||||||
yield $childLogical => $childAbs;
|
yield $childLogical => $childAbs;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if ($empty && $logical !== '' && ($after === null || self::compare($logical, $after) > 0)) {
|
||||||
|
yield $logical => $abs;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private function isExcluded(string $logical, bool $isDir): bool {
|
private function isExcluded(string $logical, bool $isDir): bool {
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Migration;
|
||||||
|
|
||||||
|
use Closure;
|
||||||
|
use OCP\DB\ISchemaWrapper;
|
||||||
|
use OCP\DB\Types;
|
||||||
|
use OCP\Migration\IOutput;
|
||||||
|
use OCP\Migration\SimpleMigrationStep;
|
||||||
|
|
||||||
|
/** Local copy of what the catalog on the location says about a snapshot: its note and pin. */
|
||||||
|
final class Version000007Date20261107000000 extends SimpleMigrationStep {
|
||||||
|
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
|
||||||
|
/** @var ISchemaWrapper $schema */
|
||||||
|
$schema = $schemaClosure();
|
||||||
|
$t = $schema->getTable('ngb_snapshots');
|
||||||
|
$changed = false;
|
||||||
|
if (!$t->hasColumn('note')) {
|
||||||
|
$t->addColumn('note', Types::STRING, ['notnull' => false, 'length' => 1000]);
|
||||||
|
$changed = true;
|
||||||
|
}
|
||||||
|
if (!$t->hasColumn('pinned')) {
|
||||||
|
$t->addColumn('pinned', Types::BOOLEAN, ['notnull' => false, 'default' => false]);
|
||||||
|
$changed = true;
|
||||||
|
}
|
||||||
|
return $changed ? $schema : null;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Migration;
|
||||||
|
|
||||||
|
use Closure;
|
||||||
|
use OCA\NgBackup\Service\LocationSchedule;
|
||||||
|
use OCA\NgBackup\Service\Schedule;
|
||||||
|
use OCP\DB\ISchemaWrapper;
|
||||||
|
use OCP\DB\Types;
|
||||||
|
use OCP\IDBConnection;
|
||||||
|
use OCP\Migration\IOutput;
|
||||||
|
use OCP\Migration\SimpleMigrationStep;
|
||||||
|
|
||||||
|
/** A schedule per location. Each existing location gets the daily time that held for all of them. */
|
||||||
|
final class Version000008Date20261107010000 extends SimpleMigrationStep {
|
||||||
|
public function __construct(
|
||||||
|
private IDBConnection $db,
|
||||||
|
private Schedule $schedule,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
|
||||||
|
/** @var ISchemaWrapper $schema */
|
||||||
|
$schema = $schemaClosure();
|
||||||
|
$t = $schema->getTable('ngb_targets');
|
||||||
|
if ($t->hasColumn('schedule')) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
$t->addColumn('schedule', Types::STRING, ['notnull' => false, 'length' => 16]);
|
||||||
|
$t->addColumn('schedule_zone', Types::STRING, ['notnull' => false, 'length' => 64]);
|
||||||
|
$t->addColumn('schedule_since', Types::BIGINT, ['notnull' => false, 'unsigned' => true]);
|
||||||
|
return $schema;
|
||||||
|
}
|
||||||
|
|
||||||
|
public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options): void {
|
||||||
|
['time' => $time, 'timezone' => $zone] = $this->schedule->get();
|
||||||
|
$qb = $this->db->getQueryBuilder();
|
||||||
|
$qb->update('ngb_targets')
|
||||||
|
->set('schedule', $qb->createNamedParameter(Schedule::validTime($time) ? LocationSchedule::normalize($time) : ''))
|
||||||
|
->set('schedule_zone', $qb->createNamedParameter($zone))
|
||||||
|
->where($qb->expr()->isNull('schedule'));
|
||||||
|
$qb->executeStatement();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Migration;
|
||||||
|
|
||||||
|
use Closure;
|
||||||
|
use OCA\NgBackup\Service\KeyService;
|
||||||
|
use OCP\Migration\IOutput;
|
||||||
|
use OCP\Migration\SimpleMigrationStep;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A key from before 1.1.0 counts as used: those versions did not keep track, and it may protect
|
||||||
|
* backups on a location that was removed from this server later (user exports included). Only a
|
||||||
|
* key made with 1.1.0 or newer can be replaced as an unused one.
|
||||||
|
*/
|
||||||
|
final class Version000009Date20261108000000 extends SimpleMigrationStep {
|
||||||
|
public function __construct(
|
||||||
|
private KeyService $keys,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options): void {
|
||||||
|
if ($this->keys->isInitialized()) {
|
||||||
|
$this->keys->markUsed();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -46,6 +46,10 @@ final class Notifier implements INotifier {
|
|||||||
$notification->setParsedSubject($l->t('Backup to %1$s finished with warnings: %2$s unreadable, %3$s changed while being read', [$p['target'], $p['unreadable'], $p['changing']]));
|
$notification->setParsedSubject($l->t('Backup to %1$s finished with warnings: %2$s unreadable, %3$s changed while being read', [$p['target'], $p['unreadable'], $p['changing']]));
|
||||||
$notification->setParsedMessage($l->t('Unreadable files and folders are missing from this backup (check their permissions). Files that kept changing were saved as they were while being read. Examples: %s', [$p['examples'] ?? '']));
|
$notification->setParsedMessage($l->t('Unreadable files and folders are missing from this backup (check their permissions). Files that kept changing were saved as they were while being read. Examples: %s', [$p['examples'] ?? '']));
|
||||||
break;
|
break;
|
||||||
|
case 'run_bad_names':
|
||||||
|
$notification->setParsedSubject($l->n('Backup to %s finished, but %n item is missing because of its name', 'Backup to %s finished, but %n items are missing because of their names', (int)$p['count'], [$p['target']]));
|
||||||
|
$notification->setParsedMessage($l->t('Names with a backslash or a control character, or that are not valid UTF-8, are not supported, so these files and folders are not in this backup. Rename them; the next backup includes them. Examples: %s', [$p['examples'] ?? '']));
|
||||||
|
break;
|
||||||
case 'quota_warning':
|
case 'quota_warning':
|
||||||
$notification->setParsedSubject($l->t('Backup location %1$s is at %2$s%% of its limit (%3$s of %4$s)', [$p['target'], $p['percent'], $p['used'], $p['limit']]));
|
$notification->setParsedSubject($l->t('Backup location %1$s is at %2$s%% of its limit (%3$s of %4$s)', [$p['target'], $p['percent'], $p['used'], $p['limit']]));
|
||||||
$notification->setParsedMessage($l->t('Backups stop when the limit is reached. Raise the limit, or let the retention policy remove older backups.'));
|
$notification->setParsedMessage($l->t('Backups stop when the limit is reached. Raise the limit, or let the retention policy remove older backups.'));
|
||||||
@@ -57,12 +61,20 @@ final class Notifier implements INotifier {
|
|||||||
: $l->t('No successful backup to %1$s since %2$s', [$p['target'], $p['since']]));
|
: $l->t('No successful backup to %1$s since %2$s', [$p['target'], $p['since']]));
|
||||||
$notification->setParsedMessage($l->t('Check the backup settings and the server log.'));
|
$notification->setParsedMessage($l->t('Check the backup settings and the server log.'));
|
||||||
break;
|
break;
|
||||||
|
case 'schedule_off':
|
||||||
|
$notification->setParsedSubject($l->t('Backup location %s has no schedule', [$p['target']]));
|
||||||
|
$notification->setParsedMessage(($p['others'] ?? '') === '1'
|
||||||
|
? $l->t('No backups run to this location until it gets a schedule or someone starts one. If that is what you want, you can ignore this reminder.')
|
||||||
|
: $l->t('No location has a schedule, so no backups run at all unless someone starts one. Set a schedule under Backups.'));
|
||||||
|
break;
|
||||||
case 'security_event':
|
case 'security_event':
|
||||||
$who = (string)($p['actor'] ?? '');
|
$who = (string)($p['actor'] ?? '');
|
||||||
$notification->setParsedSubject(match ($p['event'] ?? '') {
|
$notification->setParsedSubject(match ($p['event'] ?? '') {
|
||||||
'kit_downloaded' => $l->t('%s downloaded the backup recovery kit', [$who]),
|
'kit_downloaded' => $l->t('%s downloaded the backup recovery kit', [$who]),
|
||||||
|
'key_replaced' => $l->t('%1$s replaced the unused backup key %2$s by the key of existing backups, %3$s; download and confirm the new recovery kit', [$who, $p['old'] ?? '', $p['new'] ?? '']),
|
||||||
'target_removed' => $l->t('%1$s removed backup location %2$s', [$who, $p['target'] ?? '']),
|
'target_removed' => $l->t('%1$s removed backup location %2$s', [$who, $p['target'] ?? '']),
|
||||||
'target_changed' => $l->t('%1$s changed the connection settings of backup location %2$s', [$who, $p['target'] ?? '']),
|
'target_changed' => $l->t('%1$s changed the connection settings of backup location %2$s', [$who, $p['target'] ?? '']),
|
||||||
|
'schedule_off' => $l->t('%1$s turned off the schedule of backup location %2$s', [$who, $p['target'] ?? '']),
|
||||||
'append_only_off' =>$l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
|
'append_only_off' =>$l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
|
||||||
'retention_changed' => $l->t('%1$s changed the backup retention to %2$s', [$who, $p['policy'] ?? '']),
|
'retention_changed' => $l->t('%1$s changed the backup retention to %2$s', [$who, $p['policy'] ?? '']),
|
||||||
'snapshots_forgotten' => $l->t('%1$s moved %2$s snapshots of %3$s to the trash; they are deleted permanently after %4$s unless restored', [$who, $p['count'] ?? '', $p['target'] ?? '', $p['until'] ?? '']),
|
'snapshots_forgotten' => $l->t('%1$s moved %2$s snapshots of %3$s to the trash; they are deleted permanently after %4$s unless restored', [$who, $p['count'] ?? '', $p['target'] ?? '', $p['until'] ?? '']),
|
||||||
|
|||||||
@@ -30,6 +30,10 @@ final class Catalog {
|
|||||||
public array $trash = [];
|
public array $trash = [];
|
||||||
/** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */
|
/** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */
|
||||||
public array $userExports = [];
|
public array $userExports = [];
|
||||||
|
/** @var array<string, true> snapshots the retention policy leaves alone */
|
||||||
|
public array $pinned = [];
|
||||||
|
/** @var array<string, string> snapshot id => note, also while the snapshot is in the trash */
|
||||||
|
public array $notes = [];
|
||||||
public string $hash = '';
|
public string $hash = '';
|
||||||
|
|
||||||
public function __construct(
|
public function __construct(
|
||||||
@@ -82,6 +86,8 @@ final class Catalog {
|
|||||||
// Catalogs written before user exports existed have no such key: default to empty rather
|
// Catalogs written before user exports existed have no such key: default to empty rather
|
||||||
// than adopting the location's current users/ listing (that would skip verification).
|
// than adopting the location's current users/ listing (that would skip verification).
|
||||||
$this->userExports = array_fill_keys($data['userExports'] ?? [], true);
|
$this->userExports = array_fill_keys($data['userExports'] ?? [], true);
|
||||||
|
$this->pinned = array_fill_keys($data['pinned'] ?? [], true);
|
||||||
|
$this->notes = $data['notes'] ?? [];
|
||||||
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
|
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -93,9 +99,12 @@ final class Catalog {
|
|||||||
if ($this->backend->exists($this->path($next))) {
|
if ($this->backend->exists($this->path($next))) {
|
||||||
throw new RepositoryException("Catalog generation $next was written by another process at the same time; try again");
|
throw new RepositoryException("Catalog generation $next was written by another process at the same time; try again");
|
||||||
}
|
}
|
||||||
|
// A pin ends when its snapshot leaves the list, a note when the snapshot is gone for good.
|
||||||
|
$this->pinned = array_intersect_key($this->pinned, $this->snapshots);
|
||||||
|
$this->notes = array_intersect_key($this->notes, $this->snapshots + $this->trash);
|
||||||
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
|
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
|
||||||
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash),
|
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash),
|
||||||
'userExports' => array_keys($this->userExports)], JSON_THROW_ON_ERROR);
|
'userExports' => array_keys($this->userExports), 'pinned' => array_keys($this->pinned), 'notes' => $this->notes], JSON_THROW_ON_ERROR);
|
||||||
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
|
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
|
||||||
$stream = fopen('php://memory', 'w+b');
|
$stream = fopen('php://memory', 'w+b');
|
||||||
fwrite($stream, $encrypted);
|
fwrite($stream, $encrypted);
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ use OCA\NgBackup\Crypto\StreamCipher;
|
|||||||
*
|
*
|
||||||
* Layout on the backend:
|
* Layout on the backend:
|
||||||
* config plaintext JSON: format version, repository id, wrapped master key
|
* config plaintext JSON: format version, repository id, wrapped master key
|
||||||
|
* settings encrypted JSON: settings fixed at installation (deletion delay)
|
||||||
* packs/xx/<packId> encrypted blobs + encrypted pack header
|
* packs/xx/<packId> encrypted blobs + encrypted pack header
|
||||||
* index/<packId> encrypted list of the blobs in that pack
|
* index/<packId> encrypted list of the blobs in that pack
|
||||||
* snapshots/<snapId> encrypted JSON lines: one meta line, then one line per file
|
* snapshots/<snapId> encrypted JSON lines: one meta line, then one line per file
|
||||||
@@ -28,6 +29,7 @@ use OCA\NgBackup\Crypto\StreamCipher;
|
|||||||
final class Repository {
|
final class Repository {
|
||||||
public const FORMAT = 1;
|
public const FORMAT = 1;
|
||||||
public const BLOB_SIZE = 4 * 1048576;
|
public const BLOB_SIZE = 4 * 1048576;
|
||||||
|
private const SETTINGS = 'settings';
|
||||||
private const FLAG_DEFLATE = 1;
|
private const FLAG_DEFLATE = 1;
|
||||||
/** Blobs from this size on get a quick compression test before the full compression. */
|
/** Blobs from this size on get a quick compression test before the full compression. */
|
||||||
private const DEFLATE_PROBE_FROM = 256 * 1024;
|
private const DEFLATE_PROBE_FROM = 256 * 1024;
|
||||||
@@ -44,6 +46,8 @@ final class Repository {
|
|||||||
private ?int $maxBytes = null;
|
private ?int $maxBytes = null;
|
||||||
private Catalog $catalog;
|
private Catalog $catalog;
|
||||||
private PackReader $packReader;
|
private PackReader $packReader;
|
||||||
|
/** @var array<string, array<string, mixed>> meta record of the snapshot read last */
|
||||||
|
private array $lastMeta = [];
|
||||||
/** verify() lists the pack folders from this many packs on (see presentPacks()); tests lower it. */
|
/** verify() lists the pack folders from this many packs on (see presentPacks()); tests lower it. */
|
||||||
public int $listPacksAbove = 256;
|
public int $listPacksAbove = 256;
|
||||||
|
|
||||||
@@ -77,7 +81,7 @@ final class Repository {
|
|||||||
// opens this data; refuse instead.
|
// opens this data; refuse instead.
|
||||||
foreach (['index', 'snapshots', 'trash/snapshots', 'catalog'] as $dir) {
|
foreach (['index', 'snapshots', 'trash/snapshots', 'catalog'] as $dir) {
|
||||||
if ($backend->listFiles($dir) !== []) {
|
if ($backend->listFiles($dir) !== []) {
|
||||||
throw new RepositoryException("This location holds backup data ($dir/) but no repository config could be read: not creating a new repository over it. Check the connection and the folder; a missing config can be restored from the recovery kit.");
|
throw new RepositoryException("This location holds backup data ($dir/) but no repository config could be read: not creating a new repository over it. Check the connection and the folder. If the config is really gone, occ backup:target:add with --repair-config writes it again with this server's key (on a new server: import the recovery kit first, occ backup:key:import).");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
$id = bin2hex(random_bytes(16));
|
$id = bin2hex(random_bytes(16));
|
||||||
@@ -93,12 +97,7 @@ final class Repository {
|
|||||||
/** @param ?callable(int, int): void $progress index files downloaded so far, and in all (see BlobIndex::load()) */
|
/** @param ?callable(int, int): void $progress index files downloaded so far, and in all (see BlobIndex::load()) */
|
||||||
public static function openWithKey(IBackend $backend, KeyRing $keys, ?IIndexCache $cache = null, ?ICatalogAnchor $anchor = null, ?callable $progress = null): self {
|
public static function openWithKey(IBackend $backend, KeyRing $keys, ?IIndexCache $cache = null, ?ICatalogAnchor $anchor = null, ?callable $progress = null): self {
|
||||||
$config = self::readConfig($backend);
|
$config = self::readConfig($backend);
|
||||||
try {
|
if (!self::configMatches($config, $keys)) {
|
||||||
$ok = isset($config['check']) && (new StreamCipher($keys))->decryptString((string)base64_decode($config['check'], true), 'config-check') === $config['id'];
|
|
||||||
} catch (\OCA\NgBackup\Crypto\CryptoException) {
|
|
||||||
$ok = false;
|
|
||||||
}
|
|
||||||
if (!$ok) {
|
|
||||||
throw new RepositoryException('This repository was created with a different key');
|
throw new RepositoryException('This repository was created with a different key');
|
||||||
}
|
}
|
||||||
$repo = new self($backend, $keys, $config['id']);
|
$repo = new self($backend, $keys, $config['id']);
|
||||||
@@ -132,6 +131,49 @@ final class Repository {
|
|||||||
return $this->catalog->gen;
|
return $this->catalog->gen;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** @return list<string> snapshots the retention policy leaves alone */
|
||||||
|
public function pinnedSnapshots(): array {
|
||||||
|
return array_map('strval', array_keys($this->catalog->pinned));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array<string, string> snapshot id => note */
|
||||||
|
public function snapshotNotes(): array {
|
||||||
|
return $this->catalog->notes;
|
||||||
|
}
|
||||||
|
|
||||||
|
public function setPinned(string $snapshotId, bool $pinned): void {
|
||||||
|
$this->requireSnapshot($snapshotId);
|
||||||
|
if (isset($this->catalog->pinned[$snapshotId]) === $pinned) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if ($pinned) {
|
||||||
|
$this->catalog->pinned[$snapshotId] = true;
|
||||||
|
} else {
|
||||||
|
unset($this->catalog->pinned[$snapshotId]);
|
||||||
|
}
|
||||||
|
$this->catalog->write();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A note kept in the catalog: unlike the label inside the snapshot, it can change afterwards. */
|
||||||
|
public function setSnapshotNote(string $snapshotId, string $note): void {
|
||||||
|
$this->requireSnapshot($snapshotId);
|
||||||
|
if (($this->catalog->notes[$snapshotId] ?? '') === $note) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if ($note === '') {
|
||||||
|
unset($this->catalog->notes[$snapshotId]);
|
||||||
|
} else {
|
||||||
|
$this->catalog->notes[$snapshotId] = $note;
|
||||||
|
}
|
||||||
|
$this->catalog->write();
|
||||||
|
}
|
||||||
|
|
||||||
|
private function requireSnapshot(string $snapshotId): void {
|
||||||
|
if (!isset($this->catalog->snapshots[$snapshotId])) {
|
||||||
|
throw new \InvalidArgumentException('There is no snapshot ' . $snapshotId . ' on this location');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Index files downloaded when this repository was opened (0 when the local cache was complete). */
|
/** Index files downloaded when this repository was opened (0 when the local cache was complete). */
|
||||||
public function indexDownloads(): int {
|
public function indexDownloads(): int {
|
||||||
return $this->index->downloaded;
|
return $this->index->downloaded;
|
||||||
@@ -151,6 +193,114 @@ final class Repository {
|
|||||||
return $config;
|
return $config;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Open a local file to read it in whole blobs. PHP reads a file in pieces of 8 KiB by default;
|
||||||
|
* on a disk where each request waits (a network or cloud volume) that made reading five times
|
||||||
|
* slower: 5 MiB/s instead of 27 MiB/s on nc01. Without PHP's read buffer every fread() of a
|
||||||
|
* blob is one request to the disk.
|
||||||
|
*
|
||||||
|
* @return resource|false
|
||||||
|
*/
|
||||||
|
public static function openForReading(string $path) {
|
||||||
|
$fh = @fopen($path, 'rb');
|
||||||
|
if ($fh !== false) {
|
||||||
|
stream_set_read_buffer($fh, 0);
|
||||||
|
stream_set_chunk_size($fh, self::BLOB_SIZE);
|
||||||
|
}
|
||||||
|
return $fh;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function configMatches(array $config, KeyRing $keys): bool {
|
||||||
|
try {
|
||||||
|
return isset($config['check'], $config['id'])
|
||||||
|
&& (new StreamCipher($keys))->decryptString((string)base64_decode((string)$config['check'], true), 'config-check') === $config['id'];
|
||||||
|
} catch (\OCA\NgBackup\Crypto\CryptoException) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The passphrase-wrapped key slots in the config at this location: disaster recovery without
|
||||||
|
* the recovery kit, where any one of their passphrases gives back the master key.
|
||||||
|
*/
|
||||||
|
public static function wrappedKeyAt(IBackend $backend): array {
|
||||||
|
$config = self::readConfig($backend);
|
||||||
|
if (!isset($config['key']) || !is_array($config['key'])) {
|
||||||
|
throw new RepositoryException('The repository config at this location holds no key');
|
||||||
|
}
|
||||||
|
return $config['key'];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether the repository at this location was created with this key. Does not read the index. */
|
||||||
|
public static function madeWithKey(IBackend $backend, KeyRing $keys): bool {
|
||||||
|
return self::configMatches(self::readConfig($backend), $keys);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Settings of the installation that never change (the deletion delay), sealed with the
|
||||||
|
* master key so that, unlike the config and the recovery kit, nobody can alter them unseen.
|
||||||
|
* Empty when the location has none yet (made before 1.1.0, until its next backup); only
|
||||||
|
* ['unreadable' => true] when they are there but cannot be read.
|
||||||
|
*
|
||||||
|
* @return array<string, mixed>
|
||||||
|
*/
|
||||||
|
public static function settingsAt(IBackend $backend, KeyRing $keys): array {
|
||||||
|
try {
|
||||||
|
if (!$backend->exists(self::SETTINGS)) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
$fh = $backend->get(self::SETTINGS);
|
||||||
|
$raw = (string)stream_get_contents($fh);
|
||||||
|
fclose($fh);
|
||||||
|
$settings = json_decode((new StreamCipher($keys))->decryptString($raw, 'settings'), true, 16, JSON_THROW_ON_ERROR);
|
||||||
|
return is_array($settings) ? $settings : ['unreadable' => true];
|
||||||
|
} catch (BackendException|\OCA\NgBackup\Crypto\CryptoException|\JsonException) {
|
||||||
|
return ['unreadable' => true];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Write the settings once; a location that has them keeps them. The repository id goes with
|
||||||
|
* them: with it and the key from the recovery kit, a lost config can be written again.
|
||||||
|
*/
|
||||||
|
public function recordSettings(array $settings): void {
|
||||||
|
if ($this->backend->exists(self::SETTINGS)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
$settings['repository'] = $this->repositoryId;
|
||||||
|
self::putString($this->backend, self::SETTINGS, $this->cipher->encryptString(json_encode($settings, JSON_THROW_ON_ERROR), 'settings'));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether the location holds backup data, with or without a config. */
|
||||||
|
public static function holdsData(IBackend $backend): bool {
|
||||||
|
foreach (['index', 'snapshots', 'catalog'] as $dir) {
|
||||||
|
if ($backend->listFiles($dir) !== []) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Write a lost config again, from the key (recovery kit) and the repository id in the
|
||||||
|
* sealed settings. Only where there is no config and the settings open with this key, so
|
||||||
|
* nothing that works is ever overwritten and a wrong key writes nothing.
|
||||||
|
*/
|
||||||
|
public static function repairConfig(IBackend $backend, KeyRing $keys, array $wrappedKey): void {
|
||||||
|
if ($backend->exists('config')) {
|
||||||
|
throw new RepositoryException('This location has a config: nothing to repair, leave out --repair-config');
|
||||||
|
}
|
||||||
|
$id = self::settingsAt($backend, $keys)['repository'] ?? null;
|
||||||
|
if (!is_string($id) || !preg_match('/^[0-9a-f]{32}$/', $id)) {
|
||||||
|
throw new RepositoryException($backend->exists(self::SETTINGS)
|
||||||
|
? 'The settings at this location do not open with this key: the recovery kit is of another server, or the settings are damaged. The config cannot be written again.'
|
||||||
|
: 'This location has no settings file (it was backed up only by NG Backup before 1.1.0), so its config cannot be written again.');
|
||||||
|
}
|
||||||
|
$config = ['format' => self::FORMAT, 'id' => $id, 'created' => gmdate('c'), 'key' => $wrappedKey,
|
||||||
|
'check' => base64_encode((new StreamCipher($keys))->encryptString($id, 'config-check')), 'repaired' => true];
|
||||||
|
self::putString($backend, 'config', json_encode($config, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR));
|
||||||
|
}
|
||||||
|
|
||||||
/** Replace the passphrase-wrapped key slots in this repository's config (after a slot change). */
|
/** Replace the passphrase-wrapped key slots in this repository's config (after a slot change). */
|
||||||
public function updateWrappedKey(array $wrappedKey): void {
|
public function updateWrappedKey(array $wrappedKey): void {
|
||||||
$config = self::readConfig($this->backend);
|
$config = self::readConfig($this->backend);
|
||||||
@@ -257,8 +407,8 @@ final class Repository {
|
|||||||
// that was being written when the process was killed stays (8 MiB, DR test 2026-11-05)
|
// that was being written when the process was killed stays (8 MiB, DR test 2026-11-05)
|
||||||
$stale = glob(rtrim($target, '/') . '/.ngb-restore-spool-*', GLOB_ONLYDIR);
|
$stale = glob(rtrim($target, '/') . '/.ngb-restore-spool-*', GLOB_ONLYDIR);
|
||||||
foreach ($stale === false ? [] : $stale as $dir) {
|
foreach ($stale === false ? [] : $stale as $dir) {
|
||||||
if (is_link($dir)) {
|
if (is_link($dir) || preg_match('/^\.ngb-restore-spool-[0-9a-f]{8}$/', basename($dir)) !== 1) {
|
||||||
continue;
|
continue; // not a spool of ours, whatever it is called
|
||||||
}
|
}
|
||||||
$files = glob($dir . '/*');
|
$files = glob($dir . '/*');
|
||||||
foreach ($files === false ? [] : $files as $file) {
|
foreach ($files === false ? [] : $files as $file) {
|
||||||
@@ -313,40 +463,39 @@ final class Repository {
|
|||||||
if (!self::underPrefix($entry['p'], $prefix)) {
|
if (!self::underPrefix($entry['p'], $prefix)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
if ($heartbeat !== null) {
|
||||||
|
$heartbeat(); // also for a long row of files that are skipped or have no data
|
||||||
|
}
|
||||||
$dest = self::destination($target, $entry['p'], $prefix, $stripPrefix);
|
$dest = self::destination($target, $entry['p'], $prefix, $stripPrefix);
|
||||||
if ($keepUnchanged && self::inPlace($dest, $entry)) {
|
$dir = self::directoryInTarget($target, dirname($dest), $entry['p']);
|
||||||
// below the target as written by an earlier run of this restore (checked then)
|
if (is_link($dest)) {
|
||||||
$count++;
|
throw new RestoreTargetException('Refusing to write outside the restore directory: ' . $entry['p']);
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if (!is_dir(dirname($dest))) {
|
|
||||||
mkdir(dirname($dest), 0700, true);
|
|
||||||
}
|
|
||||||
// Defence in depth: the resolved directory must stay below the target (no symlink escapes).
|
|
||||||
$base = realpath($target);
|
|
||||||
$dir = realpath(dirname($dest));
|
|
||||||
if ($base === false || $dir === false || ($dir !== $base && !str_starts_with($dir, $base . '/')) || is_link($dest)) {
|
|
||||||
throw new RepositoryException('Refusing to write outside the restore directory: ' . $entry['p']);
|
|
||||||
}
|
}
|
||||||
if (!isset($cleaned[$dir])) {
|
if (!isset($cleaned[$dir])) {
|
||||||
self::removeStaleTemporaries($dir);
|
self::removeStaleTemporaries($dir);
|
||||||
$cleaned[$dir] = true;
|
$cleaned[$dir] = true;
|
||||||
}
|
}
|
||||||
|
if ($keepUnchanged && self::inPlace($dest, $entry)) {
|
||||||
|
$count++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
// Write next to the destination and rename after all blobs verified: at most one file
|
// Write next to the destination and rename after all blobs verified: at most one file
|
||||||
// is ever present twice.
|
// is ever present twice.
|
||||||
$tmp = $dest . '.ngb-restore-' . bin2hex(random_bytes(6));
|
$tmp = $dest . '.ngb-restore-' . bin2hex(random_bytes(6));
|
||||||
$out = @fopen($tmp, 'xb'); // fails if anything (including a symlink) already exists there
|
$out = @fopen($tmp, 'xb'); // fails if anything (including a symlink) already exists there
|
||||||
if ($out === false) {
|
if ($out === false) {
|
||||||
throw new RepositoryException('Cannot create a temporary file for ' . $entry['p']);
|
throw new RestoreTargetException('Cannot create a temporary file for ' . $entry['p'] . ' (disk full, or no permission?)');
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
foreach ($entry['b'] as $id) {
|
foreach ($entry['b'] as $id) {
|
||||||
fwrite($out, $this->restoreBlob($id, $plan));
|
self::writeAll($out, $this->restoreBlob($id, $plan), $entry['p']);
|
||||||
if ($heartbeat !== null) {
|
if ($heartbeat !== null) {
|
||||||
$heartbeat();
|
$heartbeat();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
fclose($out);
|
if (!@fclose($out)) {
|
||||||
|
throw new RestoreTargetException('Cannot write ' . $entry['p'] . ' to the restore directory (disk full?)');
|
||||||
|
}
|
||||||
clearstatcache(true, $tmp);
|
clearstatcache(true, $tmp);
|
||||||
if (filesize($tmp) !== $entry['s']) {
|
if (filesize($tmp) !== $entry['s']) {
|
||||||
throw new RepositoryException('Size mismatch restoring ' . $entry['p']);
|
throw new RepositoryException('Size mismatch restoring ' . $entry['p']);
|
||||||
@@ -363,19 +512,144 @@ final class Repository {
|
|||||||
}
|
}
|
||||||
$skipped[$entry['p']] = $e->getMessage();
|
$skipped[$entry['p']] = $e->getMessage();
|
||||||
continue;
|
continue;
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
if (is_resource($out)) {
|
||||||
|
@fclose($out);
|
||||||
|
}
|
||||||
|
@unlink($tmp);
|
||||||
|
throw $e;
|
||||||
}
|
}
|
||||||
if (!rename($tmp, $dest)) {
|
if (!rename($tmp, $dest)) {
|
||||||
@unlink($tmp);
|
@unlink($tmp);
|
||||||
throw new RepositoryException('Cannot put the restored file in place: ' . $entry['p']);
|
throw new RestoreTargetException('Cannot put the restored file in place: ' . $entry['p']);
|
||||||
}
|
}
|
||||||
if (!touch($dest, $entry['m'])) {
|
if (!touch($dest, $entry['m'])) {
|
||||||
throw new RepositoryException('Cannot set the modification time of ' . $entry['p']);
|
throw new RestoreTargetException('Cannot set the modification time of ' . $entry['p']);
|
||||||
}
|
}
|
||||||
$count++;
|
$count++;
|
||||||
}
|
}
|
||||||
|
// Last: a file can only be restored into a folder that exists, and the meta record was
|
||||||
|
// read along with the files, so this asks nothing more of the location.
|
||||||
|
foreach ($this->emptyDirs($snapshotId) as [$path, $mtime]) {
|
||||||
|
if (self::underPrefix($path, $prefix)) {
|
||||||
|
if ($heartbeat !== null) {
|
||||||
|
$heartbeat();
|
||||||
|
}
|
||||||
|
// The prefix itself, stripped, is the target (for a file it is the file's name there).
|
||||||
|
$dest = $stripPrefix && $path === rtrim($prefix, '/') ? rtrim($target, '/') : self::destination($target, $path, $prefix, $stripPrefix);
|
||||||
|
self::restoreEmptyDir($target, $dest, $path, $mtime);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Very last, nothing is written in them any more. Also on a run that goes on after a stop, and
|
||||||
|
// the deepest first. The time is not worth stopping for: all the files are there by now.
|
||||||
|
foreach (array_reverse($this->folderTimes($snapshotId)) as [$path, $mtime]) {
|
||||||
|
if (!self::underPrefix($path, $prefix) || ($stripPrefix && $path === rtrim($prefix, '/'))) {
|
||||||
|
continue; // the target itself is not ours to change
|
||||||
|
}
|
||||||
|
if ($heartbeat !== null) {
|
||||||
|
$heartbeat();
|
||||||
|
}
|
||||||
|
$dest = self::destination($target, $path, $prefix, $stripPrefix);
|
||||||
|
@touch(self::directoryInTarget($target, $dest, $path), $mtime);
|
||||||
|
}
|
||||||
return $count;
|
return $count;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The folders with something in them that a snapshot recorded, parents before children (none
|
||||||
|
* before 1.1.0). Damage here is no reason to give up on the files of a restore.
|
||||||
|
*
|
||||||
|
* @return list<array{0: string, 1: int}> path, modification time
|
||||||
|
*/
|
||||||
|
public function folderTimes(string $snapshotId): array {
|
||||||
|
$meta = $this->lastMeta[$snapshotId] ?? $this->snapshotMeta($snapshotId);
|
||||||
|
$packed = is_string($meta['folderTimes'] ?? null) ? base64_decode($meta['folderTimes'], true) : false;
|
||||||
|
$plain = $packed === false ? false : @gzinflate($packed);
|
||||||
|
$folders = [];
|
||||||
|
foreach ($plain === false ? [] : explode("\n", $plain) as $line) {
|
||||||
|
$folder = json_decode($line, true);
|
||||||
|
if (is_array($folder) && is_string($folder[0] ?? null) && self::isSafePath($folder[0]) && is_int($folder[1] ?? null)) {
|
||||||
|
$folders[] = [$folder[0], $folder[1]];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return $folders;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The folders without anything in them that a snapshot recorded (none before 1.0.3).
|
||||||
|
*
|
||||||
|
* @return list<array{0: string, 1: int}> path, modification time
|
||||||
|
*/
|
||||||
|
public function emptyDirs(string $snapshotId): array {
|
||||||
|
$meta = $this->lastMeta[$snapshotId] ?? $this->snapshotMeta($snapshotId);
|
||||||
|
$dirs = [];
|
||||||
|
foreach (is_array($meta['emptyDirs'] ?? null) ? $meta['emptyDirs'] : [] as $dir) {
|
||||||
|
// One folder that makes no sense is no reason to give up on the files of a restore.
|
||||||
|
if (is_array($dir) && is_string($dir[0] ?? null) && self::isSafePath($dir[0]) && is_int($dir[1] ?? null)) {
|
||||||
|
$dirs[] = [$dir[0], $dir[1]];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return $dirs;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function restoreEmptyDir(string $target, string $dest, string $path, int $mtime): void {
|
||||||
|
if (is_link($dest)) {
|
||||||
|
throw new RestoreTargetException('Refusing to write outside the restore directory: ' . $path);
|
||||||
|
}
|
||||||
|
$existed = is_dir($dest);
|
||||||
|
self::directoryInTarget($target, $dest, $path);
|
||||||
|
if (!$existed && !@touch($dest, $mtime)) {
|
||||||
|
throw new RestoreTargetException('Cannot set the modification time of ' . $path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Make sure a folder exists below the target, without following a link out of it: what exists
|
||||||
|
* of the path is checked before anything is created.
|
||||||
|
*
|
||||||
|
* @return string the folder's real path
|
||||||
|
*/
|
||||||
|
private static function directoryInTarget(string $target, string $dir, string $path): string {
|
||||||
|
if (!is_dir($target) && !@mkdir($target, 0700, true) && !is_dir($target)) {
|
||||||
|
throw new RestoreTargetException('Cannot create the restore directory ' . $target);
|
||||||
|
}
|
||||||
|
$base = realpath($target);
|
||||||
|
$existing = $dir;
|
||||||
|
while (!file_exists($existing) && !is_link($existing) && dirname($existing) !== $existing) {
|
||||||
|
$existing = dirname($existing);
|
||||||
|
}
|
||||||
|
$real = realpath($existing);
|
||||||
|
if ($base === false || $real === false || ($real !== $base && !str_starts_with($real, $base . '/'))) {
|
||||||
|
throw new RestoreTargetException('Refusing to write outside the restore directory: ' . $path);
|
||||||
|
}
|
||||||
|
if ($existing === $dir) {
|
||||||
|
if (!is_dir($real)) {
|
||||||
|
throw new RestoreTargetException('Cannot create the folder for ' . $path . ': a file is in the way');
|
||||||
|
}
|
||||||
|
return $real;
|
||||||
|
}
|
||||||
|
if (!@mkdir($dir, 0700, true) && !is_dir($dir)) {
|
||||||
|
throw new RestoreTargetException('Cannot create the folder for ' . $path . ' (disk full, no permission, or a file in the way?)');
|
||||||
|
}
|
||||||
|
return (string)realpath($dir);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Write all of $data, or stop: a short write leaves a file that looks damaged only by its size.
|
||||||
|
* A write may take only part of the data and still be fine; one that takes nothing is not.
|
||||||
|
*
|
||||||
|
* @param resource $out
|
||||||
|
*/
|
||||||
|
public static function writeAll($out, string $data, string $path): void {
|
||||||
|
while ($data !== '') {
|
||||||
|
$written = @fwrite($out, $data);
|
||||||
|
if ($written === false || $written === 0) {
|
||||||
|
throw new RestoreTargetException('Cannot write ' . $path . ' to the restore directory (disk full?)');
|
||||||
|
}
|
||||||
|
$data = substr($data, $written);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Where restore() writes the snapshot path $path. */
|
/** Where restore() writes the snapshot path $path. */
|
||||||
private static function destination(string $target, string $path, string $prefix, bool $stripPrefix): string {
|
private static function destination(string $target, string $path, string $prefix, bool $stripPrefix): string {
|
||||||
$relPath = $path;
|
$relPath = $path;
|
||||||
@@ -722,6 +996,34 @@ final class Repository {
|
|||||||
return array_keys($doomed);
|
return array_keys($doomed);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Part of trash(), for a page that shows how far it is: each entry is a read of its own on the
|
||||||
|
* location. Reads the entries with an id after $after, in order, until $deadline
|
||||||
|
* (microtime(true)), at least one. The id, not a position: a cleanup that changes the trash
|
||||||
|
* between two parts then makes no entry come twice or be missed.
|
||||||
|
*
|
||||||
|
* @return array{total: int, items: array<string, array<string, mixed>>}
|
||||||
|
*/
|
||||||
|
public function trashPage(string $after, float $deadline): array {
|
||||||
|
$ids = array_map('strval', array_keys($this->catalog->trash));
|
||||||
|
sort($ids, SORT_STRING);
|
||||||
|
$items = [];
|
||||||
|
foreach ($ids as $id) {
|
||||||
|
if (strcmp($id, $after) <= 0) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
$items[$id] = $this->trashInfo($id);
|
||||||
|
} catch (\Throwable) {
|
||||||
|
throw new RepositoryException("Trash entry $id listed in the catalog is missing on the location (possible tampering)");
|
||||||
|
}
|
||||||
|
if (microtime(true) >= $deadline) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ['total' => count($ids), 'items' => $items];
|
||||||
|
}
|
||||||
|
|
||||||
private function trashInfo(string $snapshotId): array {
|
private function trashInfo(string $snapshotId): array {
|
||||||
return json_decode($this->getObject('trash/info/' . $snapshotId), true, 512, JSON_THROW_ON_ERROR);
|
return json_decode($this->getObject('trash/info/' . $snapshotId), true, 512, JSON_THROW_ON_ERROR);
|
||||||
}
|
}
|
||||||
@@ -863,7 +1165,10 @@ final class Repository {
|
|||||||
for ($attempt = 1; $attempt <= 3; $attempt++) {
|
for ($attempt = 1; $attempt <= 3; $attempt++) {
|
||||||
$before = self::fileVersion($path);
|
$before = self::fileVersion($path);
|
||||||
$blobs = [];
|
$blobs = [];
|
||||||
$fh = fopen($path, 'rb');
|
$fh = self::openForReading($path);
|
||||||
|
if ($fh === false) {
|
||||||
|
throw new RepositoryException('Cannot read ' . $path);
|
||||||
|
}
|
||||||
while (($data = self::readFull($fh, self::BLOB_SIZE)) !== '') {
|
while (($data = self::readFull($fh, self::BLOB_SIZE)) !== '') {
|
||||||
$blobs[] = $this->storeBlob($data, $packs, $stats);
|
$blobs[] = $this->storeBlob($data, $packs, $stats);
|
||||||
}
|
}
|
||||||
@@ -1281,7 +1586,10 @@ final class Repository {
|
|||||||
$this->cipher->decrypt($fh, $plain, 'snapshot:' . $snapshotId);
|
$this->cipher->decrypt($fh, $plain, 'snapshot:' . $snapshotId);
|
||||||
fclose($fh);
|
fclose($fh);
|
||||||
rewind($plain);
|
rewind($plain);
|
||||||
fgets($plain); // meta line
|
$meta = json_decode((string)fgets($plain), true);
|
||||||
|
if (is_array($meta)) {
|
||||||
|
$this->lastMeta = [$snapshotId => $meta]; // a snapshot never changes
|
||||||
|
}
|
||||||
while (($line = fgets($plain)) !== false) {
|
while (($line = fgets($plain)) !== false) {
|
||||||
$entry = json_decode($line, true, 512, JSON_THROW_ON_ERROR);
|
$entry = json_decode($line, true, 512, JSON_THROW_ON_ERROR);
|
||||||
if (!is_array($entry) || !is_string($entry['p'] ?? null) || !self::isSafePath($entry['p'])
|
if (!is_array($entry) || !is_string($entry['p'] ?? null) || !self::isSafePath($entry['p'])
|
||||||
|
|||||||
@@ -0,0 +1,15 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Repository;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A restore cannot write to where it restores to (a full disk, permissions, something in the way).
|
||||||
|
* The backup is fine, so this never counts as damaged data: the restore stops, also with --salvage.
|
||||||
|
*/
|
||||||
|
final class RestoreTargetException extends \RuntimeException {
|
||||||
|
}
|
||||||
@@ -61,6 +61,16 @@ final class AlertService {
|
|||||||
'examples' => mb_strimwidth(implode(', ', array_merge($examples['unreadable'], $examples['changing'])), 0, 300, '…')]);
|
'examples' => mb_strimwidth(implode(', ', array_merge($examples['unreadable'], $examples['changing'])), 0, 300, '…')]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A backup that finished but left out files or folders because of their name.
|
||||||
|
*
|
||||||
|
* @param list<string> $examples
|
||||||
|
*/
|
||||||
|
public function runBadNames(Run $run, string $targetName, int $count, array $examples): void {
|
||||||
|
$this->notifyAdmins('run_bad_names', 'run-' . $run->getId(), ['target' => $targetName, 'count' => (string)$count,
|
||||||
|
'examples' => mb_strimwidth(implode(', ', $examples), 0, 300, '…')]);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Remember how much is stored on a location (for the settings page) and warn administrators
|
* Remember how much is stored on a location (for the settings page) and warn administrators
|
||||||
* once a day when it is at 80% of its limit or more.
|
* once a day when it is at 80% of its limit or more.
|
||||||
@@ -91,8 +101,35 @@ final class AlertService {
|
|||||||
return $v === '' ? null : (int)$v;
|
return $v === '' ? null : (int)$v;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether it is time to remind administrators of a location without a schedule: once a week
|
||||||
|
* while another location is still backed up, every day when none is. An hour of margin, so
|
||||||
|
* that the moment of the cron run does not push a reminder to the next day.
|
||||||
|
*/
|
||||||
|
public static function offReminderDue(int $last, int $now, bool $othersActive): bool {
|
||||||
|
return $now - $last >= ($othersActive ? 7 * 86400 : 86400) - 3600;
|
||||||
|
}
|
||||||
|
|
||||||
|
public function checkOff(Target $target, bool $othersActive): void {
|
||||||
|
if ((string)$target->getSchedule() !== '') {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
$key = 'off_notified_' . $target->getId();
|
||||||
|
$now = time();
|
||||||
|
$last = max((int)$target->getScheduleSince(), $this->appConfig->getValueInt(Application::APP_ID, $key, 0));
|
||||||
|
if (!self::offReminderDue($last, $now, $othersActive)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, $key, $now);
|
||||||
|
$this->notifyAdmins('schedule_off', 'off-' . $target->getId() . '-' . date('Y-m-d', $now),
|
||||||
|
['target' => $target->getName(), 'others' => $othersActive ? '1' : '0']);
|
||||||
|
}
|
||||||
|
|
||||||
/** At most one notification per location per day. */
|
/** At most one notification per location per day. */
|
||||||
public function checkStale(Target $target): void {
|
public function checkStale(Target $target): void {
|
||||||
|
if ((string)$target->getSchedule() === '') {
|
||||||
|
return; // no schedule is a choice, as for a location that is only connected now and then
|
||||||
|
}
|
||||||
$last =$this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
|
$last =$this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
|
||||||
$lastTime = $last?->getFinishedAt() ?? $target->getCreatedAt();
|
$lastTime = $last?->getFinishedAt() ?? $target->getCreatedAt();
|
||||||
if (time() - $lastTime < self::STALE_AFTER || $this->runs->findRunning($target->getId(), BackupService::KIND_FULL) !== []) {
|
if (time() - $lastTime < self::STALE_AFTER || $this->runs->findRunning($target->getId(), BackupService::KIND_FULL) !== []) {
|
||||||
|
|||||||
@@ -58,6 +58,18 @@ final class BackupService {
|
|||||||
$this->openedWith = $opened;
|
$this->openedWith = $opened;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The running backups in the order they get their turn: first come, first served. The first one
|
||||||
|
* runs, the others are queued; only one at a time reads the disks.
|
||||||
|
*
|
||||||
|
* @return list<Run>
|
||||||
|
*/
|
||||||
|
public function runsInTurn(): array {
|
||||||
|
$running = $this->runs->findRunning(null, self::KIND_FULL);
|
||||||
|
usort($running, static fn (Run $a, Run $b): int => [$a->getStartedAt(), $a->getId()] <=> [$b->getStartedAt(), $b->getId()]);
|
||||||
|
return $running;
|
||||||
|
}
|
||||||
|
|
||||||
public function start(Target $target, string $label = ''): Run {
|
public function start(Target $target, string $label = ''): Run {
|
||||||
$label = trim($label);
|
$label = trim($label);
|
||||||
if (mb_strlen($label) > self::LABEL_MAX) {
|
if (mb_strlen($label) > self::LABEL_MAX) {
|
||||||
@@ -79,7 +91,7 @@ final class BackupService {
|
|||||||
try {
|
try {
|
||||||
return $this->leases->with(PruneService::lockKey($target), LeaseService::EXCLUSIVE, 60, fn () => $this->createRun($target, $label));
|
return $this->leases->with(PruneService::lockKey($target), LeaseService::EXCLUSIVE, 60, fn () => $this->createRun($target, $label));
|
||||||
} catch (LockedException) {
|
} catch (LockedException) {
|
||||||
throw new \RuntimeException('A cleanup or another start for ' . $target->getName() . ' is running; try again later');
|
throw new \RuntimeException('A cleanup, a restore or another start for ' . $target->getName() . ' is running, or one was stopped by force (that clears by itself within two hours); try again later');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -210,6 +222,13 @@ final class BackupService {
|
|||||||
$state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
|
$state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
|
||||||
if ($run->getPhase() === 'db') {
|
if ($run->getPhase() === 'db') {
|
||||||
|
if (!isset($state['dump'])) {
|
||||||
|
// It may have been queued for hours: the restore point has the time of its data.
|
||||||
|
$state['files']['started'] = gmdate('c');
|
||||||
|
// Locations from before 1.1.0 get their sealed settings at their next backup.
|
||||||
|
$this->targets->recordSettings($repo);
|
||||||
|
$this->targets->catchUpWrappedKey($target, $repo);
|
||||||
|
}
|
||||||
$dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_'));
|
$dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_'));
|
||||||
$dbStats = $state['dbStats'] ?? [];
|
$dbStats = $state['dbStats'] ?? [];
|
||||||
// With system cron the whole dump usually fits in one step (one transaction, fully
|
// With system cron the whole dump usually fits in one step (one transaction, fully
|
||||||
@@ -253,8 +272,13 @@ final class BackupService {
|
|||||||
try {
|
try {
|
||||||
$this->alerts->storedBytes($target, $repo->storedBytes());
|
$this->alerts->storedBytes($target, $repo->storedBytes());
|
||||||
$stats = $state['files']['stats'];
|
$stats = $state['files']['stats'];
|
||||||
if (($stats['unreadable'] ?? 0) > 0 || ($stats['changing'] ?? 0) > 0) {
|
|
||||||
$warnings = BackupRun::warnings($state['files']);
|
$warnings = BackupRun::warnings($state['files']);
|
||||||
|
if (($stats['badNames'] ?? 0) > 0) {
|
||||||
|
$this->logger->warning(sprintf('NG Backup run %d to %s: %d left out because their name is not supported (rename them): %s',
|
||||||
|
$run->getId(), $target->getName(), $stats['badNames'], implode(', ', $warnings['badNames'])));
|
||||||
|
$this->alerts->runBadNames($run, $target->getName(), (int)$stats['badNames'], $warnings['badNames']);
|
||||||
|
}
|
||||||
|
if (($stats['unreadable'] ?? 0) > 0 || ($stats['changing'] ?? 0) > 0) {
|
||||||
$this->logger->warning(sprintf('NG Backup run %d to %s: %d unreadable (missing from the snapshot): %s; %d changed while being read: %s',
|
$this->logger->warning(sprintf('NG Backup run %d to %s: %d unreadable (missing from the snapshot): %s; %d changed while being read: %s',
|
||||||
$run->getId(), $target->getName(), $stats['unreadable'] ?? 0, implode(', ', $warnings['unreadable']),
|
$run->getId(), $target->getName(), $stats['unreadable'] ?? 0, implode(', ', $warnings['unreadable']),
|
||||||
$stats['changing'] ?? 0, implode(', ', $warnings['changing'])));
|
$stats['changing'] ?? 0, implode(', ', $warnings['changing'])));
|
||||||
|
|||||||
@@ -0,0 +1,12 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
/** A step after the database was committed failed: the database is no longer the fresh one. */
|
||||||
|
final class DatabaseAlreadyRestoredException extends \RuntimeException {
|
||||||
|
}
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\AppInfo\Application;
|
||||||
|
use OCP\DB\QueryBuilder\IQueryBuilder;
|
||||||
|
use OCP\IAppConfig;
|
||||||
|
use OCP\IDBConnection;
|
||||||
|
use Psr\Log\LoggerInterface;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Keeps heavy background jobs of other apps from running while a backup reads the same disks, by
|
||||||
|
* reserving them in Nextcloud's job list: no app is turned off, and a job that is running goes on.
|
||||||
|
* Nextcloud itself gives up on a reservation after 12 hours, so a pause that is never ended, because
|
||||||
|
* cron or this app is gone, ends by itself. It lasts at most MAX_SECONDS, then the jobs get REST_SECONDS.
|
||||||
|
*/
|
||||||
|
final class HeavyJobPause {
|
||||||
|
public const ENABLED_KEY = 'pause_heavy_jobs';
|
||||||
|
private const SINCE_KEY = 'heavy_jobs_paused_since';
|
||||||
|
private const REST_KEY = 'heavy_jobs_rest_until';
|
||||||
|
public const MAX_SECONDS = 6 * 3600;
|
||||||
|
public const REST_SECONDS = 3600;
|
||||||
|
/** Nextcloud runs a job again this long after it was reserved. */
|
||||||
|
private const RESERVATION_SECONDS = 12 * 3600;
|
||||||
|
|
||||||
|
/** Class, or the start of a class name ending in a backslash => what to call it */
|
||||||
|
private const JOBS = [
|
||||||
|
'OCA\\Memories\\' => 'Memories',
|
||||||
|
'OCA\\Files_Antivirus\\BackgroundJob\\BackgroundScanner' => 'Antivirus background scan',
|
||||||
|
'OCA\\Recognize\\' => 'Recognize',
|
||||||
|
'OCA\\PreviewGenerator\\' => 'Preview Generator',
|
||||||
|
'OCA\\FullTextSearch\\' => 'Full text search',
|
||||||
|
];
|
||||||
|
|
||||||
|
public const NOTHING = 'nothing';
|
||||||
|
public const PAUSE = 'pause';
|
||||||
|
public const RELEASE = 'release';
|
||||||
|
public const REST = 'rest';
|
||||||
|
|
||||||
|
public function __construct(
|
||||||
|
private IDBConnection $db,
|
||||||
|
private IAppConfig $appConfig,
|
||||||
|
private LoggerInterface $logger,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return self::NOTHING|self::PAUSE|self::RELEASE|self::REST */
|
||||||
|
public static function decide(bool $enabled, bool $backupRunning, int $since, int $restUntil, int $now): string {
|
||||||
|
if ($since > 0 && $now - $since >= self::MAX_SECONDS) {
|
||||||
|
return self::REST;
|
||||||
|
}
|
||||||
|
if ($enabled && $backupRunning && $now >= $restUntil) {
|
||||||
|
return self::PAUSE;
|
||||||
|
}
|
||||||
|
return $since > 0 ? self::RELEASE : self::NOTHING;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Call regularly: pauses or ends the pause as needed. A problem here never stops a backup. */
|
||||||
|
public function sync(bool $backupRunning): void {
|
||||||
|
try {
|
||||||
|
$this->apply($backupRunning, time());
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$this->logger->warning('NG Backup: pausing heavy background jobs failed: ' . $e->getMessage(), ['exception' => $e]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private function apply(bool $backupRunning, int $now): void {
|
||||||
|
$since = $this->appConfig->getValueInt(Application::APP_ID, self::SINCE_KEY, 0);
|
||||||
|
$restUntil = $this->appConfig->getValueInt(Application::APP_ID, self::REST_KEY, 0);
|
||||||
|
switch (self::decide($this->enabled(), $backupRunning, $since, $restUntil, $now)) {
|
||||||
|
case self::PAUSE:
|
||||||
|
if ($since === 0) {
|
||||||
|
$since = $now;
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, self::SINCE_KEY, $since);
|
||||||
|
$this->logger->info('NG Backup: heavy background jobs paused while a backup runs');
|
||||||
|
}
|
||||||
|
$this->reserve($since, $now); // also the jobs that were added or running until now
|
||||||
|
break;
|
||||||
|
case self::REST:
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, self::REST_KEY, $now + self::REST_SECONDS);
|
||||||
|
$this->release($since);
|
||||||
|
$this->logger->info('NG Backup: heavy background jobs get their turn for an hour during a long backup');
|
||||||
|
break;
|
||||||
|
case self::RELEASE:
|
||||||
|
$this->release($since);
|
||||||
|
$this->logger->info('NG Backup: heavy background jobs run again');
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private function reserve(int $stamp, int $now): void {
|
||||||
|
$qb = $this->db->getQueryBuilder();
|
||||||
|
$qb->update('jobs')
|
||||||
|
->set('reserved_at', $qb->createNamedParameter($stamp, IQueryBuilder::PARAM_INT))
|
||||||
|
->where($qb->expr()->lte('reserved_at', $qb->createNamedParameter($now - self::RESERVATION_SECONDS, IQueryBuilder::PARAM_INT)))
|
||||||
|
->andWhere($this->isHeavy($qb));
|
||||||
|
$qb->executeStatement();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Only our own reservations: a job that Nextcloud reserved at another moment is running. */
|
||||||
|
private function release(int $stamp): void {
|
||||||
|
$qb = $this->db->getQueryBuilder();
|
||||||
|
$qb->update('jobs')
|
||||||
|
->set('reserved_at', $qb->createNamedParameter(0, IQueryBuilder::PARAM_INT))
|
||||||
|
->where($qb->expr()->eq('reserved_at', $qb->createNamedParameter($stamp, IQueryBuilder::PARAM_INT)))
|
||||||
|
->andWhere($this->isHeavy($qb));
|
||||||
|
$qb->executeStatement();
|
||||||
|
$this->appConfig->deleteKey(Application::APP_ID, self::SINCE_KEY);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return \OCP\DB\QueryBuilder\ICompositeExpression */
|
||||||
|
private function isHeavy(IQueryBuilder $qb) {
|
||||||
|
$conditions = [];
|
||||||
|
foreach (array_keys(self::JOBS) as $class) {
|
||||||
|
$conditions[] = str_ends_with($class, '\\')
|
||||||
|
? $qb->expr()->like('class', $qb->createNamedParameter($this->db->escapeLikeParameter($class) . '%'))
|
||||||
|
: $qb->expr()->eq('class', $qb->createNamedParameter($class));
|
||||||
|
}
|
||||||
|
return $qb->expr()->orX(...$conditions);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function enabled(): bool {
|
||||||
|
return $this->appConfig->getValueString(Application::APP_ID, self::ENABLED_KEY, '1') !== '0';
|
||||||
|
}
|
||||||
|
|
||||||
|
public function setEnabled(bool $enabled): void {
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::ENABLED_KEY, $enabled ? '1' : '0');
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array{enabled: bool, since: int, jobs: list<string>} what is paused now */
|
||||||
|
public function status(): array {
|
||||||
|
$since = $this->appConfig->getValueInt(Application::APP_ID, self::SINCE_KEY, 0);
|
||||||
|
$jobs = [];
|
||||||
|
if ($since > 0) {
|
||||||
|
$qb = $this->db->getQueryBuilder();
|
||||||
|
$qb->selectDistinct('class')->from('jobs')
|
||||||
|
->where($qb->expr()->eq('reserved_at', $qb->createNamedParameter($since, IQueryBuilder::PARAM_INT)))
|
||||||
|
->andWhere($this->isHeavy($qb));
|
||||||
|
$result = $qb->executeQuery();
|
||||||
|
while (($class = $result->fetchOne()) !== false) {
|
||||||
|
foreach (self::JOBS as $start => $name) {
|
||||||
|
if (str_starts_with((string)$class, $start)) {
|
||||||
|
$jobs[$name] = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$result->closeCursor();
|
||||||
|
}
|
||||||
|
return ['enabled' => $this->enabled(), 'since' => $since, 'jobs' => array_map('strval', array_keys($jobs))];
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -30,6 +30,9 @@ final class KeyService {
|
|||||||
private const KEY_DELETE_DELAY = 'delete_delay_days';
|
private const KEY_DELETE_DELAY = 'delete_delay_days';
|
||||||
private const KEY_KIT_VERSION = 'recovery_kit_version';
|
private const KEY_KIT_VERSION = 'recovery_kit_version';
|
||||||
private const KEY_KIT_CODE = 'recovery_kit_code';
|
private const KEY_KIT_CODE = 'recovery_kit_code';
|
||||||
|
/** Set once a location was created or adopted with this key: from then on it is never replaced. */
|
||||||
|
private const KEY_USED = 'master_key_used';
|
||||||
|
private const KEY_WRAPPED_PENDING = 'master_key_wrapped_pending';
|
||||||
public const DEFAULT_DELETE_DELAY = 7;
|
public const DEFAULT_DELETE_DELAY = 7;
|
||||||
/**
|
/**
|
||||||
* The location holds the passphrase-wrapped key too, so whoever can read it can try
|
* The location holds the passphrase-wrapped key too, so whoever can read it can try
|
||||||
@@ -83,28 +86,100 @@ final class KeyService {
|
|||||||
* @param array<string, mixed> $kit as produced by recoveryKit()
|
* @param array<string, mixed> $kit as produced by recoveryKit()
|
||||||
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase, or a damaged/foreign kit
|
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase, or a damaged/foreign kit
|
||||||
*/
|
*/
|
||||||
public function importRecoveryKit(array $kit, #[\SensitiveParameter] string $passphrase): void {
|
public function importRecoveryKit(array $kit, #[\SensitiveParameter] string $passphrase, bool $replaceUnused = false): void {
|
||||||
if ($this->isInitialized()) {
|
if ($this->isInitialized() && !$replaceUnused) {
|
||||||
throw new \RuntimeException('A backup key already exists');
|
throw new \RuntimeException('A backup key already exists');
|
||||||
}
|
}
|
||||||
if (!isset($kit['wrapped_key'])) {
|
if (!isset($kit['wrapped_key'])) {
|
||||||
throw new \InvalidArgumentException('Not a NG Backup recovery kit file');
|
throw new \InvalidArgumentException('Not a NG Backup recovery kit file');
|
||||||
}
|
}
|
||||||
$keys = KeyRing::unwrapAny($kit['wrapped_key'], $passphrase);
|
$keys = KeyRing::unwrapAny($kit['wrapped_key'], $passphrase);
|
||||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
|
$this->discardUnused($keys, $replaceUnused);
|
||||||
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
|
|
||||||
$this->cached = $keys;
|
|
||||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode($kit['wrapped_key'], JSON_THROW_ON_ERROR), true, true);
|
|
||||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
|
|
||||||
// Kits from before this field existed fall back to the default.
|
// Kits from before this field existed fall back to the default.
|
||||||
$delay = (int)($kit['delete_delay_days'] ?? self::DEFAULT_DELETE_DELAY);
|
$this->store($keys, $kit['wrapped_key'], (int)($kit['delete_delay_days'] ?? self::DEFAULT_DELETE_DELAY), (int)($kit['kit_version'] ?? 1));
|
||||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY, ($delay >= 1 && $delay <= 365) ? $delay : self::DEFAULT_DELETE_DELAY, true);
|
|
||||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, (int)($kit['kit_version'] ?? 1), true);
|
|
||||||
// Having and successfully using a real kit file is stronger proof than the normal
|
// Having and successfully using a real kit file is stronger proof than the normal
|
||||||
// download-and-confirm flow it replaces.
|
// download-and-confirm flow it replaces.
|
||||||
$this->confirmRecoveryKit('cli-import');
|
$this->confirmRecoveryKit('cli-import');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Disaster recovery without the recovery kit: adopt the master key that a passphrase opened
|
||||||
|
* from the key slots in a location's config (TargetService::addWithPassphrase). There is no
|
||||||
|
* kit, so none is confirmed: backups wait until a new kit is downloaded and confirmed.
|
||||||
|
*
|
||||||
|
* @param ?int $deleteDelayDays from the location's sealed settings, null when it has none
|
||||||
|
*/
|
||||||
|
public function adoptKey(KeyRing $keys, array $wrappedKey, ?int $deleteDelayDays, bool $replaceUnused = false): void {
|
||||||
|
if ($this->isInitialized() && !$replaceUnused) {
|
||||||
|
throw new \RuntimeException('A backup key already exists');
|
||||||
|
}
|
||||||
|
$this->discardUnused($keys, $replaceUnused);
|
||||||
|
$this->store($keys, $wrappedKey, $deleteDelayDays ?? self::DEFAULT_DELETE_DELAY, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether a location was ever created or adopted with this key (TargetService::keyIsUnused() also looks at the database). */
|
||||||
|
public function wasUsed(): bool {
|
||||||
|
return $this->appConfig->getValueBool(Application::APP_ID, self::KEY_USED, false, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function markUsed(): void {
|
||||||
|
if (!$this->wasUsed()) {
|
||||||
|
$this->appConfig->setValueBool(Application::APP_ID, self::KEY_USED, true, true);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Locations that missed the last passphrase change (offline at the time): they still hold
|
||||||
|
* the previous slots until the change reaches them, which is tried again at each backup.
|
||||||
|
*/
|
||||||
|
public function wrappedKeyPending(int $targetId): bool {
|
||||||
|
return $this->appConfig->getValueBool(Application::APP_ID, self::KEY_WRAPPED_PENDING . '_' . $targetId);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One flag per location, so that two processes never overwrite each other's. */
|
||||||
|
public function setWrappedKeyPending(int $targetId, bool $pending): void {
|
||||||
|
$key = self::KEY_WRAPPED_PENDING . '_' . $targetId;
|
||||||
|
if ($pending) {
|
||||||
|
$this->appConfig->setValueBool(Application::APP_ID, $key, true);
|
||||||
|
} elseif ($this->wrappedKeyPending($targetId)) {
|
||||||
|
$this->appConfig->deleteKey(Application::APP_ID, $key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A key made by mistake on a server that is being restored: remove it, so the key of the
|
||||||
|
* backups can take its place. Only after the new key was opened with its passphrase, and
|
||||||
|
* only when the caller checked that the old key never protected anything
|
||||||
|
* (TargetService::keyIsUnused()). Its passphrases and recovery kit become worthless.
|
||||||
|
*/
|
||||||
|
private function discardUnused(KeyRing $replacement, bool $replaceUnused): void {
|
||||||
|
if (!$replaceUnused || !$this->isInitialized()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if ($this->wasUsed()) {
|
||||||
|
throw new \RuntimeException('The backup key of this server has been used for a backup location: it is never replaced');
|
||||||
|
}
|
||||||
|
if ($this->fingerprintOf($replacement) === $this->fingerprint()) {
|
||||||
|
throw new \RuntimeException('This server already has this backup key: nothing to replace');
|
||||||
|
}
|
||||||
|
foreach ([self::KEY_MASTER, self::KEY_WRAPPED, self::KEY_CONFIRMED, self::KEY_FINGERPRINT, self::KEY_DELETE_DELAY,
|
||||||
|
self::KEY_KIT_VERSION, self::KEY_KIT_CODE, self::KEY_USED] as $key) {
|
||||||
|
$this->appConfig->deleteKey(Application::APP_ID, $key);
|
||||||
|
}
|
||||||
|
$this->cached = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private function store(KeyRing $keys, array $wrappedKey, int $deleteDelayDays, int $kitVersion): void {
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
|
||||||
|
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
|
||||||
|
$this->cached = $keys;
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode($wrappedKey, JSON_THROW_ON_ERROR), true, true);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY,
|
||||||
|
($deleteDelayDays >= 1 && $deleteDelayDays <= 365) ? $deleteDelayDays : self::DEFAULT_DELETE_DELAY, true);
|
||||||
|
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, max(1, $kitVersion), true);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The master key and the recovery kit code are stored encrypted with Nextcloud's own
|
* The master key and the recovery kit code are stored encrypted with Nextcloud's own
|
||||||
* instance secret (ICrypto), not with themselves; the wrapped key is not (it is sealed with
|
* instance secret (ICrypto), not with themselves; the wrapped key is not (it is sealed with
|
||||||
|
|||||||
@@ -43,8 +43,10 @@ final class KeySlotService {
|
|||||||
foreach ($this->targets->list() as $t) {
|
foreach ($this->targets->list() as $t) {
|
||||||
try {
|
try {
|
||||||
$this->targets->repository($t)->updateWrappedKey($wrapped);
|
$this->targets->repository($t)->updateWrappedKey($wrapped);
|
||||||
|
$this->keys->setWrappedKeyPending($t->getId(), false);
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
$failed[] = $t->getName();
|
$failed[] = $t->getName();
|
||||||
|
$this->keys->setWrappedKeyPending($t->getId(), true);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
$label = '';
|
$label = '';
|
||||||
@@ -55,7 +57,7 @@ final class KeySlotService {
|
|||||||
}
|
}
|
||||||
$this->alerts->securityEvent($event, ['slot' => (string)$slot, 'label' => $label]);
|
$this->alerts->securityEvent($event, ['slot' => (string)$slot, 'label' => $label]);
|
||||||
if ($failed !== []) {
|
if ($failed !== []) {
|
||||||
throw new \RuntimeException('Saved, but these locations could not be updated (they still accept the previous passphrases): ' . implode(', ', $failed));
|
throw new \RuntimeException('Saved, but these locations could not be updated (they still accept the previous passphrases; the change is tried again at their next backup): ' . implode(', ', $failed));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -76,6 +76,15 @@ final class LeaseService {
|
|||||||
$qb->executeStatement();
|
$qb->executeStatement();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Remove the shared leases on $name, for a caller that knows their holders were stopped. */
|
||||||
|
public function dropShared(string $name): void {
|
||||||
|
$qb = $this->db->getQueryBuilder();
|
||||||
|
$qb->delete('ngb_leases')
|
||||||
|
->where($qb->expr()->eq('name', $qb->createNamedParameter($name)))
|
||||||
|
->andWhere($qb->expr()->eq('mode', $qb->createNamedParameter(self::SHARED)));
|
||||||
|
$qb->executeStatement();
|
||||||
|
}
|
||||||
|
|
||||||
/** Wraps $refresh so calling it often (per file, per row batch) only touches the database every $seconds. */
|
/** Wraps $refresh so calling it often (per file, per row batch) only touches the database every $seconds. */
|
||||||
public static function throttled(callable $refresh, int $seconds = 60): \Closure {
|
public static function throttled(callable $refresh, int $seconds = 60): \Closure {
|
||||||
$last = time();
|
$last = time();
|
||||||
|
|||||||
@@ -0,0 +1,78 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* When a location is backed up: "" (off), "HH:MM" (daily) or "HH:MM/N" (from HH:MM every N hours,
|
||||||
|
* starting over at HH:MM the next day), in the location's time zone.
|
||||||
|
*/
|
||||||
|
final class LocationSchedule {
|
||||||
|
/** Attempts per moment after a failure, an hour apart, instead of one every few minutes. */
|
||||||
|
public const MAX_ATTEMPTS = 3;
|
||||||
|
public const RETRY_AFTER = 3600;
|
||||||
|
|
||||||
|
/** @throws \InvalidArgumentException */
|
||||||
|
public static function normalize(string $schedule): string {
|
||||||
|
$schedule = trim($schedule);
|
||||||
|
if ($schedule === '' || strtolower($schedule) === 'off') {
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
if (preg_match('/^(([01]\d|2[0-3]):[0-5]\d)(?:\/(\d{1,2}))?$/', $schedule, $m) !== 1) {
|
||||||
|
throw new \InvalidArgumentException('Use "off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM');
|
||||||
|
}
|
||||||
|
$hours = (int)($m[3] ?? 24);
|
||||||
|
if ($hours < 1 || $hours > 24) {
|
||||||
|
throw new \InvalidArgumentException('Every 1 to 24 hours');
|
||||||
|
}
|
||||||
|
return $hours === 24 ? $m[1] : $m[1] . '/' . $hours;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The most recent scheduled moment at or before $now; null without a schedule.
|
||||||
|
*
|
||||||
|
* @param non-empty-string $zone
|
||||||
|
*/
|
||||||
|
public static function lastMoment(string $schedule, string $zone, int $now): ?int {
|
||||||
|
if ($schedule === '') {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
$tz = new \DateTimeZone($zone);
|
||||||
|
$today = (new \DateTimeImmutable('@' . $now))->setTimezone($tz);
|
||||||
|
$moments = array_merge(self::moments($schedule, $today->modify('-1 day'), $tz), self::moments($schedule, $today, $tz));
|
||||||
|
$last = null;
|
||||||
|
foreach ($moments as $moment) {
|
||||||
|
if ($moment <= $now && ($last === null || $moment > $last)) {
|
||||||
|
$last = $moment;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return $last;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return list<int> the moments of one day */
|
||||||
|
private static function moments(string $schedule, \DateTimeImmutable $day, \DateTimeZone $tz): array {
|
||||||
|
[$time, $every] = array_pad(explode('/', $schedule), 2, '24');
|
||||||
|
[$hour, $minute] = array_map('intval', explode(':', $time));
|
||||||
|
$moments = [];
|
||||||
|
for ($h = $hour; $h < 24; $h += (int)$every) {
|
||||||
|
$moments[] = (new \DateTimeImmutable($day->format('Y-m-d') . sprintf(' %02d:%02d', $h, $minute), $tz))->getTimestamp();
|
||||||
|
}
|
||||||
|
return $moments;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether a backup has to start for $moment. One that was made since then is enough, also when
|
||||||
|
* several moments went by during a long backup. A moment from before the schedule was set does
|
||||||
|
* not count: choosing a time must not start a backup right away.
|
||||||
|
*/
|
||||||
|
public static function isDue(int $moment, int $since, ?int $lastDoneStarted, int $attempts, ?int $lastAttemptFinished, int $now): bool {
|
||||||
|
if ($moment < $since || ($lastDoneStarted !== null && $lastDoneStarted >= $moment) || $attempts >= self::MAX_ATTEMPTS) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return $attempts === 0 || $now - (int)$lastAttemptFinished >= self::RETRY_AFTER;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -11,6 +11,7 @@ use OCA\NgBackup\AppInfo\Application;
|
|||||||
use OCA\NgBackup\Db\RunMapper;
|
use OCA\NgBackup\Db\RunMapper;
|
||||||
use OCA\NgBackup\Db\SnapshotMapper;
|
use OCA\NgBackup\Db\SnapshotMapper;
|
||||||
use OCA\NgBackup\Db\Target;
|
use OCA\NgBackup\Db\Target;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
use OCP\IAppConfig;
|
use OCP\IAppConfig;
|
||||||
use OCP\Lock\ILockingProvider;
|
use OCP\Lock\ILockingProvider;
|
||||||
use OCP\Lock\LockedException;
|
use OCP\Lock\LockedException;
|
||||||
@@ -33,6 +34,54 @@ final class PruneService {
|
|||||||
) {
|
) {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Characters of a note on a snapshot (the column holds 1000) */
|
||||||
|
public const NOTE_MAX = 500;
|
||||||
|
|
||||||
|
/** Pin or unpin a snapshot: the retention policy leaves a pinned one alone. */
|
||||||
|
public function setPinned(Target $target, string $snapshotId, bool $pinned): void {
|
||||||
|
$changed = $this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $pinned): bool {
|
||||||
|
$was = in_array($snapshotId, $repo->pinnedSnapshots(), true);
|
||||||
|
$repo->setPinned($snapshotId, $pinned);
|
||||||
|
return $was !== $pinned;
|
||||||
|
});
|
||||||
|
$this->targets->syncSnapshot($target, $snapshotId, pinned: $pinned);
|
||||||
|
if ($changed) {
|
||||||
|
// Unpinning opens the way to removing it, so every administrator hears about that.
|
||||||
|
$this->alerts->securityEvent($pinned ? 'snapshot_pinned' : 'snapshot_unpinned', ['target' => $target->getName(), 'snapshot' => $snapshotId], !$pinned);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public function setNote(Target $target, string $snapshotId, string $note): void {
|
||||||
|
$note = trim((string)preg_replace('/[\x00-\x1f\x7f]+/', ' ', $note));
|
||||||
|
if (!mb_check_encoding($note, 'UTF-8')) {
|
||||||
|
throw new \InvalidArgumentException('A note must be valid UTF-8');
|
||||||
|
}
|
||||||
|
if (mb_strlen($note) > self::NOTE_MAX) {
|
||||||
|
throw new \InvalidArgumentException('A note has at most ' . self::NOTE_MAX . ' characters');
|
||||||
|
}
|
||||||
|
$this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $note): bool {
|
||||||
|
$repo->setSnapshotNote($snapshotId, $note);
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
$this->targets->syncSnapshot($target, $snapshotId, note: $note);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Write to the catalog next to a running backup or restore, which only takes it for a moment to
|
||||||
|
* commit. Not during a cleanup, which holds a repository opened earlier and writes from that.
|
||||||
|
*
|
||||||
|
* @param callable(Repository): bool $change
|
||||||
|
*/
|
||||||
|
private function changeCatalog(Target $target, callable $change): bool {
|
||||||
|
try {
|
||||||
|
return $this->leases->with(self::lockKey($target), LeaseService::SHARED, 300,
|
||||||
|
fn (): bool => $this->leases->with(self::catalogLockKey($target), LeaseService::EXCLUSIVE, 300,
|
||||||
|
fn (): bool => $change($this->targets->repository($target))));
|
||||||
|
} catch (LockedException) {
|
||||||
|
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' or another change to its restore points is in progress; try again later');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public function untrash(Target $target, string $snapshotId): void {
|
public function untrash(Target $target, string $snapshotId): void {
|
||||||
$lease = $this->lockExclusive($target, 300);
|
$lease = $this->lockExclusive($target, 300);
|
||||||
try {
|
try {
|
||||||
@@ -87,6 +136,7 @@ final class PruneService {
|
|||||||
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
||||||
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
||||||
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
||||||
|
$s->setNote($repo->snapshotNotes()[$snapshotId] ?? null);
|
||||||
try {
|
try {
|
||||||
$this->snapshots->insert($s);
|
$this->snapshots->insert($s);
|
||||||
} catch (\Throwable) {
|
} catch (\Throwable) {
|
||||||
@@ -208,7 +258,7 @@ final class PruneService {
|
|||||||
$meta = $repo->snapshotMeta($id);
|
$meta = $repo->snapshotMeta($id);
|
||||||
$times[$id] = (int)strtotime((string)($meta['time'] ?? '@0'));
|
$times[$id] = (int)strtotime((string)($meta['time'] ?? '@0'));
|
||||||
}
|
}
|
||||||
$keep = $this->policy()->keep($times);
|
$keep = $this->policy()->keepWithPinned($times, $repo->pinnedSnapshots());
|
||||||
$forget = array_values(array_diff(array_keys($times), array_keys($keep)));
|
$forget = array_values(array_diff(array_keys($times), array_keys($keep)));
|
||||||
$actor = $scheduled ? 'retention policy' : $this->alerts->actor();
|
$actor = $scheduled ? 'retention policy' : $this->alerts->actor();
|
||||||
$purged = [];
|
$purged = [];
|
||||||
|
|||||||
@@ -151,6 +151,12 @@ final class RestoreFullService {
|
|||||||
// The database cannot go into this installation; the files can.
|
// The database cannot go into this installation; the files can.
|
||||||
$check['dbUnreadable'] = 'the database cannot be restored here: ' . $this->describeBlockers($check);
|
$check['dbUnreadable'] = 'the database cannot be restored here: ' . $this->describeBlockers($check);
|
||||||
}
|
}
|
||||||
|
if ($interrupted) {
|
||||||
|
// A restore that was killed leaves its leases, for up to two hours: they would keep
|
||||||
|
// the first backup of the recovered server waiting after this restore completes.
|
||||||
|
// In maintenance mode, on a partly restored instance, nothing else reads the location.
|
||||||
|
$this->leases->dropShared(PruneService::lockKey($target));
|
||||||
|
}
|
||||||
$appConfig->setValueInt(\OCA\NgBackup\AppInfo\Application::APP_ID, self::INTERRUPTED_KEY, time());
|
$appConfig->setValueInt(\OCA\NgBackup\AppInfo\Application::APP_ID, self::INTERRUPTED_KEY, time());
|
||||||
$result = $this->restoreInMaintenance($target, $snapshotId, $skipMissingTables, $check, $salvage);
|
$result = $this->restoreInMaintenance($target, $snapshotId, $skipMissingTables, $check, $salvage);
|
||||||
$appConfig->deleteKey(\OCA\NgBackup\AppInfo\Application::APP_ID, self::INTERRUPTED_KEY);
|
$appConfig->deleteKey(\OCA\NgBackup\AppInfo\Application::APP_ID, self::INTERRUPTED_KEY);
|
||||||
@@ -175,6 +181,8 @@ final class RestoreFullService {
|
|||||||
if ($dbError === null) {
|
if ($dbError === null) {
|
||||||
try {
|
try {
|
||||||
$this->restoreDatabase($repo, $meta, $manifest, $dbReport, $beat, $skipMissingTables);
|
$this->restoreDatabase($repo, $meta, $manifest, $dbReport, $beat, $skipMissingTables);
|
||||||
|
} catch (DatabaseAlreadyRestoredException $e) {
|
||||||
|
throw $e;
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
if (!$salvage) {
|
if (!$salvage) {
|
||||||
throw $e;
|
throw $e;
|
||||||
@@ -201,6 +209,8 @@ final class RestoreFullService {
|
|||||||
try {
|
try {
|
||||||
/** @var array<string, mixed> $backupConfig */
|
/** @var array<string, mixed> $backupConfig */
|
||||||
$backupConfig = $this->fetchBackupConfig($target, $snapshotId, $salvage);
|
$backupConfig = $this->fetchBackupConfig($target, $snapshotId, $salvage);
|
||||||
|
} catch (\OCA\NgBackup\Repository\RestoreTargetException $e) {
|
||||||
|
throw $e; // nowhere to put it: not a config.php that cannot be read
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
if (!$salvage) {
|
if (!$salvage) {
|
||||||
throw $e;
|
throw $e;
|
||||||
@@ -259,8 +269,20 @@ final class RestoreFullService {
|
|||||||
$dumper = new DbDumper($this->db->getInner(), $prefix);
|
$dumper = new DbDumper($this->db->getInner(), $prefix);
|
||||||
try {
|
try {
|
||||||
$dumper->restore($repo, $manifest, $dbReport, $beat, $skipMissingTables);
|
$dumper->restore($repo, $manifest, $dbReport, $beat, $skipMissingTables);
|
||||||
} finally {
|
} catch (\Throwable $e) {
|
||||||
|
try {
|
||||||
$this->restoreOwnAppConfig($ownAppConfig);
|
$this->restoreOwnAppConfig($ownAppConfig);
|
||||||
|
} catch (\Throwable) {
|
||||||
|
// rolled back, so they are still there: the error that brought us here is the one to report
|
||||||
|
}
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
$this->restoreOwnAppConfig($ownAppConfig);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
// The old server's database is in place by now: not a case for "the database stays as
|
||||||
|
// the fresh installation made it" under --salvage. A run again restores these rows.
|
||||||
|
throw new DatabaseAlreadyRestoredException('The database was restored, but NG Backup could not put back its own settings: ' . $e->getMessage(), 0, $e);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -61,7 +61,8 @@ final class RestoreService {
|
|||||||
$prefix = trim($path, '/');
|
$prefix = trim($path, '/');
|
||||||
$prefix = $prefix === '' ? '' : $prefix . '/';
|
$prefix = $prefix === '' ? '' : $prefix . '/';
|
||||||
$children = [];
|
$children = [];
|
||||||
foreach ($this->targets->repository($target)->entries($snapshotId) as $e) {
|
$repo = $this->targets->repository($target);
|
||||||
|
foreach ($repo->entries($snapshotId) as $e) {
|
||||||
if ($prefix !== '' && !str_starts_with($e['p'], $prefix)) {
|
if ($prefix !== '' && !str_starts_with($e['p'], $prefix)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
@@ -75,6 +76,12 @@ final class RestoreService {
|
|||||||
}
|
}
|
||||||
$children[$name] = $c;
|
$children[$name] = $c;
|
||||||
}
|
}
|
||||||
|
foreach ($repo->emptyDirs($snapshotId) as [$dir]) { // read along with the entries
|
||||||
|
if ($prefix === '' || str_starts_with($dir, $prefix)) {
|
||||||
|
$name = explode('/', substr($dir, strlen($prefix)), 2)[0];
|
||||||
|
$children[$name] ??= ['name' => $name, 'type' => 'dir', 'size' => 0, 'mtime' => null, 'files' => 0];
|
||||||
|
}
|
||||||
|
}
|
||||||
ksort($children, SORT_STRING);
|
ksort($children, SORT_STRING);
|
||||||
return array_values($children);
|
return array_values($children);
|
||||||
}
|
}
|
||||||
@@ -123,7 +130,17 @@ final class RestoreService {
|
|||||||
$entries[substr($e['p'], strlen($source) + 1)] = $e;
|
$entries[substr($e['p'], strlen($source) + 1)] = $e;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if ($entries === []) {
|
// Folders without anything in them, relative to $source ('' when $source itself is one).
|
||||||
|
/** @var array<array-key, int> $emptyDirs modification times; a folder called "2026" is an integer key */
|
||||||
|
$emptyDirs = [];
|
||||||
|
foreach ($single ? [] : $repo->emptyDirs($snapshotId) as [$dir, $mtime]) {
|
||||||
|
if ($dir === $source) {
|
||||||
|
$emptyDirs[''] = $mtime;
|
||||||
|
} elseif (str_starts_with($dir, $source . '/')) {
|
||||||
|
$emptyDirs[substr($dir, strlen($source) + 1)] = $mtime;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($entries === [] && $emptyDirs === []) {
|
||||||
throw new NotFoundException("Nothing at $source in snapshot $snapshotId");
|
throw new NotFoundException("Nothing at $source in snapshot $snapshotId");
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -134,9 +151,21 @@ final class RestoreService {
|
|||||||
} elseif ($mode !== self::MODE_MERGE && $mode !== self::MODE_REPLACE) {
|
} elseif ($mode !== self::MODE_MERGE && $mode !== self::MODE_REPLACE) {
|
||||||
throw new \InvalidArgumentException("Unknown mode $mode");
|
throw new \InvalidArgumentException("Unknown mode $mode");
|
||||||
}
|
}
|
||||||
|
$destExisted = $parentRel === '' || $userFolder->nodeExists($parentRel);
|
||||||
$dest = $this->ensureFolder($userFolder, $parentRel);
|
$dest = $this->ensureFolder($userFolder, $parentRel);
|
||||||
$names = $single ? [basename($rel) => $entries['']] : $entries;
|
$names = $single ? [basename($rel) => $entries['']] : $entries;
|
||||||
|
|
||||||
|
// The folders with something in them that this restore creates get their time back; one that
|
||||||
|
// was already there keeps its own, as its content changes now. Noted before anything is written.
|
||||||
|
/** @var array<array-key, int> $newFolders a folder called "2026" is an integer key */
|
||||||
|
$newFolders = [];
|
||||||
|
foreach ($single ? [] : $repo->folderTimes($snapshotId) as [$dir, $mtime]) {
|
||||||
|
$relative = $dir === $source ? '' : (str_starts_with($dir, $source . '/') ? substr($dir, strlen($source) + 1) : null);
|
||||||
|
if ($relative !== null && ($relative === '' ? !$destExisted : !$dest->nodeExists($relative))) {
|
||||||
|
$newFolders[$relative] = $mtime;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
$this->cleaned = [];
|
$this->cleaned = [];
|
||||||
$restored = 0;
|
$restored = 0;
|
||||||
foreach ($names as $relPath => $entry) {
|
foreach ($names as $relPath => $entry) {
|
||||||
@@ -156,9 +185,28 @@ final class RestoreService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
foreach ($emptyDirs as $dir => $mtime) {
|
||||||
|
$dir = (string)$dir;
|
||||||
|
$existed = $dir === '' ? $destExisted : $dest->nodeExists($dir); // one that is there keeps its own time
|
||||||
|
$folder = $this->ensureFolder($dest, $dir);
|
||||||
|
if (!$existed) {
|
||||||
|
$folder->touch($mtime);
|
||||||
|
}
|
||||||
|
if ($progress !== null) {
|
||||||
|
$progress($restored, count($names), $dir); // keeps the lease alive too
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
$trashed = 0;
|
$trashed = 0;
|
||||||
if ($mode === self::MODE_REPLACE && !$single) {
|
if ($mode === self::MODE_REPLACE && !$single) {
|
||||||
$trashed = $this->trashExtras($dest, '', array_flip(array_keys($names)));
|
$trashed = $this->trashExtras($dest, '', array_flip(array_keys($names)) + $emptyDirs);
|
||||||
|
}
|
||||||
|
// Last and deepest first: a change in a folder also touches the folders above it.
|
||||||
|
foreach (array_reverse($newFolders, true) as $dir => $mtime) {
|
||||||
|
$this->ensureFolder($dest, (string)$dir)->touch($mtime);
|
||||||
|
if ($progress !== null) {
|
||||||
|
$progress($restored, count($names), (string)$dir); // keeps the lease alive too
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
|
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
|
||||||
}
|
}
|
||||||
@@ -178,7 +226,7 @@ final class RestoreService {
|
|||||||
throw new \InvalidArgumentException('Invalid path: ' . $prefix);
|
throw new \InvalidArgumentException('Invalid path: ' . $prefix);
|
||||||
}
|
}
|
||||||
if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) {
|
if (!is_dir($directory) && !mkdir($directory, 0750, true) && !is_dir($directory)) {
|
||||||
throw new \RuntimeException("Cannot create $directory");
|
throw new \OCA\NgBackup\Repository\RestoreTargetException("Cannot create $directory (disk full, or no permission?)");
|
||||||
}
|
}
|
||||||
return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $prefix, $directory, $stripPrefix, $heartbeat, &$skipped, $keepUnchanged): int {
|
return $this->withSharedLock($target, function (callable $refresh) use ($target, $snapshotId, $prefix, $directory, $stripPrefix, $heartbeat, &$skipped, $keepUnchanged): int {
|
||||||
$own = LeaseService::throttled($refresh);
|
$own = LeaseService::throttled($refresh);
|
||||||
@@ -262,7 +310,7 @@ final class RestoreService {
|
|||||||
foreach ($folder->getDirectoryListing() as $node) {
|
foreach ($folder->getDirectoryListing() as $node) {
|
||||||
$rel = $prefix === '' ? $node->getName() : $prefix . '/' . $node->getName();
|
$rel = $prefix === '' ? $node->getName() : $prefix . '/' . $node->getName();
|
||||||
if ($node instanceof Folder) {
|
if ($node instanceof Folder) {
|
||||||
$hasKept = false;
|
$hasKept = isset($keep[$rel]); // a folder that was empty in the snapshot
|
||||||
foreach (array_keys($keep) as $k) {
|
foreach (array_keys($keep) as $k) {
|
||||||
if (str_starts_with((string)$k, $rel . '/')) {
|
if (str_starts_with((string)$k, $rel . '/')) {
|
||||||
$hasKept = true;
|
$hasKept = true;
|
||||||
|
|||||||
@@ -28,6 +28,19 @@ final class RetentionPolicy {
|
|||||||
return ['last' => $this->last, 'daily' => $this->daily, 'weekly' => $this->weekly, 'monthly' => $this->monthly];
|
return ['last' => $this->last, 'daily' => $this->daily, 'weekly' => $this->weekly, 'monthly' => $this->monthly];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* keep() for the snapshots that are not pinned, plus the pinned ones. A pinned snapshot does not
|
||||||
|
* take the place of a daily, weekly or monthly one.
|
||||||
|
*
|
||||||
|
* @param array<string, int> $snapshots id => unix time
|
||||||
|
* @param list<string> $pinned
|
||||||
|
* @return array<string, list<string>>
|
||||||
|
*/
|
||||||
|
public function keepWithPinned(array $snapshots, array $pinned): array {
|
||||||
|
$pins = array_intersect_key($snapshots, array_flip($pinned));
|
||||||
|
return $this->keep(array_diff_key($snapshots, $pins)) + array_fill_keys(array_keys($pins), ['pinned']);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @param array<string, int> $snapshots id => unix time
|
* @param array<string, int> $snapshots id => unix time
|
||||||
* @return array<string, list<string>> id => reasons it is kept (ids not in the result are removed)
|
* @return array<string, list<string>> id => reasons it is kept (ids not in the result are removed)
|
||||||
|
|||||||
@@ -13,10 +13,11 @@ use OCP\IConfig;
|
|||||||
use OCP\IGroupManager;
|
use OCP\IGroupManager;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The daily backup time: "HH:MM" in a time zone. Nextcloud runs PHP in UTC, so a time without a
|
* The time zone times are shown in, and the daily time from before each location had its own
|
||||||
* zone meant UTC (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone
|
* schedule (see LocationSchedule). Nextcloud runs PHP in UTC, so a time without a zone meant UTC
|
||||||
* is converted once to the zone of the server or of an administrator, at today's offset: it then
|
* (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone is converted once
|
||||||
* shows as it effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
* to the zone of the server or of an administrator, at today's offset: it then shows as it
|
||||||
|
* effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
||||||
*/
|
*/
|
||||||
final class Schedule {
|
final class Schedule {
|
||||||
public const TIME_KEY = 'schedule_time';
|
public const TIME_KEY = 'schedule_time';
|
||||||
@@ -44,27 +45,9 @@ final class Schedule {
|
|||||||
return ['time' => $time, 'timezone' => $zone];
|
return ['time' => $time, 'timezone' => $zone];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** @throws \InvalidArgumentException for a malformed time or an unknown zone */
|
/** @return non-empty-string the zone of a location's schedule, or else this one */
|
||||||
public function set(string $time, string $zone): void {
|
public function zoneOr(?string $zone): string {
|
||||||
if ($time !== '' && !self::validTime($time)) {
|
return $zone !== null && self::validZone($zone) ? $zone : $this->get()['timezone'];
|
||||||
throw new \InvalidArgumentException('Use HH:MM or empty');
|
|
||||||
}
|
|
||||||
if (!self::validZone($zone)) {
|
|
||||||
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
|
||||||
}
|
|
||||||
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
|
|
||||||
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Today's scheduled moment as a Unix time, in the schedule's zone; null without a schedule. */
|
|
||||||
public function todayAt(?int $now = null): ?int {
|
|
||||||
['time' => $time, 'timezone' => $zone] = $this->get();
|
|
||||||
if ($time === '' || !self::validTime($time)) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
$tz = new \DateTimeZone($zone);
|
|
||||||
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone($tz)->format('Y-m-d');
|
|
||||||
return (new \DateTimeImmutable($day . ' ' . $time, $tz))->getTimestamp();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
@@ -27,6 +27,9 @@ use OCP\Server;
|
|||||||
* classes but its enabled/disabled state is left as the admin set it.
|
* classes but its enabled/disabled state is left as the admin set it.
|
||||||
*/
|
*/
|
||||||
final class TargetService {
|
final class TargetService {
|
||||||
|
/** Deletion delay taken when a location's own cannot be read: nothing leaves the Bin sooner than on the old server. */
|
||||||
|
public const LONGEST_DELETE_DELAY = 365;
|
||||||
|
|
||||||
public function __construct(
|
public function __construct(
|
||||||
private TargetMapper $mapper,
|
private TargetMapper $mapper,
|
||||||
private KeyService $keys,
|
private KeyService $keys,
|
||||||
@@ -35,6 +38,8 @@ final class TargetService {
|
|||||||
private \OCA\NgBackup\Db\DbIndexCache $indexCache,
|
private \OCA\NgBackup\Db\DbIndexCache $indexCache,
|
||||||
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
|
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
|
||||||
private SnapshotMapper $snapshots,
|
private SnapshotMapper $snapshots,
|
||||||
|
private AlertService $alerts,
|
||||||
|
private \OCA\NgBackup\Db\RunMapper $runs,
|
||||||
) {
|
) {
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -51,11 +56,15 @@ final class TargetService {
|
|||||||
* @param ?callable(int, int): void $progress reading an existing repository's index: files done, and in all
|
* @param ?callable(int, int): void $progress reading an existing repository's index: files done, and in all
|
||||||
* @return array{target: Target, created: bool}
|
* @return array{target: Target, created: bool}
|
||||||
*/
|
*/
|
||||||
public function add(string $name, string $backend, string $auth, array $options, string $basePath, ?callable $progress = null): array {
|
public function add(string $name, string $backend, string $auth, array $options, string $basePath, ?callable $progress = null, bool $repairConfig = false): array {
|
||||||
$this->validateName($name);
|
$this->validateName($name);
|
||||||
$this->requireUsableBackend($backend);
|
$this->requireUsableBackend($backend);
|
||||||
$keys = $this->keys->keyRing();
|
$keys = $this->keys->keyRing();
|
||||||
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath);
|
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath, !$repairConfig);
|
||||||
|
try {
|
||||||
|
if ($repairConfig) {
|
||||||
|
Repository::repairConfig($storageBackend, $keys, $this->keys->wrappedKey());
|
||||||
|
}
|
||||||
if ($storageBackend->exists('config')) {
|
if ($storageBackend->exists('config')) {
|
||||||
$repo = Repository::openWithKey($storageBackend, $keys, $this->indexCache, $this->anchor, $progress);
|
$repo = Repository::openWithKey($storageBackend, $keys, $this->indexCache, $this->anchor, $progress);
|
||||||
$created = false;
|
$created = false;
|
||||||
@@ -63,6 +72,15 @@ final class TargetService {
|
|||||||
$repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey(), $this->anchor);
|
$repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey(), $this->anchor);
|
||||||
$created = true;
|
$created = true;
|
||||||
}
|
}
|
||||||
|
} catch (\OCA\NgBackup\Repository\RepositoryException $e) {
|
||||||
|
$otherKey = str_contains($e->getMessage(), 'different key') || str_contains($e->getMessage(), 'do not open with this key');
|
||||||
|
if ($otherKey && $this->keyIsUnused()) {
|
||||||
|
throw new \OCA\NgBackup\Repository\RepositoryException(rtrim($e->getMessage(), '.') . '. ' . self::UNUSED_KEY_HINT, 0, $e);
|
||||||
|
}
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
|
$this->keys->markUsed();
|
||||||
|
$this->recordSettings($repo);
|
||||||
|
|
||||||
foreach ($this->mapper->findAll() as $existing) {
|
foreach ($this->mapper->findAll() as $existing) {
|
||||||
if ($existing->getRepositoryId() === $repo->id()) {
|
if ($existing->getRepositoryId() === $repo->id()) {
|
||||||
@@ -78,6 +96,11 @@ final class TargetService {
|
|||||||
$target->setBasePath(trim($basePath, '/'));
|
$target->setBasePath(trim($basePath, '/'));
|
||||||
$target->setRepositoryId($repo->id());
|
$target->setRepositoryId($repo->id());
|
||||||
$target->setCreatedAt(time());
|
$target->setCreatedAt(time());
|
||||||
|
// Like the location before it: a backup scheme is usually one scheme.
|
||||||
|
$first = $this->mapper->findAll()[0] ?? null;
|
||||||
|
$target->setSchedule($first?->getSchedule() ?? '');
|
||||||
|
$target->setScheduleZone($first?->getScheduleZone());
|
||||||
|
$target->setScheduleSince(time());
|
||||||
$target = $this->mapper->insert($target);
|
$target = $this->mapper->insert($target);
|
||||||
if (!$created) {
|
if (!$created) {
|
||||||
// Disaster recovery: occ backup:list reads this local cache, not the repository
|
// Disaster recovery: occ backup:list reads this local cache, not the repository
|
||||||
@@ -87,6 +110,128 @@ final class TargetService {
|
|||||||
return ['target' => $target, 'created' => $created];
|
return ['target' => $target, 'created' => $created];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public const UNUSED_KEY_HINT = 'The backup key of this server was never used (no location, no backup), so it can make way for the key of these backups: '
|
||||||
|
. 'with the old server\'s recovery kit, occ backup:key:import <kit file> --replace-unused-key; without the kit, '
|
||||||
|
. 'occ backup:target:add with the same details and --replace-unused-key (asks for a passphrase of the old server)';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A key that never protected anything: no location was created or adopted with it, and the
|
||||||
|
* database has no location and no run (installations from before the flag existed). Only such
|
||||||
|
* a key may make way for the key of existing backups, e.g. after a new key was made by
|
||||||
|
* mistake on a server that is being restored.
|
||||||
|
*/
|
||||||
|
public function keyIsUnused(): bool {
|
||||||
|
return $this->keys->isInitialized() && !$this->keys->wasUsed()
|
||||||
|
&& $this->mapper->findAll() === [] && $this->runs->countAll() === 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* occ backup:key:import on a server whose own key was never used: the kit's key takes its place.
|
||||||
|
*
|
||||||
|
* @param array<string, mixed> $kit
|
||||||
|
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase: nothing changed
|
||||||
|
*/
|
||||||
|
public function importKitReplacingUnusedKey(array $kit, #[\SensitiveParameter] string $passphrase): void {
|
||||||
|
$old = $this->requireUnusedKey();
|
||||||
|
$this->keys->importRecoveryKit($kit, $passphrase, true);
|
||||||
|
$this->alerts->securityEvent('key_replaced', ['old' => $old, 'new' => $this->keys->fingerprint()]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return string the fingerprint of the unused key */
|
||||||
|
private function requireUnusedKey(): string {
|
||||||
|
if (!$this->keys->isInitialized()) {
|
||||||
|
throw new \RuntimeException('There is no backup key here to replace');
|
||||||
|
}
|
||||||
|
if (!$this->keyIsUnused()) {
|
||||||
|
throw new \RuntimeException('The backup key of this server (fingerprint ' . $this->keys->fingerprint() . ') has been used for a backup location: it is never replaced');
|
||||||
|
}
|
||||||
|
return $this->keys->fingerprint();
|
||||||
|
}
|
||||||
|
|
||||||
|
public const LOCATION_REPOSITORY = 'repository';
|
||||||
|
public const LOCATION_DATA_WITHOUT_CONFIG = 'data-without-config';
|
||||||
|
public const LOCATION_EMPTY = 'empty';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What these settings lead to: a repository, backup data whose config is gone, or nothing
|
||||||
|
* of NG Backup. Creates nothing.
|
||||||
|
*
|
||||||
|
* @return self::LOCATION_*
|
||||||
|
*/
|
||||||
|
public function locationState(string $backend, string $auth, array $options, string $basePath): string {
|
||||||
|
$this->requireUsableBackend($backend);
|
||||||
|
$storage = $this->buildBackend($backend, $auth, $options, $basePath, false);
|
||||||
|
if ($storage->exists('config')) {
|
||||||
|
return self::LOCATION_REPOSITORY;
|
||||||
|
}
|
||||||
|
return Repository::holdsData($storage) ? self::LOCATION_DATA_WITHOUT_CONFIG : self::LOCATION_EMPTY;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Disaster recovery without the recovery kit, on an installation without a backup key: take
|
||||||
|
* the key from the location's config with one of its passphrases, then add the location as
|
||||||
|
* usual. The key is only kept when it really is the one the repository was made with.
|
||||||
|
* With $replaceUnusedKey, a key this server has but never used makes way (keyIsUnused()).
|
||||||
|
*
|
||||||
|
* @param array<string, mixed> $options
|
||||||
|
* @param ?callable(int, int): void $progress
|
||||||
|
* @return array{target: Target, created: bool, deleteDelayDays: ?int, settingsUnreadable: bool} the deletion delay the location had stored, if any, and whether its settings could not be read
|
||||||
|
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase
|
||||||
|
*/
|
||||||
|
public function addWithPassphrase(string $name, string $backend, string $auth, array $options, string $basePath,
|
||||||
|
#[\SensitiveParameter] string $passphrase, ?callable $progress = null, bool $replaceUnusedKey = false): array {
|
||||||
|
$old = null;
|
||||||
|
if ($replaceUnusedKey) {
|
||||||
|
$old = $this->requireUnusedKey();
|
||||||
|
} elseif ($this->keys->isInitialized()) {
|
||||||
|
throw new \RuntimeException('A backup key already exists here');
|
||||||
|
}
|
||||||
|
$this->validateName($name);
|
||||||
|
$this->requireUsableBackend($backend);
|
||||||
|
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath, false);
|
||||||
|
if (!$storageBackend->exists('config')) {
|
||||||
|
throw new \InvalidArgumentException("No NG Backup repository found in folder '" . trim($basePath, '/') . "'");
|
||||||
|
}
|
||||||
|
$wrapped = Repository::wrappedKeyAt($storageBackend);
|
||||||
|
$keys = \OCA\NgBackup\Crypto\KeyRing::unwrapAny($wrapped, $passphrase);
|
||||||
|
if (!Repository::madeWithKey($storageBackend, $keys)) {
|
||||||
|
throw new \RuntimeException('The key in the config at this location does not open its backups: the config is damaged or was replaced. Only the recovery kit can help now: occ backup:key:import <kit file>, then this command again.');
|
||||||
|
}
|
||||||
|
$settings = Repository::settingsAt($storageBackend, $keys);
|
||||||
|
$stored = $settings['delete_delay_days'] ?? null;
|
||||||
|
$delay = is_int($stored) && $stored >= 1 && $stored <= 365 ? $stored : null;
|
||||||
|
// Settings that are there but cannot be read, or hold no valid delay (also none at all): the old server's delay
|
||||||
|
// is unknown, so take the longest one; a shorter guess could empty the Bin too soon.
|
||||||
|
$unknown = ($settings['unreadable'] ?? false) === true || ($settings !== [] && $delay === null);
|
||||||
|
$this->keys->adoptKey($keys, $wrapped, $delay ?? ($unknown ? self::LONGEST_DELETE_DELAY : null), $replaceUnusedKey);
|
||||||
|
if ($old !== null) {
|
||||||
|
$this->alerts->securityEvent('key_replaced', ['old' => $old, 'new' => $this->keys->fingerprint()]);
|
||||||
|
}
|
||||||
|
return $this->add($name, $backend, $auth, $options, $basePath, $progress) + ['deleteDelayDays' => $delay, 'settingsUnreadable' => $unknown];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A passphrase change that missed this location (offline at the time) reaches it now. A failure is tried again at the next backup. */
|
||||||
|
public function catchUpWrappedKey(Target $target, Repository $repo): void {
|
||||||
|
if (!$this->keys->wrappedKeyPending($target->getId())) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
$repo->updateWrappedKey($this->keys->wrappedKey());
|
||||||
|
$this->keys->setWrappedKeyPending($target->getId(), false);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: the last passphrase change still does not reach ' . $target->getName()
|
||||||
|
. ' (it keeps the previous passphrases); tried again at its next backup: ' . $e->getMessage(), ['exception' => $e]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Seal the settings fixed at installation on the location, once. A failure is tried again at the next backup. */
|
||||||
|
public function recordSettings(Repository $repo): void {
|
||||||
|
try {
|
||||||
|
$repo->recordSettings(['delete_delay_days' => $this->keys->deleteDelayDays()]);
|
||||||
|
} catch (\Throwable) {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Change how an existing location is reached (credentials, host, folder, even the connection
|
* Change how an existing location is reached (credentials, host, folder, even the connection
|
||||||
* type) or its name. The new settings must lead to the same repository: the history stays
|
* type) or its name. The new settings must lead to the same repository: the history stays
|
||||||
@@ -208,7 +353,25 @@ final class TargetService {
|
|||||||
return $result;
|
return $result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Update the local copy of a snapshot's note or pin, if the snapshot is listed here at all. */
|
||||||
|
public function syncSnapshot(Target $target, string $snapshotId, ?string $note = null, ?bool $pinned = null): void {
|
||||||
|
try {
|
||||||
|
$s = $this->snapshots->findOne($target->getId(), $snapshotId);
|
||||||
|
} catch (DoesNotExistException) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if ($note !== null) {
|
||||||
|
$s->setNote($note !== '' ? $note : null);
|
||||||
|
}
|
||||||
|
if ($pinned !== null) {
|
||||||
|
$s->setPinned($pinned);
|
||||||
|
}
|
||||||
|
$this->snapshots->update($s);
|
||||||
|
}
|
||||||
|
|
||||||
private function adoptSnapshots(Target $target, Repository $repo): void {
|
private function adoptSnapshots(Target $target, Repository $repo): void {
|
||||||
|
$notes = $repo->snapshotNotes();
|
||||||
|
$pinned = array_flip($repo->pinnedSnapshots());
|
||||||
foreach ($repo->listSnapshots() as $snapshotId) {
|
foreach ($repo->listSnapshots() as $snapshotId) {
|
||||||
try {
|
try {
|
||||||
$this->snapshots->findOne($target->getId(), $snapshotId);
|
$this->snapshots->findOne($target->getId(), $snapshotId);
|
||||||
@@ -227,6 +390,8 @@ final class TargetService {
|
|||||||
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
||||||
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
||||||
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
||||||
|
$s->setNote($notes[$snapshotId] ?? null);
|
||||||
|
$s->setPinned(isset($pinned[$snapshotId]));
|
||||||
$this->snapshots->insert($s);
|
$this->snapshots->insert($s);
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: snapshot ' . $snapshotId . ' on ' . $target->getName() . ' not listed: ' . $e->getMessage(), ['exception' => $e]);
|
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: snapshot ' . $snapshotId . ' on ' . $target->getName() . ' not listed: ' . $e->getMessage(), ['exception' => $e]);
|
||||||
@@ -248,6 +413,30 @@ final class TargetService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/** Size limit for the data stored on this location (null = none). */
|
/** Size limit for the data stored on this location (null = none). */
|
||||||
|
/**
|
||||||
|
* @param string $schedule see LocationSchedule
|
||||||
|
* @throws \InvalidArgumentException
|
||||||
|
*/
|
||||||
|
public function setSchedule(Target $target, string $schedule, string $zone): Target {
|
||||||
|
if (!Schedule::validZone($zone)) {
|
||||||
|
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
||||||
|
}
|
||||||
|
$new = LocationSchedule::normalize($schedule);
|
||||||
|
$turnedOff = $new === '' && (string)$target->getSchedule() !== '';
|
||||||
|
if ($new === (string)$target->getSchedule() && $zone === $target->getScheduleZone()) {
|
||||||
|
// Saved again unchanged: a moment that already passed still gets its backup
|
||||||
|
return $target;
|
||||||
|
}
|
||||||
|
$target->setSchedule($new);
|
||||||
|
$target->setScheduleZone($zone);
|
||||||
|
$target->setScheduleSince(time());
|
||||||
|
$target = $this->mapper->update($target);
|
||||||
|
if ($turnedOff) {
|
||||||
|
$this->alerts->securityEvent('schedule_off', ['target' => $target->getName()]);
|
||||||
|
}
|
||||||
|
return $target;
|
||||||
|
}
|
||||||
|
|
||||||
public function setMaxBytes(Target $target, ?int $maxBytes): Target {
|
public function setMaxBytes(Target $target, ?int $maxBytes): Target {
|
||||||
$target->setMaxBytes($maxBytes);
|
$target->setMaxBytes($maxBytes);
|
||||||
return $this->mapper->update($target);
|
return $this->mapper->update($target);
|
||||||
@@ -261,11 +450,30 @@ final class TargetService {
|
|||||||
/** Remove the target from NG Backup. The backups on the target itself are not touched. */
|
/** Remove the target from NG Backup. The backups on the target itself are not touched. */
|
||||||
public function remove(Target $target): void {
|
public function remove(Target $target): void {
|
||||||
$this->mapper->delete($target);
|
$this->mapper->delete($target);
|
||||||
|
// The location is gone now: tidying up must not fail the removal, or the caller would skip
|
||||||
|
// the removal event and a retry would no longer find the location
|
||||||
|
$this->tidyUp($target, fn () => $this->keys->setWrappedKeyPending($target->getId(), false));
|
||||||
// The cached index of its repository is only of use while a location points at it
|
// The cached index of its repository is only of use while a location points at it
|
||||||
|
$this->tidyUp($target, function () use ($target): void {
|
||||||
$repositoryId = $target->getRepositoryId();
|
$repositoryId = $target->getRepositoryId();
|
||||||
if ($repositoryId !== null && !array_filter($this->list(), fn (Target $t) => $t->getRepositoryId() === $repositoryId)) {
|
if ($repositoryId !== null && !array_filter($this->list(), fn (Target $t) => $t->getRepositoryId() === $repositoryId)) {
|
||||||
$this->indexCache->forget($repositoryId);
|
$this->indexCache->forget($repositoryId);
|
||||||
}
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One tidy-up step after a removal: a failure is logged (if even that works) and never thrown. */
|
||||||
|
private function tidyUp(Target $target, callable $step): void {
|
||||||
|
try {
|
||||||
|
$step();
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
try {
|
||||||
|
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: location ' . $target->getName()
|
||||||
|
. ' removed, tidying up after it failed: ' . $e->getMessage(), ['exception' => $e]);
|
||||||
|
} catch (\Throwable) {
|
||||||
|
// nothing left to tell it to
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public function backend(Target $target): IBackend {
|
public function backend(Target $target): IBackend {
|
||||||
|
|||||||
@@ -186,6 +186,7 @@
|
|||||||
<code><![CDATA[$this->db]]></code>
|
<code><![CDATA[$this->db]]></code>
|
||||||
<code><![CDATA[$this->db]]></code>
|
<code><![CDATA[$this->db]]></code>
|
||||||
<code><![CDATA[$this->db]]></code>
|
<code><![CDATA[$this->db]]></code>
|
||||||
|
<code><![CDATA[$this->db]]></code>
|
||||||
<code><![CDATA[TransactionIsolationLevel]]></code>
|
<code><![CDATA[TransactionIsolationLevel]]></code>
|
||||||
<code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
|
<code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
|
||||||
<code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
|
<code><![CDATA[\Doctrine\DBAL\ParameterType]]></code>
|
||||||
@@ -299,6 +300,22 @@
|
|||||||
<code><![CDATA[$t]]></code>
|
<code><![CDATA[$t]]></code>
|
||||||
</UndefinedDocblockClass>
|
</UndefinedDocblockClass>
|
||||||
</file>
|
</file>
|
||||||
|
<file src="lib/Migration/Version000008Date20261107010000.php">
|
||||||
|
<UndefinedDocblockClass>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
</UndefinedDocblockClass>
|
||||||
|
</file>
|
||||||
|
<file src="lib/Migration/Version000007Date20261107000000.php">
|
||||||
|
<UndefinedDocblockClass>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
<code><![CDATA[$t]]></code>
|
||||||
|
</UndefinedDocblockClass>
|
||||||
|
</file>
|
||||||
<file src="lib/Migration/Version000005Date20261002040000.php">
|
<file src="lib/Migration/Version000005Date20261002040000.php">
|
||||||
<UndefinedDocblockClass>
|
<UndefinedDocblockClass>
|
||||||
<code><![CDATA[$t]]></code>
|
<code><![CDATA[$t]]></code>
|
||||||
@@ -350,7 +367,6 @@
|
|||||||
<code><![CDATA[$encrypted]]></code>
|
<code><![CDATA[$encrypted]]></code>
|
||||||
<code><![CDATA[$encrypted]]></code>
|
<code><![CDATA[$encrypted]]></code>
|
||||||
<code><![CDATA[$encrypted]]></code>
|
<code><![CDATA[$encrypted]]></code>
|
||||||
<code><![CDATA[$fh]]></code>
|
|
||||||
<code><![CDATA[$plain]]></code>
|
<code><![CDATA[$plain]]></code>
|
||||||
<code><![CDATA[$plain]]></code>
|
<code><![CDATA[$plain]]></code>
|
||||||
<code><![CDATA[$plain]]></code>
|
<code><![CDATA[$plain]]></code>
|
||||||
|
|||||||
|
Before Width: | Height: | Size: 259 KiB After Width: | Height: | Size: 244 KiB |
|
Before Width: | Height: | Size: 268 KiB After Width: | Height: | Size: 228 KiB |
|
Before Width: | Height: | Size: 252 KiB After Width: | Height: | Size: 222 KiB |
|
Before Width: | Height: | Size: 189 KiB After Width: | Height: | Size: 172 KiB |
@@ -21,7 +21,7 @@ export default {
|
|||||||
addSlot: (passphrase, label) => call('post', '/key/slots', { passphrase, label }),
|
addSlot: (passphrase, label) => call('post', '/key/slots', { passphrase, label }),
|
||||||
replaceSlot: (slot, passphrase, label) => call('put', `/key/slots/${slot}`, { passphrase, label }),
|
replaceSlot: (slot, passphrase, label) => call('put', `/key/slots/${slot}`, { passphrase, label }),
|
||||||
removeSlot: (slot) => call('delete', `/key/slots/${slot}`),
|
removeSlot: (slot) => call('delete', `/key/slots/${slot}`),
|
||||||
trash: (targetId) => call('get', `/targets/${targetId}/trash`),
|
trash: (targetId, after = '') => call('get', `/targets/${targetId}/trash`, { after }),
|
||||||
untrash: (targetId, snapshotId) => call('post', `/targets/${targetId}/trash/${snapshotId}`),
|
untrash: (targetId, snapshotId) => call('post', `/targets/${targetId}/trash/${snapshotId}`),
|
||||||
kitUrl: () => url('/key/kit'),
|
kitUrl: () => url('/key/kit'),
|
||||||
confirmKit: (accepted, phrase, code) => call('post', '/key/confirm', { accepted, phrase, code }),
|
confirmKit: (accepted, phrase, code) => call('post', '/key/confirm', { accepted, phrase, code }),
|
||||||
@@ -34,8 +34,11 @@ export default {
|
|||||||
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
|
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
|
||||||
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
|
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
|
||||||
tick: () => call('post', '/tick'),
|
tick: () => call('post', '/tick'),
|
||||||
setSchedule: (time, timezone) => call('put', '/schedule', { time, timezone }),
|
setPauseHeavyJobs: (enabled) => call('put', '/settings/pause-heavy-jobs', { enabled }),
|
||||||
|
setTargetSchedule: (id, schedule, timezone) => call('put', `/targets/${id}/schedule`, { schedule, timezone }),
|
||||||
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
|
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
|
||||||
|
setSnapshotNote: (targetId, snapshotId, note) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/note`, { note }),
|
||||||
|
setSnapshotPinned: (targetId, snapshotId, pinned) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/pin`, { pinned }),
|
||||||
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
|
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
|
||||||
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
|
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,10 +5,11 @@
|
|||||||
<template>
|
<template>
|
||||||
<NcSettingsSection :name="t('ng_backup', 'Backups')"
|
<NcSettingsSection :name="t('ng_backup', 'Backups')"
|
||||||
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
|
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
|
||||||
<div class="ngb-row">
|
<LocationScheduleRow v-for="target in status.targets" :key="target.id" :target="target" @changed="emit('changed')" @error="error = $event" />
|
||||||
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, {zone}, empty = off)', { zone: browserZone })" />
|
<NcCheckboxRadioSwitch :model-value="status.heavyJobs.enabled" @update:model-value="savePause">
|
||||||
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
{{ t('ng_backup', 'Pause heavy background jobs (Memories, Recognize, Preview Generator, full text search, antivirus scan) while a backup runs, at most 6 hours in a row') }}
|
||||||
</div>
|
</NcCheckboxRadioSwitch>
|
||||||
|
<p v-if="status.heavyJobs.jobs.length" class="ngb-muted">{{ t('ng_backup', 'Paused now: {jobs}', { jobs: status.heavyJobs.jobs.join(', ') }) }}</p>
|
||||||
<div class="ngb-row">
|
<div class="ngb-row">
|
||||||
<NcTextField v-model="label" class="ngb-label" :maxlength="100" :label="t('ng_backup', 'Label (optional)')"
|
<NcTextField v-model="label" class="ngb-label" :maxlength="100" :label="t('ng_backup', 'Label (optional)')"
|
||||||
:placeholder="t('ng_backup', 'e.g. before the upgrade to Nextcloud 35')" />
|
:placeholder="t('ng_backup', 'e.g. before the upgrade to Nextcloud 35')" />
|
||||||
@@ -37,11 +38,11 @@
|
|||||||
<td>{{ formatDateTime(run.startedAt) }}</td>
|
<td>{{ formatDateTime(run.startedAt) }}</td>
|
||||||
<td>{{ targetName(run.targetId) }}</td>
|
<td>{{ targetName(run.targetId) }}</td>
|
||||||
<td>{{ run.kind }}</td>
|
<td>{{ run.kind }}</td>
|
||||||
<td>{{ run.snapshot ? formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}</td>
|
<td class="ngb-point">{{ run.snapshot ? (run.snapshot.pinned ? '📌 ' : '') + formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') + (run.snapshot.note ? ' – ' + run.snapshot.note : '') : '' }}</td>
|
||||||
<td :class="'ngb-' + run.status">{{ runStatus(run) }}{{ run.error ? ': ' + run.error : '' }}</td>
|
<td :class="'ngb-' + run.status">{{ runStatus(run) }}{{ run.error ? ': ' + run.error : '' }}</td>
|
||||||
<td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
|
<td class="ngb-num">{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
|
||||||
<td>{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td>
|
<td class="ngb-num">{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td>
|
||||||
<td>{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}</td>
|
<td class="ngb-num">{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}</td>
|
||||||
</tr>
|
</tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
@@ -60,24 +61,20 @@
|
|||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup>
|
<script setup>
|
||||||
import { ref, watch } from 'vue'
|
import { ref } from 'vue'
|
||||||
import { n, t } from '@nextcloud/l10n'
|
import { n, t } from '@nextcloud/l10n'
|
||||||
import NcButton from '@nextcloud/vue/components/NcButton'
|
import NcButton from '@nextcloud/vue/components/NcButton'
|
||||||
|
import NcCheckboxRadioSwitch from '@nextcloud/vue/components/NcCheckboxRadioSwitch'
|
||||||
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||||
|
import LocationScheduleRow from './LocationScheduleRow.vue'
|
||||||
import api from '../api.js'
|
import api from '../api.js'
|
||||||
import { formatDateTime, formatSize, toZone } from '../format.js'
|
import { formatDateTime, formatSize } from '../format.js'
|
||||||
|
|
||||||
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
|
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
|
||||||
const emit = defineEmits(['changed', 'show-all-runs'])
|
const emit = defineEmits(['changed', 'show-all-runs'])
|
||||||
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
|
|
||||||
// zone is shown in this one, so it reads as the moment it runs here.
|
|
||||||
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
|
||||||
const shown = (s) => (s && props.status.scheduleTimezone && props.status.scheduleTimezone !== browserZone) ? toZone(s, props.status.scheduleTimezone, browserZone) : s
|
|
||||||
const schedule = ref(shown(props.status.schedule))
|
|
||||||
const error = ref('')
|
const error = ref('')
|
||||||
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
|
|
||||||
|
|
||||||
const label = ref('')
|
const label = ref('')
|
||||||
|
|
||||||
@@ -97,6 +94,9 @@ const runStatus = (run) => {
|
|||||||
if (run.status !== 'running') {
|
if (run.status !== 'running') {
|
||||||
return run.status
|
return run.status
|
||||||
}
|
}
|
||||||
|
if (run.queued) {
|
||||||
|
return t('ng_backup', 'queued, after the backup that is running now')
|
||||||
|
}
|
||||||
const detail = run.phase + percent(run)
|
const detail = run.phase + percent(run)
|
||||||
if (run.waitingFor) {
|
if (run.waitingFor) {
|
||||||
return t('ng_backup', 'waiting for background jobs ({detail}, last step {minutes} min ago)', { detail, minutes: Math.floor(run.waitingFor / 60) })
|
return t('ng_backup', 'waiting for background jobs ({detail}, last step {minutes} min ago)', { detail, minutes: Math.floor(run.waitingFor / 60) })
|
||||||
@@ -106,6 +106,16 @@ const runStatus = (run) => {
|
|||||||
const isRunning = (targetId) => props.status.runs.some(r => r.targetId === targetId && r.kind === 'full' && r.status === 'running')
|
const isRunning = (targetId) => props.status.runs.some(r => r.targetId === targetId && r.kind === 'full' && r.status === 'running')
|
||||||
const targetName = (id) => props.status.targets.find(tg => tg.id === id)?.name ?? '#' + id
|
const targetName = (id) => props.status.targets.find(tg => tg.id === id)?.name ?? '#' + id
|
||||||
|
|
||||||
|
async function savePause(enabled) {
|
||||||
|
error.value = ''
|
||||||
|
try {
|
||||||
|
await api.setPauseHeavyJobs(enabled)
|
||||||
|
emit('changed')
|
||||||
|
} catch (e) {
|
||||||
|
error.value = e.message
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async function start(target) {
|
async function start(target) {
|
||||||
error.value = ''
|
error.value = ''
|
||||||
try {
|
try {
|
||||||
@@ -116,25 +126,17 @@ async function start(target) {
|
|||||||
error.value = e.message
|
error.value = e.message
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async function saveSchedule() {
|
|
||||||
error.value = ''
|
|
||||||
try {
|
|
||||||
await api.setSchedule(schedule.value.trim(), browserZone)
|
|
||||||
emit('changed')
|
|
||||||
} catch (e) {
|
|
||||||
error.value = e.message
|
|
||||||
}
|
|
||||||
}
|
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<style scoped>
|
<style scoped>
|
||||||
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
||||||
.ngb-time { max-width: 260px; }
|
|
||||||
.ngb-label { max-width: 360px; }
|
.ngb-label { max-width: 360px; }
|
||||||
.ngb-table-wrap { overflow-x: auto; }
|
.ngb-table-wrap { max-width: 90%; overflow-x: auto; } /* scrolls only on a really narrow screen */
|
||||||
.ngb-table { width: 100%; border-collapse: collapse; }
|
.ngb-table { width: 100%; border-collapse: collapse; }
|
||||||
.ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); }
|
.ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); white-space: normal; overflow-wrap: break-word; }
|
||||||
|
/* Text wraps; numbers stay on one line, the restore point keeps room for its label and note */
|
||||||
|
.ngb-table td.ngb-num { white-space: nowrap; }
|
||||||
|
.ngb-table td.ngb-point { min-width: 9em; }
|
||||||
.ngb-failed { color: var(--color-error-text); }
|
.ngb-failed { color: var(--color-error-text); }
|
||||||
.ngb-muted { color: var(--color-text-maxcontrast); }
|
.ngb-muted { color: var(--color-text-maxcontrast); }
|
||||||
.ngb-done { color: var(--color-success-text); }
|
.ngb-done { color: var(--color-success-text); }
|
||||||
|
|||||||
@@ -6,8 +6,18 @@
|
|||||||
<NcSettingsSection :name="t('ng_backup', 'Bin')"
|
<NcSettingsSection :name="t('ng_backup', 'Bin')"
|
||||||
:description="t('ng_backup', 'Restore points that were removed, most of them by the retention policy. They can be brought back until they are deleted permanently.')">
|
:description="t('ng_backup', 'Restore points that were removed, most of them by the retention policy. They can be brought back until they are deleted permanently.')">
|
||||||
<NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard>
|
<NcNoteCard v-if="error" type="error">{{ error }}</NcNoteCard>
|
||||||
<NcLoadingIcon v-if="loading" :size="32" />
|
<div v-if="reading" class="ngb-reading" role="status">
|
||||||
<NcNoteCard v-else-if="empty" type="info">{{ t('ng_backup', 'The bin is empty.') }}</NcNoteCard>
|
<template v-if="reading.total === null">
|
||||||
|
<NcLoadingIcon :size="20" />
|
||||||
|
<span>{{ t('ng_backup', 'Connecting to {location} …', { location: reading.name }) }}</span>
|
||||||
|
</template>
|
||||||
|
<template v-else>
|
||||||
|
<span>{{ t('ng_backup', 'Reading removed restore points from {location}: {done} of {total}', { location: reading.name, done: reading.done, total: reading.total }) }}</span>
|
||||||
|
<NcProgressBar :value="reading.total ? Math.round(100 * reading.done / reading.total) : 0" />
|
||||||
|
</template>
|
||||||
|
<p class="ngb-muted">{{ t('ng_backup', 'This can take a while: every removed restore point is read from the location.') }}</p>
|
||||||
|
</div>
|
||||||
|
<NcNoteCard v-else-if="empty && !error" type="info">{{ t('ng_backup', 'The bin is empty.') }}</NcNoteCard>
|
||||||
|
|
||||||
<template v-for="target in status.targets" :key="'trash' + target.id">
|
<template v-for="target in status.targets" :key="'trash' + target.id">
|
||||||
<div v-if="trash[target.id]?.length" class="ngb-trash">
|
<div v-if="trash[target.id]?.length" class="ngb-trash">
|
||||||
@@ -39,12 +49,13 @@
|
|||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script setup>
|
<script setup>
|
||||||
import { computed, reactive, ref, watch } from 'vue'
|
import { computed, onBeforeUnmount, reactive, ref, watch } from 'vue'
|
||||||
import { n, t } from '@nextcloud/l10n'
|
import { n, t } from '@nextcloud/l10n'
|
||||||
import { confirmPassword } from '@nextcloud/password-confirmation'
|
import { confirmPassword } from '@nextcloud/password-confirmation'
|
||||||
import NcButton from '@nextcloud/vue/components/NcButton'
|
import NcButton from '@nextcloud/vue/components/NcButton'
|
||||||
import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
|
import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
|
||||||
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||||
|
import NcProgressBar from '@nextcloud/vue/components/NcProgressBar'
|
||||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||||
import api from '../api.js'
|
import api from '../api.js'
|
||||||
import { formatDate, formatDateTime } from '../format.js'
|
import { formatDate, formatDateTime } from '../format.js'
|
||||||
@@ -56,21 +67,49 @@ const emit = defineEmits(['changed'])
|
|||||||
const RETENTION = 'retention policy'
|
const RETENTION = 'retention policy'
|
||||||
const retentionInfo = ref(false)
|
const retentionInfo = ref(false)
|
||||||
const trash = reactive({})
|
const trash = reactive({})
|
||||||
const loading = ref(true)
|
const reading = ref({ name: '', total: null, done: 0 }) // the location being read; null when done
|
||||||
const error = ref('')
|
const error = ref('')
|
||||||
const empty = computed(() => props.status.targets.every(target => !trash[target.id]?.length))
|
const empty = computed(() => props.status.targets.every(target => !trash[target.id]?.length))
|
||||||
|
|
||||||
|
let current = 0 // the newest load; an older one stops (another took over, or the tab was left)
|
||||||
|
onBeforeUnmount(() => { current++ })
|
||||||
|
|
||||||
|
async function loadTarget(target, load) {
|
||||||
|
reading.value = { name: target.name, total: null, done: 0 }
|
||||||
|
const items = []
|
||||||
|
for (;;) {
|
||||||
|
const page = await api.trash(target.id, items.length ? items[items.length - 1].id : '')
|
||||||
|
if (load !== current) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
items.push(...page.items)
|
||||||
|
reading.value = { name: target.name, total: page.total, done: Math.min(items.length, page.total) }
|
||||||
|
if (!page.items.length || items.length >= page.total) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
trash[target.id] = items.sort((a, b) => String(b.time ?? '').localeCompare(String(a.time ?? '')))
|
||||||
|
}
|
||||||
|
|
||||||
async function loadTrash() {
|
async function loadTrash() {
|
||||||
|
const load = ++current
|
||||||
|
const failed = []
|
||||||
for (const target of props.status.targets) {
|
for (const target of props.status.targets) {
|
||||||
try {
|
try {
|
||||||
trash[target.id] = await api.trash(target.id)
|
await loadTarget(target, load)
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
trash[target.id] = []
|
// keep what was shown: a location that cannot be reached does not mean its bin is empty
|
||||||
|
failed.push(target.name + (e?.message ? ': ' + e.message : ''))
|
||||||
|
}
|
||||||
|
if (load !== current) {
|
||||||
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
loading.value = false
|
error.value = failed.length ? t('ng_backup', 'The bin could not be read from {locations}', { locations: failed.join('; ') }) : ''
|
||||||
|
reading.value = null
|
||||||
}
|
}
|
||||||
watch(() => props.status.targets.map(tg => tg.id).join(','), loadTrash, { immediate: true })
|
// Again when a run ends: a cleanup moves restore points here.
|
||||||
|
watch(() => props.status.targets.map(tg => tg.id).join(',') + '|' + (props.status.runs ?? []).filter(run => run.status !== 'running').map(run => run.id).join(','), loadTrash, { immediate: true })
|
||||||
|
|
||||||
async function untrash(target, item) {
|
async function untrash(target, item) {
|
||||||
error.value = ''
|
error.value = ''
|
||||||
@@ -88,6 +127,8 @@ async function untrash(target, item) {
|
|||||||
<style scoped>
|
<style scoped>
|
||||||
.ngb-info { background: none; border: none; padding: 0 4px; min-height: auto; cursor: pointer; color: var(--color-primary-element); }
|
.ngb-info { background: none; border: none; padding: 0 4px; min-height: auto; cursor: pointer; color: var(--color-primary-element); }
|
||||||
.ngb-muted { color: var(--color-text-maxcontrast); }
|
.ngb-muted { color: var(--color-text-maxcontrast); }
|
||||||
|
.ngb-reading { margin: 12px 0; display: flex; flex-direction: column; gap: 6px; max-width: 600px; }
|
||||||
|
.ngb-reading p { margin: 0; }
|
||||||
.ngb-trash { margin: 12px 0; }
|
.ngb-trash { margin: 12px 0; }
|
||||||
.ngb-trash h3 { font-weight: bold; }
|
.ngb-trash h3 { font-weight: bold; }
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
@@ -4,8 +4,11 @@
|
|||||||
-->
|
-->
|
||||||
<template>
|
<template>
|
||||||
<NcSettingsSection :name="t('ng_backup', 'Backup key and recovery kit')"
|
<NcSettingsSection :name="t('ng_backup', 'Backup key and recovery kit')"
|
||||||
:description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups. To restore on a new server you need the recovery kit and one of the passphrases.')">
|
:description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups. To restore on a new server you need one of the passphrases; the recovery kit also helps when the key on the backup location is damaged.')">
|
||||||
<!-- 1. Create the key -->
|
<!-- 1. Create the key -->
|
||||||
|
<NcNoteCard v-if="!status.key.initialized" type="warning">
|
||||||
|
{{ t('ng_backup', 'Restoring a server after a crash? Do not create a key here: a new key cannot read your backups. Follow the disaster recovery guide instead (occ backup:target:add asks for one of your passphrases, or import the recovery kit with occ backup:key:import).') }}
|
||||||
|
</NcNoteCard>
|
||||||
<form v-if="!status.key.initialized" class="ngb-form" @submit.prevent="createKey">
|
<form v-if="!status.key.initialized" class="ngb-form" @submit.prevent="createKey">
|
||||||
<NcTextField v-model="label" :label="t('ng_backup', 'Who keeps this passphrase (e.g. Safe, CTO)')" />
|
<NcTextField v-model="label" :label="t('ng_backup', 'Who keeps this passphrase (e.g. Safe, CTO)')" />
|
||||||
<NcPasswordField v-model="pass1" :label="t('ng_backup', 'Passphrase (at least 16 characters)')" autocomplete="new-password" />
|
<NcPasswordField v-model="pass1" :label="t('ng_backup', 'Passphrase (at least 16 characters)')" autocomplete="new-password" />
|
||||||
|
|||||||
@@ -0,0 +1,78 @@
|
|||||||
|
<!--
|
||||||
|
- SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
- SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
-->
|
||||||
|
<template>
|
||||||
|
<div class="ngb-row">
|
||||||
|
<strong class="ngb-name">{{ target.name }}</strong>
|
||||||
|
<NcSelect v-model="kind" :options="kinds" label="label" :input-label="t('ng_backup', 'Schedule')" :clearable="false" />
|
||||||
|
<NcTextField v-if="kind.id !== 'off'" v-model="time" class="ngb-time" :label="kind.id === 'daily' ? t('ng_backup', 'At (HH:MM, {zone})', { zone }) : t('ng_backup', 'From (HH:MM, {zone})', { zone })" />
|
||||||
|
<NcTextField v-if="kind.id === 'hours'" v-model="hours" class="ngb-hours" type="number" min="1" max="23" :label="t('ng_backup', 'Every … hours')" />
|
||||||
|
<NcButton :disabled="busy || wanted === current" @click="save">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup>
|
||||||
|
import { computed, ref, watch } from 'vue'
|
||||||
|
import { t } from '@nextcloud/l10n'
|
||||||
|
import NcButton from '@nextcloud/vue/components/NcButton'
|
||||||
|
import NcSelect from '@nextcloud/vue/components/NcSelect'
|
||||||
|
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||||
|
import api from '../api.js'
|
||||||
|
|
||||||
|
const props = defineProps({ target: { type: Object, required: true } })
|
||||||
|
const emit = defineEmits(['changed', 'error'])
|
||||||
|
|
||||||
|
const kinds = [
|
||||||
|
{ id: 'off', label: t('ng_backup', 'Off') },
|
||||||
|
{ id: 'daily', label: t('ng_backup', 'Once a day') },
|
||||||
|
{ id: 'hours', label: t('ng_backup', 'Every few hours') },
|
||||||
|
]
|
||||||
|
// A schedule is shown and edited in the zone it was saved in, named next to the time: "every 6
|
||||||
|
// hours from 02:00" runs until midnight of that zone, so moving it to another zone would change
|
||||||
|
// how often it runs. A location without a schedule gets the administrator's own zone (the browser's).
|
||||||
|
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
||||||
|
const zone = computed(() => props.target.schedule !== '' && props.target.scheduleZone ? props.target.scheduleZone : browserZone)
|
||||||
|
const kind = ref(kinds[0])
|
||||||
|
const time = ref('02:00')
|
||||||
|
const hours = ref('6')
|
||||||
|
const busy = ref(false)
|
||||||
|
|
||||||
|
const current = computed(() => {
|
||||||
|
const [at, every] = props.target.schedule.split('/')
|
||||||
|
return at === '' ? '' : at + (every ? '/' + every : '')
|
||||||
|
})
|
||||||
|
const wanted = computed(() => {
|
||||||
|
if (kind.value.id === 'off') {
|
||||||
|
return ''
|
||||||
|
}
|
||||||
|
return time.value.trim() + (kind.value.id === 'hours' ? '/' + String(hours.value).trim() : '')
|
||||||
|
})
|
||||||
|
|
||||||
|
watch(current, (schedule) => {
|
||||||
|
const [at, every] = schedule.split('/')
|
||||||
|
kind.value = kinds[at === '' ? 0 : (every ? 2 : 1)]
|
||||||
|
time.value = at === '' ? time.value : at
|
||||||
|
hours.value = every ?? hours.value
|
||||||
|
}, { immediate: true })
|
||||||
|
|
||||||
|
async function save() {
|
||||||
|
busy.value = true
|
||||||
|
emit('error', '')
|
||||||
|
try {
|
||||||
|
await api.setTargetSchedule(props.target.id, wanted.value, zone.value)
|
||||||
|
emit('changed')
|
||||||
|
} catch (e) {
|
||||||
|
emit('error', e.message)
|
||||||
|
} finally {
|
||||||
|
busy.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style scoped>
|
||||||
|
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
||||||
|
.ngb-name { min-width: 120px; align-self: center; }
|
||||||
|
.ngb-time { max-width: 220px; }
|
||||||
|
.ngb-hours { max-width: 120px; }
|
||||||
|
</style>
|
||||||
@@ -11,6 +11,13 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<template v-if="snapshot">
|
<template v-if="snapshot">
|
||||||
|
<div class="ngb-row ngb-marks">
|
||||||
|
<NcTextField v-model="note" class="ngb-note" :maxlength="500" :label="t('ng_backup', 'Note (can be changed later)')" />
|
||||||
|
<NcButton :disabled="marking || note.trim() === (snapshot.note ?? '')" @click="saveNote">{{ t('ng_backup', 'Save note') }}</NcButton>
|
||||||
|
<NcCheckboxRadioSwitch :model-value="snapshot.pinned" :disabled="marking" @update:model-value="savePinned">
|
||||||
|
{{ t('ng_backup', 'Pin: the retention policy does not remove this restore point') }}
|
||||||
|
</NcCheckboxRadioSwitch>
|
||||||
|
</div>
|
||||||
<div class="ngb-crumbs">
|
<div class="ngb-crumbs">
|
||||||
<NcButton variant="tertiary" @click="open('data')">data</NcButton>
|
<NcButton variant="tertiary" @click="open('data')">data</NcButton>
|
||||||
<template v-for="(crumb, i) in crumbs" :key="crumb.path">
|
<template v-for="(crumb, i) in crumbs" :key="crumb.path">
|
||||||
@@ -49,6 +56,7 @@ import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
|
|||||||
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||||
import NcSelect from '@nextcloud/vue/components/NcSelect'
|
import NcSelect from '@nextcloud/vue/components/NcSelect'
|
||||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||||
|
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||||
import { confirmPassword } from '@nextcloud/password-confirmation'
|
import { confirmPassword } from '@nextcloud/password-confirmation'
|
||||||
import '@nextcloud/password-confirmation/style.css'
|
import '@nextcloud/password-confirmation/style.css'
|
||||||
import api from '../api.js'
|
import api from '../api.js'
|
||||||
@@ -74,7 +82,10 @@ const modes = [
|
|||||||
{ id: 'replace', label: t('ng_backup', 'Into the original place, overwrite and move files that are not in the backup to the trash') },
|
{ id: 'replace', label: t('ng_backup', 'Into the original place, overwrite and move files that are not in the backup to the trash') },
|
||||||
]
|
]
|
||||||
|
|
||||||
const snapshotLabel = (s) => formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '')
|
const note = ref('')
|
||||||
|
const marking = ref(false)
|
||||||
|
|
||||||
|
const snapshotLabel = (s) => (s.pinned ? '📌 ' : '') + formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '') + (s.note ? ' – ' + s.note : '')
|
||||||
const join = (name) => path.value + '/' + name
|
const join = (name) => path.value + '/' + name
|
||||||
const restorable = (p) => /^data\/[^/]+\/files(\/.*)?$/.test(p)
|
const restorable = (p) => /^data\/[^/]+\/files(\/.*)?$/.test(p)
|
||||||
const crumbs = computed(() => {
|
const crumbs = computed(() => {
|
||||||
@@ -88,7 +99,29 @@ watch(target, async (tg) => {
|
|||||||
snapshots.value = list.map(s => ({ ...s, displayLabel: snapshotLabel(s) }))
|
snapshots.value = list.map(s => ({ ...s, displayLabel: snapshotLabel(s) }))
|
||||||
snapshot.value = snapshots.value[0] ?? null
|
snapshot.value = snapshots.value[0] ?? null
|
||||||
}, { immediate: true })
|
}, { immediate: true })
|
||||||
watch(snapshot, () => { selected.value = ''; open('data') })
|
watch(snapshot, (s) => { selected.value = ''; note.value = s?.note ?? ''; open('data') })
|
||||||
|
|
||||||
|
// Changes the snapshot in place: choosing another one, which opens its folders again, is not needed
|
||||||
|
async function mark(call, change) {
|
||||||
|
const s = snapshot.value
|
||||||
|
marking.value = true
|
||||||
|
error.value = message.value = ''
|
||||||
|
try {
|
||||||
|
await call(target.value.id, s.id)
|
||||||
|
change(s)
|
||||||
|
s.displayLabel = snapshotLabel(s)
|
||||||
|
emit('changed')
|
||||||
|
} catch (e) {
|
||||||
|
error.value = e.message
|
||||||
|
} finally {
|
||||||
|
marking.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const saveNote = () => {
|
||||||
|
const text = note.value.trim()
|
||||||
|
return mark((targetId, id) => api.setSnapshotNote(targetId, id, text), (s) => { s.note = text })
|
||||||
|
}
|
||||||
|
const savePinned = (pinned) => mark((targetId, id) => api.setSnapshotPinned(targetId, id, pinned), (s) => { s.pinned = pinned })
|
||||||
|
|
||||||
async function open(p) {
|
async function open(p) {
|
||||||
if (!snapshot.value) {
|
if (!snapshot.value) {
|
||||||
@@ -125,6 +158,8 @@ async function restore() {
|
|||||||
<style scoped>
|
<style scoped>
|
||||||
.ngb-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 12px; }
|
.ngb-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 12px; }
|
||||||
.ngb-row :deep(.ngb-snapshot) { min-width: min(360px, 100%); }
|
.ngb-row :deep(.ngb-snapshot) { min-width: min(360px, 100%); }
|
||||||
|
.ngb-marks { align-items: flex-end; }
|
||||||
|
.ngb-note { max-width: 360px; }
|
||||||
.ngb-crumbs { display: flex; align-items: center; flex-wrap: wrap; }
|
.ngb-crumbs { display: flex; align-items: center; flex-wrap: wrap; }
|
||||||
.ngb-entries li { display: flex; align-items: center; gap: 12px; min-height: 36px; }
|
.ngb-entries li { display: flex; align-items: center; gap: 12px; min-height: 36px; }
|
||||||
.ngb-entries li.selected { background: var(--color-primary-element-light); border-radius: var(--border-radius); }
|
.ngb-entries li.selected { background: var(--color-primary-element-light); border-radius: var(--border-radius); }
|
||||||
|
|||||||
@@ -26,30 +26,3 @@ export function formatSize(bytes) {
|
|||||||
}
|
}
|
||||||
return i === 0 ? `${b} B` : `${b.toFixed(1)} ${units[i]}`
|
return i === 0 ? `${b} B` : `${b.toFixed(1)} ${units[i]}`
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* A time of day "HH:MM" in zone `from`, as "HH:MM" in zone `to`, at today's offsets.
|
|
||||||
*
|
|
||||||
* @param {string} hhmm the time
|
|
||||||
* @param {string} from IANA zone it is in
|
|
||||||
* @param {string} to IANA zone to show it in
|
|
||||||
* @param {Date} now the day to convert on (today)
|
|
||||||
* @return {string}
|
|
||||||
*/
|
|
||||||
export function toZone(hhmm, from, to, now = new Date()) {
|
|
||||||
const [h, m] = hhmm.split(':').map(Number)
|
|
||||||
const parts = (zone, date) => Object.fromEntries(new Intl.DateTimeFormat('en-GB', {
|
|
||||||
timeZone: zone, hourCycle: 'h23', year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit',
|
|
||||||
}).formatToParts(date).map((p) => [p.type, p.value]))
|
|
||||||
const offsetAt = (at) => {
|
|
||||||
const seen = parts(from, new Date(at))
|
|
||||||
return Date.UTC(+seen.year, +seen.month - 1, +seen.day, +seen.hour, +seen.minute) - at
|
|
||||||
}
|
|
||||||
const today = parts(from, now)
|
|
||||||
const wall = Date.UTC(+today.year, +today.month - 1, +today.day, h, m)
|
|
||||||
// the offset at the wall time itself: on a day the clocks change it differs from the one at
|
|
||||||
// midnight, so it is looked up a second time at the moment found first
|
|
||||||
const at = wall - offsetAt(wall - offsetAt(wall))
|
|
||||||
const there = parts(to, new Date(at))
|
|
||||||
return `${there.hour}:${there.minute}`
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -22,6 +22,27 @@ class KeyRingTest extends TestCase {
|
|||||||
KeyRing::unwrap($wrapped, 'wrong');
|
KeyRing::unwrap($wrapped, 'wrong');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testInvalidBase64IsADamagedKey(): void {
|
||||||
|
$wrapped = KeyRing::generate()->wrap('right');
|
||||||
|
$wrapped['salt'] = '***not base64***';
|
||||||
|
$this->expectException(CryptoException::class);
|
||||||
|
$this->expectExceptionMessage('Damaged key file');
|
||||||
|
KeyRing::unwrapAny($wrapped, 'right');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAShortSaltOrNonceIsADamagedKey(): void {
|
||||||
|
foreach (['salt', 'nonce'] as $part) {
|
||||||
|
$wrapped = KeyRing::generate()->wrap('right');
|
||||||
|
$wrapped[$part] = base64_encode('short');
|
||||||
|
try {
|
||||||
|
KeyRing::unwrapAny($wrapped, 'right');
|
||||||
|
$this->fail("a short $part was accepted");
|
||||||
|
} catch (CryptoException $e) {
|
||||||
|
$this->assertStringContainsString('Damaged key file', $e->getMessage(), $part);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public function testBlobIdsAreKeyed(): void {
|
public function testBlobIdsAreKeyed(): void {
|
||||||
$this->assertNotSame(KeyRing::generate()->blobId('same'), KeyRing::generate()->blobId('same'));
|
$this->assertNotSame(KeyRing::generate()->blobId('same'), KeyRing::generate()->blobId('same'));
|
||||||
$k = KeyRing::generate();
|
$k = KeyRing::generate();
|
||||||
@@ -34,6 +55,20 @@ class KeyRingTest extends TestCase {
|
|||||||
new KeyRing('short');
|
new KeyRing('short');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testADamagedSlotDoesNotStopTheNextOne(): void {
|
||||||
|
$keys = KeyRing::generate();
|
||||||
|
$broken = $keys->wrap('passphrase in the safe');
|
||||||
|
$broken['salt'] = '*** not base64 ***';
|
||||||
|
$field = ['slots' => [
|
||||||
|
['slot' => 1, 'label' => 'Safe', 'key' => $broken],
|
||||||
|
['slot' => 2, 'label' => 'Missing parts', 'key' => ['v' => 1]],
|
||||||
|
['slot' => 3, 'label' => 'CTO', 'key' => $keys->wrap('passphrase of the cto')],
|
||||||
|
]];
|
||||||
|
$this->assertSame($keys->blobId('x'), KeyRing::unwrapAny($field, 'passphrase of the cto')->blobId('x'));
|
||||||
|
$this->expectException(CryptoException::class);
|
||||||
|
KeyRing::unwrapAny($field, 'passphrase in the safe');
|
||||||
|
}
|
||||||
|
|
||||||
public function testAnySlotOpensTheSameKey(): void {
|
public function testAnySlotOpensTheSameKey(): void {
|
||||||
$keys = KeyRing::generate();
|
$keys = KeyRing::generate();
|
||||||
$field = ['slots' => [
|
$field = ['slots' => [
|
||||||
|
|||||||
@@ -28,6 +28,28 @@ class TreeWalkerTest extends TestCase {
|
|||||||
$this->assertSame(['Z', 'a', 'a-b', 'a.txt', 'b', 'skip'], array_values(array_unique(array_map(fn ($p) => explode('/', $p)[0], $paths))));
|
$this->assertSame(['Z', 'a', 'a-b', 'a.txt', 'b', 'skip'], array_values(array_unique(array_map(fn ($p) => explode('/', $p)[0], $paths))));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testEmptyDirectoriesAreYieldedInOrderAndOnResume(): void {
|
||||||
|
$d = $this->tree();
|
||||||
|
mkdir("$d/a/empty");
|
||||||
|
mkdir("$d/a-hollow/inner", 0700, true);
|
||||||
|
symlink("$d/a.txt", "$d/skip/link");
|
||||||
|
mkdir("$d/only-link");
|
||||||
|
symlink("$d/a.txt", "$d/only-link/link");
|
||||||
|
$w = new TreeWalker(['' => $d]);
|
||||||
|
$all = array_keys(iterator_to_array($w->walk()));
|
||||||
|
$this->assertContains('a/empty', $all);
|
||||||
|
$this->assertContains('a-hollow/inner', $all);
|
||||||
|
$this->assertNotContains('a-hollow', $all, 'only the deepest');
|
||||||
|
$this->assertContains('only-link', $all, 'a link is not walked');
|
||||||
|
$this->assertNotContains('skip', $all);
|
||||||
|
$sorted = $all;
|
||||||
|
usort($sorted, [TreeWalker::class, 'compare']);
|
||||||
|
$this->assertSame($sorted, $all);
|
||||||
|
foreach ($all as $i => $cursor) {
|
||||||
|
$this->assertSame(array_slice($all, $i + 1), array_keys(iterator_to_array($w->walk($cursor))), "after $cursor");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public function testResumeAfterCursorYieldsTheRest(): void {
|
public function testResumeAfterCursorYieldsTheRest(): void {
|
||||||
$w = new TreeWalker(['' => $this->tree()]);
|
$w = new TreeWalker(['' => $this->tree()]);
|
||||||
$all = array_keys(iterator_to_array($w->walk()));
|
$all = array_keys(iterator_to_array($w->walk()));
|
||||||
|
|||||||
@@ -50,6 +50,40 @@ class CatalogTest extends TestCase {
|
|||||||
$this->assertSame(4, $again->catalogGeneration());
|
$this->assertSame(4, $again->catalogGeneration());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testTheTrashIsReadInPartsThatTogetherAreAll(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
for ($i = 0; $i < 3; $i++) {
|
||||||
|
$repo->forget($repo->backupDirectory($this->src)['snapshot']);
|
||||||
|
}
|
||||||
|
$again = $this->open();
|
||||||
|
$seen = [];
|
||||||
|
// A deadline already past: one entry per request, as on a slow location
|
||||||
|
$after = '';
|
||||||
|
for ($request = 0; count($seen) < 3 && $request < 5; $request++) {
|
||||||
|
$page = $again->trashPage($after, 0.0);
|
||||||
|
$this->assertSame(3, $page['total']);
|
||||||
|
$this->assertCount(1, $page['items']);
|
||||||
|
$seen += $page['items'];
|
||||||
|
$after = (string)array_key_last($page['items']);
|
||||||
|
}
|
||||||
|
$this->assertEqualsCanonicalizing(array_keys($again->trash()), array_keys($seen));
|
||||||
|
$this->assertSame(['total' => 3, 'items' => []], $again->trashPage($after, 0.0));
|
||||||
|
$this->assertCount(3, $again->trashPage('', microtime(true) + 60)['items']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAnEntryThatLeavesTheTrashBetweenPartsMakesNoneComeTwiceOrBeMissed(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
for ($i = 0; $i < 3; $i++) {
|
||||||
|
$repo->forget($repo->backupDirectory($this->src)['snapshot']);
|
||||||
|
}
|
||||||
|
$reader = $this->open();
|
||||||
|
$ids = array_keys($reader->trashPage('', microtime(true) + 60)['items']);
|
||||||
|
$first = $reader->trashPage('', 0.0)['items'];
|
||||||
|
$this->assertSame([$ids[0]], array_keys($first));
|
||||||
|
$this->open()->untrash($ids[0]); // another administrator, between two parts
|
||||||
|
$this->assertSame([$ids[1], $ids[2]], array_keys($this->open()->trashPage($ids[0], microtime(true) + 60)['items']));
|
||||||
|
}
|
||||||
|
|
||||||
public function testRollbackIsDetected(): void {
|
public function testRollbackIsDetected(): void {
|
||||||
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
$repo->backupDirectory($this->src);
|
$repo->backupDirectory($this->src);
|
||||||
|
|||||||
@@ -0,0 +1,158 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Job\BackupRun;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** Folders without files are recorded in the snapshot's meta record and come back on a restore. */
|
||||||
|
class EmptyDirsTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private LocalBackend $backend;
|
||||||
|
private KeyRing $keys;
|
||||||
|
private string $src;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->backend = new LocalBackend($this->tempDir());
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$this->src = $this->tempDir();
|
||||||
|
mkdir("$this->src/data/alice/files/Docs", 0700, true);
|
||||||
|
mkdir("$this->src/data/alice/files/Hollow/Nested", 0700, true);
|
||||||
|
mkdir("$this->src/data/skipped/inner", 0700, true);
|
||||||
|
file_put_contents("$this->src/data/alice/files/Docs/note.txt", "hello\n");
|
||||||
|
touch("$this->src/data/alice/files/Hollow/Nested", 1700000000);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function repo(): Repository {
|
||||||
|
return Repository::openWithKey($this->backend, $this->keys);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param list<string> $exclude */
|
||||||
|
private function backup(array $exclude = [], float $seconds = 30.0): string {
|
||||||
|
$state = BackupRun::start(['data' => "$this->src/data"], null, '', $exclude);
|
||||||
|
for ($i = 0; $i < 500 && $state['phase'] !== 'commit'; $i++) {
|
||||||
|
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, microtime(true) + $seconds)), true);
|
||||||
|
}
|
||||||
|
$this->assertSame('commit', $state['phase']);
|
||||||
|
$this->repo()->recordSnapshot($state['snapshot']);
|
||||||
|
return $state['snapshot'];
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testOnlyTheDeepestEmptyFolderIsRecordedAndFilesStayAsTheyAre(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$paths = array_column($this->repo()->emptyDirs($snapshot), 0);
|
||||||
|
$this->assertSame(['data/alice/files/Hollow/Nested', 'data/skipped/inner'], $paths);
|
||||||
|
$this->assertSame(['data/alice/files/Docs/note.txt'], array_column(iterator_to_array($this->repo()->entries($snapshot), false), 'p'));
|
||||||
|
$this->assertSame(2, $this->repo()->snapshotMeta($snapshot)['stats']['emptyDirs']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAFolderWithOnlyExcludedContentCountsAsEmpty(): void {
|
||||||
|
$snapshot = $this->backup(['data/skipped/inner']);
|
||||||
|
$this->assertContains('data/skipped', array_column($this->repo()->emptyDirs($snapshot), 0));
|
||||||
|
$this->assertNotContains('data/skipped/inner', array_column($this->repo()->emptyDirs($snapshot), 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAResumedWalkRecordsEveryFolderOnce(): void {
|
||||||
|
for ($i = 0; $i < 30; $i++) {
|
||||||
|
mkdir("$this->src/data/many/d$i", 0700, true);
|
||||||
|
file_put_contents("$this->src/data/many/f$i", 'x');
|
||||||
|
}
|
||||||
|
$snapshot = $this->backup([], 0.0); // a step per entry
|
||||||
|
$paths = array_column($this->repo()->emptyDirs($snapshot), 0);
|
||||||
|
$this->assertCount(32, $paths);
|
||||||
|
$this->assertSame(array_unique($paths), $paths);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testANameASnapshotCannotHoldIsReportedAndTheSnapshotStaysReadable(): void {
|
||||||
|
mkdir("$this->src/data/odd\\folder", 0700, true);
|
||||||
|
mkdir("$this->src/data/alice/files/line\nbreak", 0700, true);
|
||||||
|
file_put_contents("$this->src/data/alice/files/back\\slash.txt", 'x');
|
||||||
|
file_put_contents("$this->src/data/alice/files/\xff.txt", 'x');
|
||||||
|
$state = BackupRun::start(['data' => "$this->src/data"]);
|
||||||
|
for ($i = 0; $i < 50 && $state['phase'] !== 'commit'; $i++) {
|
||||||
|
// A step per entry, state through JSON: as separate processes would
|
||||||
|
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
}
|
||||||
|
$this->assertSame('commit', $state['phase']);
|
||||||
|
$this->assertSame(4, $state['stats']['badNames'], 'each reported once, over several steps');
|
||||||
|
$this->assertArrayNotHasKey('unreadable', $state['stats'], 'not a problem with permissions');
|
||||||
|
$shown = BackupRun::warnings($state)['badNames'];
|
||||||
|
$this->assertContains('data/alice/files/line?break', $shown, 'no control character in a message');
|
||||||
|
$this->assertSame($shown, array_filter($shown, fn (string $name): bool => mb_check_encoding($name, 'UTF-8')));
|
||||||
|
$this->repo()->recordSnapshot($state['snapshot']);
|
||||||
|
|
||||||
|
$this->assertSame(['data/alice/files/Hollow/Nested', 'data/skipped/inner'], array_column($this->repo()->emptyDirs($state['snapshot']), 0));
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->assertSame(1, $this->repo()->restore($state['snapshot'], $target));
|
||||||
|
$this->assertFileExists("$target/data/alice/files/Docs/note.txt");
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAnEmptyFolderThatIsThePrefixItselfIsTheTarget(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($snapshot, $target, 'data/skipped/inner/', true);
|
||||||
|
$this->assertSame([], glob("$target/*"), 'not a folder "inner" inside the target');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRestoreBringsThemBackWithTheirTime(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->assertSame(1, $this->repo()->restore($snapshot, $target));
|
||||||
|
$this->assertDirectoryExists("$target/data/alice/files/Hollow/Nested");
|
||||||
|
clearstatcache();
|
||||||
|
$this->assertSame(1700000000, filemtime("$target/data/alice/files/Hollow/Nested"));
|
||||||
|
$this->assertDirectoryExists("$target/data/skipped/inner");
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRestoreOfAPrefixStripsAndLeavesOthersOut(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($snapshot, $target, 'data/alice/', true);
|
||||||
|
$this->assertDirectoryExists("$target/files/Hollow/Nested");
|
||||||
|
$this->assertDirectoryDoesNotExist("$target/skipped");
|
||||||
|
$this->assertDirectoryDoesNotExist("$target/inner");
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRunningAgainLeavesAFolderThatIsThereAlone(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($snapshot, $target);
|
||||||
|
touch("$target/data/alice/files/Hollow/Nested", 1800000000);
|
||||||
|
file_put_contents("$target/data/alice/files/Hollow/Nested/added", 'kept');
|
||||||
|
touch("$target/data/alice/files/Hollow/Nested", 1800000000);
|
||||||
|
$this->repo()->restore($snapshot, $target, keepUnchanged: true);
|
||||||
|
clearstatcache();
|
||||||
|
$this->assertSame(1800000000, filemtime("$target/data/alice/files/Hollow/Nested"));
|
||||||
|
$this->assertFileExists("$target/data/alice/files/Hollow/Nested/added");
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testALinkOutOfTheTargetIsRefused(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$outside = $this->tempDir();
|
||||||
|
mkdir("$target/data/alice/files", 0700, true);
|
||||||
|
symlink($outside, "$target/data/alice/files/Hollow");
|
||||||
|
try {
|
||||||
|
$this->repo()->restore($snapshot, $target);
|
||||||
|
$this->fail('restored through a link');
|
||||||
|
} catch (\OCA\NgBackup\Repository\RestoreTargetException $e) {
|
||||||
|
$this->assertStringContainsString('outside', $e->getMessage());
|
||||||
|
}
|
||||||
|
$this->assertSame([], glob("$outside/*"));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testASnapshotWithoutThemHasNone(): void {
|
||||||
|
$src = $this->tempDir();
|
||||||
|
file_put_contents("$src/a", 'x');
|
||||||
|
$snapshot = $this->repo()->backupDirectory($src)['snapshot'];
|
||||||
|
$this->assertSame([], $this->repo()->emptyDirs($snapshot));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Job\BackupRun;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** Folders with something in them get their original modification time back on a restore. */
|
||||||
|
class FolderTimesTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private const OLD = 1500000000;
|
||||||
|
private const OLDER = 1400000000;
|
||||||
|
|
||||||
|
private LocalBackend $backend;
|
||||||
|
private KeyRing $keys;
|
||||||
|
private string $src;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->backend = new LocalBackend($this->tempDir());
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$this->src = $this->tempDir();
|
||||||
|
mkdir("$this->src/data/alice/files/Old/Deep", 0700, true);
|
||||||
|
mkdir("$this->src/data/alice/files/2026", 0700, true);
|
||||||
|
file_put_contents("$this->src/data/alice/files/Old/Deep/a.txt", 'a');
|
||||||
|
file_put_contents("$this->src/data/alice/files/Old/b.txt", 'b');
|
||||||
|
file_put_contents("$this->src/data/alice/files/2026/c.txt", 'c');
|
||||||
|
touch("$this->src/data/alice/files/Old/Deep", self::OLDER);
|
||||||
|
touch("$this->src/data/alice/files/Old", self::OLD);
|
||||||
|
touch("$this->src/data/alice/files/2026", self::OLD);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function repo(): Repository {
|
||||||
|
return Repository::openWithKey($this->backend, $this->keys);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function backup(float $seconds = 30.0): string {
|
||||||
|
$state = BackupRun::start(['data' => "$this->src/data"]);
|
||||||
|
for ($i = 0; $i < 500 && $state['phase'] !== 'commit'; $i++) {
|
||||||
|
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, microtime(true) + $seconds)), true);
|
||||||
|
}
|
||||||
|
$this->assertSame('commit', $state['phase']);
|
||||||
|
$this->repo()->recordSnapshot($state['snapshot']);
|
||||||
|
return $state['snapshot'];
|
||||||
|
}
|
||||||
|
|
||||||
|
private function time(string $path): int {
|
||||||
|
clearstatcache();
|
||||||
|
return (int)filemtime($path);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testEachFolderIsRecordedOnceAlsoInSmallSteps(): void {
|
||||||
|
$expected = ['data', 'data/alice', 'data/alice/files', 'data/alice/files/2026', 'data/alice/files/Old', 'data/alice/files/Old/Deep'];
|
||||||
|
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup()), 0));
|
||||||
|
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup(0.0)), 0), 'a step per entry');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testNoEmptyFolderIsAmongThem(): void {
|
||||||
|
mkdir("$this->src/data/alice/files/Hollow");
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$this->assertNotContains('data/alice/files/Hollow', array_column($this->repo()->folderTimes($snapshot), 0));
|
||||||
|
$this->assertContains('data/alice/files/Hollow', array_column($this->repo()->emptyDirs($snapshot), 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testARestoreBringsThemBack(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($snapshot, $target);
|
||||||
|
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
|
||||||
|
$this->assertSame(self::OLDER, $this->time("$target/data/alice/files/Old/Deep"));
|
||||||
|
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/2026"));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testARestoreThatGoesOnAfterAStopToo(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($snapshot, $target);
|
||||||
|
touch("$target/data/alice/files/Old", 1800000000);
|
||||||
|
$this->repo()->restore($snapshot, $target, keepUnchanged: true);
|
||||||
|
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAPrefixLeavesTheTargetAndOthersAlone(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($snapshot, $target, 'data/alice/files/Old/', true);
|
||||||
|
$this->assertSame(self::OLDER, $this->time("$target/Deep"));
|
||||||
|
$this->assertNotSame(self::OLD, $this->time($target), 'the directory that was chosen is not ours to change');
|
||||||
|
$this->assertDirectoryDoesNotExist("$target/2026");
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testALinkOutOfTheTargetIsRefused(): void {
|
||||||
|
$snapshot = $this->backup();
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$outside = $this->tempDir();
|
||||||
|
touch($outside, 1800000000);
|
||||||
|
mkdir("$target/data/alice/files", 0700, true);
|
||||||
|
symlink($outside, "$target/data/alice/files/Old");
|
||||||
|
$this->expectException(\OCA\NgBackup\Repository\RestoreTargetException::class);
|
||||||
|
$this->repo()->restore($snapshot, $target);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAFolderWithABadNameIsLeftOutAndWhatIsInItReported(): void {
|
||||||
|
mkdir("$this->src/data/odd\\folder");
|
||||||
|
file_put_contents("$this->src/data/odd\\folder/x", 'x');
|
||||||
|
$state = BackupRun::start(['data' => "$this->src/data"]);
|
||||||
|
for ($i = 0; $i < 50 && $state['phase'] !== 'commit'; $i++) {
|
||||||
|
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
|
||||||
|
}
|
||||||
|
$this->assertSame('commit', $state['phase']);
|
||||||
|
$this->assertSame(1, $state['stats']['badNames']);
|
||||||
|
$this->repo()->recordSnapshot($state['snapshot']);
|
||||||
|
$this->assertNotContains("data/odd\\folder", array_column($this->repo()->folderTimes($state['snapshot']), 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testASnapshotWithoutThemHasNone(): void {
|
||||||
|
$snapshot = $this->repo()->backupDirectory($this->src)['snapshot'];
|
||||||
|
$this->assertSame([], $this->repo()->folderTimes($snapshot));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Crypto\CryptoException;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** Disaster recovery without the recovery kit: the location gives its key with a passphrase. */
|
||||||
|
class KeyFromLocationTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private string $dir;
|
||||||
|
private LocalBackend $backend;
|
||||||
|
private KeyRing $keys;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->dir = $this->tempDir();
|
||||||
|
$this->backend = new LocalBackend($this->dir);
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
$slots = ['slots' => [
|
||||||
|
['slot' => 1, 'label' => 'A', 'created' => 0, 'key' => $this->keys->wrap('first passphrase')],
|
||||||
|
['slot' => 2, 'label' => 'B', 'created' => 0, 'key' => $this->keys->wrap('second passphrase')],
|
||||||
|
]];
|
||||||
|
Repository::initWithKey($this->backend, $this->keys, $slots)->recordSettings(['delete_delay_days' => 30]);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testEachPassphraseGivesTheKeyThatOpensTheBackups(): void {
|
||||||
|
foreach (['first passphrase', 'second passphrase'] as $passphrase) {
|
||||||
|
$keys = KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), $passphrase);
|
||||||
|
$this->assertTrue(Repository::madeWithKey($this->backend, $keys));
|
||||||
|
$this->assertSame($this->keys->exportMasterKey(), $keys->exportMasterKey());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAWrongPassphraseGivesNothing(): void {
|
||||||
|
$this->expectException(CryptoException::class);
|
||||||
|
KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), 'not the passphrase');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAnotherKeyIsNotTakenForTheRightOne(): void {
|
||||||
|
$this->assertFalse(Repository::madeWithKey($this->backend, KeyRing::generate()));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testTheSettingsAreSealedAndWrittenOnce(): void {
|
||||||
|
$this->assertSame(30, Repository::settingsAt($this->backend, $this->keys)['delete_delay_days']);
|
||||||
|
Repository::openWithKey($this->backend, $this->keys)->recordSettings(['delete_delay_days' => 1]);
|
||||||
|
$this->assertSame(30, Repository::settingsAt($this->backend, $this->keys)['delete_delay_days']);
|
||||||
|
$this->assertSame(['unreadable' => true], Repository::settingsAt($this->backend, KeyRing::generate()), 'another key: there, but not readable');
|
||||||
|
unlink($this->dir . '/settings');
|
||||||
|
$this->assertSame([], Repository::settingsAt($this->backend, $this->keys), 'none at all: a location from before 1.1.0');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAlteredSettingsAreNotBelieved(): void {
|
||||||
|
$file = $this->dir . '/settings';
|
||||||
|
$raw = file_get_contents($file);
|
||||||
|
$raw[strlen($raw) - 1] = $raw[strlen($raw) - 1] ^ "\x01";
|
||||||
|
file_put_contents($file, $raw);
|
||||||
|
$this->assertSame(['unreadable' => true], Repository::settingsAt($this->backend, $this->keys));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testTheKitWritesALostConfigAgainAndTheBackupsOpen(): void {
|
||||||
|
$src = $this->tempDir();
|
||||||
|
file_put_contents("$src/a", 'still here');
|
||||||
|
$repo = Repository::openWithKey($this->backend, $this->keys);
|
||||||
|
$id = $repo->id();
|
||||||
|
$snapshot = $repo->backupDirectory($src)['snapshot'];
|
||||||
|
unlink($this->dir . '/config');
|
||||||
|
$this->assertTrue(Repository::holdsData($this->backend));
|
||||||
|
|
||||||
|
Repository::repairConfig($this->backend, $this->keys, $this->keys->wrap('kit passphrase'));
|
||||||
|
$repo = Repository::openWithKey($this->backend, $this->keys);
|
||||||
|
$this->assertSame($id, $repo->id());
|
||||||
|
$this->assertSame([$snapshot], $repo->listSnapshots());
|
||||||
|
$out = $this->tempDir();
|
||||||
|
$repo->restore($snapshot, $out);
|
||||||
|
$this->assertSame('still here', file_get_contents("$out/a"));
|
||||||
|
$this->assertSame($this->keys->exportMasterKey(), KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), 'kit passphrase')->exportMasterKey());
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAnotherKeyWritesNoConfig(): void {
|
||||||
|
unlink($this->dir . '/config');
|
||||||
|
$other = KeyRing::generate();
|
||||||
|
try {
|
||||||
|
Repository::repairConfig($this->backend, $other, $other->wrap('x'));
|
||||||
|
$this->fail('a config was written with another key');
|
||||||
|
} catch (\OCA\NgBackup\Repository\RepositoryException $e) {
|
||||||
|
$this->assertStringContainsString('another server', $e->getMessage());
|
||||||
|
}
|
||||||
|
$this->assertFileDoesNotExist($this->dir . '/config');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAConfigThatIsThereIsNeverRepaired(): void {
|
||||||
|
$before = file_get_contents($this->dir . '/config');
|
||||||
|
try {
|
||||||
|
Repository::repairConfig($this->backend, $this->keys, $this->keys->wrap('x'));
|
||||||
|
$this->fail('an existing config was replaced');
|
||||||
|
} catch (\OCA\NgBackup\Repository\RepositoryException) {
|
||||||
|
}
|
||||||
|
$this->assertSame($before, file_get_contents($this->dir . '/config'));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testALocationWithoutSettingsHasNone(): void {
|
||||||
|
$other = new LocalBackend($this->tempDir());
|
||||||
|
Repository::initWithKey($other, $this->keys, $this->keys->wrap('x'));
|
||||||
|
$this->assertSame([], Repository::settingsAt($other, $this->keys));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Command\Restore;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Repository\RestoreTargetException;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** A stream that takes at most 3 bytes per write. */
|
||||||
|
class PartialWriteStream {
|
||||||
|
public static string $written = '';
|
||||||
|
/** @var resource|null set by PHP */
|
||||||
|
public $context;
|
||||||
|
|
||||||
|
public function stream_open(string $path, string $mode, int $options, ?string &$opened): bool {
|
||||||
|
self::$written = '';
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
public function stream_write(string $data): int {
|
||||||
|
$part = substr($data, 0, 3);
|
||||||
|
self::$written .= $part;
|
||||||
|
return strlen($part);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What goes wrong where a restore writes to is not damage in the backup: it stops the restore, also
|
||||||
|
* with --salvage, and the message does not point at --salvage.
|
||||||
|
*/
|
||||||
|
class RestoreTargetTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private LocalBackend $backend;
|
||||||
|
private KeyRing $keys;
|
||||||
|
private string $snapshot;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->backend = new LocalBackend($this->tempDir());
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
$src = $this->tempDir();
|
||||||
|
mkdir("$src/dir");
|
||||||
|
file_put_contents("$src/dir/a.bin", random_bytes(6 * 1048576));
|
||||||
|
file_put_contents("$src/dir/b.bin", random_bytes(6 * 1048576));
|
||||||
|
file_put_contents("$src/dir/c.txt", 'small');
|
||||||
|
$this->snapshot = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'))->backupDirectory($src)['snapshot'];
|
||||||
|
}
|
||||||
|
|
||||||
|
private function repo(): Repository {
|
||||||
|
return Repository::openWithKey($this->backend, $this->keys);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAShortWriteIsATargetError(): void {
|
||||||
|
if (!is_writable('/dev/full')) {
|
||||||
|
$this->markTestSkipped('needs /dev/full');
|
||||||
|
}
|
||||||
|
$out = fopen('/dev/full', 'wb');
|
||||||
|
stream_set_write_buffer($out, 0);
|
||||||
|
$this->expectException(RestoreTargetException::class);
|
||||||
|
Repository::writeAll($out, 'data', 'dir/a.bin');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAWriteThatTakesPartOfTheDataGoesOn(): void {
|
||||||
|
stream_wrapper_register('ngbpartial', PartialWriteStream::class);
|
||||||
|
try {
|
||||||
|
$out = fopen('ngbpartial://x', 'wb');
|
||||||
|
Repository::writeAll($out, 'abcdefgh', 'dir/a.bin');
|
||||||
|
fclose($out);
|
||||||
|
$this->assertSame('abcdefgh', PartialWriteStream::$written);
|
||||||
|
} finally {
|
||||||
|
stream_wrapper_unregister('ngbpartial');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testATargetErrorDoesNotLookLikeDamage(): void {
|
||||||
|
$this->assertFalse(Restore::backupLooksDamaged(new RestoreTargetException('Cannot write')));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A real full disk: run with NGB_TEST_SMALL_DISK set to an empty, writable tmpfs of a few MiB
|
||||||
|
* (docker run --tmpfs /small:size=8m,uid=...), with and without --salvage.
|
||||||
|
*
|
||||||
|
* @dataProvider salvageOrNot
|
||||||
|
*/
|
||||||
|
public function testAFullDiskStopsTheRestoreAndLeavesNoTemporary(bool $salvage): void {
|
||||||
|
$disk = getenv('NGB_TEST_SMALL_DISK');
|
||||||
|
if ($disk === false || $disk === '') {
|
||||||
|
$this->markTestSkipped('needs NGB_TEST_SMALL_DISK');
|
||||||
|
}
|
||||||
|
$target = $disk . '/' . bin2hex(random_bytes(4));
|
||||||
|
$skipped = $salvage ? [] : null;
|
||||||
|
try {
|
||||||
|
$this->repo()->restore($this->snapshot, $target, skipped: $skipped);
|
||||||
|
$this->fail('12 MiB fitted on the small disk');
|
||||||
|
} catch (RestoreTargetException $e) {
|
||||||
|
$this->assertStringContainsString('disk full', $e->getMessage());
|
||||||
|
} finally {
|
||||||
|
$left = glob("$target/dir/*.ngb-restore-*");
|
||||||
|
foreach (glob("$target/dir/*") ?: [] as $f) {
|
||||||
|
unlink($f);
|
||||||
|
}
|
||||||
|
@rmdir("$target/dir");
|
||||||
|
@rmdir($target);
|
||||||
|
}
|
||||||
|
$this->assertSame([], $left);
|
||||||
|
if ($salvage) {
|
||||||
|
$this->assertSame([], $skipped, 'nothing is listed as damaged');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array<string, array{bool}> */
|
||||||
|
public static function salvageOrNot(): array {
|
||||||
|
return ['without --salvage' => [false], 'with --salvage' => [true]];
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResumeChecksTheFolderBeforeSkipping(): void {
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($this->snapshot, $target);
|
||||||
|
$outside = $this->tempDir();
|
||||||
|
rename("$target/dir", "$outside/dir");
|
||||||
|
symlink("$outside/dir", "$target/dir");
|
||||||
|
$this->expectException(RestoreTargetException::class);
|
||||||
|
$this->repo()->restore($this->snapshot, $target, keepUnchanged: true);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResumeRemovesTemporariesWhereEverythingIsInPlace(): void {
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($this->snapshot, $target);
|
||||||
|
file_put_contents("$target/dir/a.bin.ngb-restore-0123456789ab", 'left by a stopped restore');
|
||||||
|
$this->repo()->restore($this->snapshot, $target, keepUnchanged: true);
|
||||||
|
$this->assertFileDoesNotExist("$target/dir/a.bin.ngb-restore-0123456789ab");
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testHeartbeatRunsForFilesThatAreSkipped(): void {
|
||||||
|
$target = $this->tempDir();
|
||||||
|
$this->repo()->restore($this->snapshot, $target);
|
||||||
|
$beats = 0;
|
||||||
|
$this->repo()->restore($this->snapshot, $target, '', false, function () use (&$beats): void {
|
||||||
|
$beats++;
|
||||||
|
}, keepUnchanged: true);
|
||||||
|
$this->assertSame(3, $beats);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** A pin and a note on a snapshot live in the catalog on the location, so a new server sees them too. */
|
||||||
|
class SnapshotMarksTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private LocalBackend $backend;
|
||||||
|
private KeyRing $keys;
|
||||||
|
private string $snapshot;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->backend = new LocalBackend($this->tempDir());
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
$src = $this->tempDir();
|
||||||
|
file_put_contents("$src/a", 'x');
|
||||||
|
$this->snapshot = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'))->backupDirectory($src)['snapshot'];
|
||||||
|
}
|
||||||
|
|
||||||
|
private function repo(): Repository {
|
||||||
|
return Repository::openWithKey($this->backend, $this->keys);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testTheyAreThereAfterOpeningAgain(): void {
|
||||||
|
$this->repo()->setPinned($this->snapshot, true);
|
||||||
|
$this->repo()->setSnapshotNote($this->snapshot, 'before the upgrade');
|
||||||
|
$this->assertSame([$this->snapshot], $this->repo()->pinnedSnapshots());
|
||||||
|
$this->assertSame([$this->snapshot => 'before the upgrade'], $this->repo()->snapshotNotes());
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testTheyCanBeTakenOffAgain(): void {
|
||||||
|
$this->repo()->setPinned($this->snapshot, true);
|
||||||
|
$this->repo()->setSnapshotNote($this->snapshot, 'x');
|
||||||
|
$this->repo()->setPinned($this->snapshot, false);
|
||||||
|
$this->repo()->setSnapshotNote($this->snapshot, '');
|
||||||
|
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||||
|
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testNoChangeWritesNoCatalog(): void {
|
||||||
|
$repo = $this->repo();
|
||||||
|
$repo->setPinned($this->snapshot, true);
|
||||||
|
$generation = $repo->catalogGeneration();
|
||||||
|
$repo->setPinned($this->snapshot, true);
|
||||||
|
$repo->setSnapshotNote($this->snapshot, '');
|
||||||
|
$this->assertSame($generation, $repo->catalogGeneration());
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testASnapshotThatIsNotThereIsRefused(): void {
|
||||||
|
$this->expectException(\InvalidArgumentException::class);
|
||||||
|
$this->repo()->setPinned('0123456789abcdef0123456789abcdef', true);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testATrashedSnapshotLosesItsPinAndKeepsItsNote(): void {
|
||||||
|
$repo = $this->repo();
|
||||||
|
$repo->setPinned($this->snapshot, true);
|
||||||
|
$repo->setSnapshotNote($this->snapshot, 'kept');
|
||||||
|
$repo->forget($this->snapshot);
|
||||||
|
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||||
|
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
|
||||||
|
|
||||||
|
$repo = $this->repo();
|
||||||
|
$repo->untrash($this->snapshot);
|
||||||
|
$this->assertSame([], $this->repo()->pinnedSnapshots(), 'back in the list, as a normal one');
|
||||||
|
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testThePurgeTakesTheNoteAlong(): void {
|
||||||
|
$repo = $this->repo();
|
||||||
|
$repo->setSnapshotNote($this->snapshot, 'kept');
|
||||||
|
$repo->forget($this->snapshot);
|
||||||
|
$repo->purgeTrash(0);
|
||||||
|
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testACatalogFromBeforeThisVersionReadsAsNone(): void {
|
||||||
|
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||||
|
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\HeavyJobPause as P;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
class HeavyJobPauseTest extends TestCase {
|
||||||
|
private const NOW = 1000000;
|
||||||
|
|
||||||
|
public function testDuringABackupTheyArePaused(): void {
|
||||||
|
$this->assertSame(P::PAUSE, P::decide(true, true, 0, 0, self::NOW));
|
||||||
|
$this->assertSame(P::PAUSE, P::decide(true, true, self::NOW - 60, 0, self::NOW), 'jobs added since then too');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testWithoutABackupNothingHappensOrThePauseEnds(): void {
|
||||||
|
$this->assertSame(P::NOTHING, P::decide(true, false, 0, 0, self::NOW));
|
||||||
|
$this->assertSame(P::RELEASE, P::decide(true, false, self::NOW - 60, 0, self::NOW));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testTurningItOffEndsThePause(): void {
|
||||||
|
$this->assertSame(P::NOTHING, P::decide(false, true, 0, 0, self::NOW));
|
||||||
|
$this->assertSame(P::RELEASE, P::decide(false, true, self::NOW - 60, 0, self::NOW));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAfterSixHoursTheyGetTheirTurnAndThenPauseAgain(): void {
|
||||||
|
$this->assertSame(P::PAUSE, P::decide(true, true, self::NOW - P::MAX_SECONDS + 1, 0, self::NOW));
|
||||||
|
$this->assertSame(P::REST, P::decide(true, true, self::NOW - P::MAX_SECONDS, 0, self::NOW));
|
||||||
|
$rest = self::NOW + P::REST_SECONDS;
|
||||||
|
$this->assertSame(P::NOTHING, P::decide(true, true, 0, $rest, $rest - 1));
|
||||||
|
$this->assertSame(P::PAUSE, P::decide(true, true, 0, $rest, $rest));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\LocationSchedule as S;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
class LocationScheduleTest extends TestCase {
|
||||||
|
private const BERLIN = 'Europe/Berlin';
|
||||||
|
|
||||||
|
private static function at(string $time): int {
|
||||||
|
return (int)strtotime($time);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testWhatCanBeSaved(): void {
|
||||||
|
$this->assertSame('', S::normalize('off'));
|
||||||
|
$this->assertSame('', S::normalize(' '));
|
||||||
|
$this->assertSame('02:30', S::normalize('02:30'));
|
||||||
|
$this->assertSame('02:30', S::normalize('02:30/24'));
|
||||||
|
$this->assertSame('00:00/6', S::normalize('00:00/6'));
|
||||||
|
foreach (['2:30', '24:00', '02:30/0', '02:30/25', 'daily', '02:30/x'] as $bad) {
|
||||||
|
try {
|
||||||
|
S::normalize($bad);
|
||||||
|
$this->fail("$bad accepted");
|
||||||
|
} catch (\InvalidArgumentException) {
|
||||||
|
$this->addToAssertionCount(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testOffHasNoMoment(): void {
|
||||||
|
$this->assertNull(S::lastMoment('', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testDailyIsInTheZoneOfTheLocation(): void {
|
||||||
|
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
|
||||||
|
$this->assertSame(self::at('2026-11-09 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 01:00 UTC')), 'not yet today');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testEveryNHoursStartsOverTheNextDay(): void {
|
||||||
|
$zone = 'UTC';
|
||||||
|
$this->assertSame(self::at('2026-11-10 12:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 13:00 UTC')));
|
||||||
|
$this->assertSame(self::at('2026-11-09 22:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 01:59 UTC')), '02, 07, 12, 17, 22, then 02 again');
|
||||||
|
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 02:00 UTC')), 'at the moment itself');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAClockChangeInTheNightStillGivesOneMoment(): void {
|
||||||
|
$this->assertSame(self::at('2026-03-29 01:30 UTC'), S::lastMoment('02:30', self::BERLIN, self::at('2026-03-29 12:00 UTC')), '02:30 does not exist that night');
|
||||||
|
$autumn = S::lastMoment('02:30', self::BERLIN, self::at('2026-10-25 12:00 UTC'));
|
||||||
|
$this->assertNotNull($autumn);
|
||||||
|
$local = (new \DateTimeImmutable('@' . $autumn))->setTimezone(new \DateTimeZone(self::BERLIN));
|
||||||
|
$this->assertSame('2026-10-25 02:30', $local->format('Y-m-d H:i'), '02:30 happens twice that night: one of them, on that day');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testChoosingATimeDoesNotStartABackup(): void {
|
||||||
|
$moment = self::at('2026-11-10 02:00 UTC');
|
||||||
|
$this->assertFalse(S::isDue($moment, $moment + 1, null, 0, null, $moment + 3600));
|
||||||
|
$this->assertTrue(S::isDue($moment, $moment - 1, null, 0, null, $moment + 3600));
|
||||||
|
$this->assertTrue(S::isDue($moment, 0, null, 0, null, $moment), 'a location from before schedules per location');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testABackupSinceTheMomentIsEnough(): void {
|
||||||
|
$moment = 1000000;
|
||||||
|
$this->assertFalse(S::isDue($moment, 0, $moment, 1, $moment + 10, $moment + 20));
|
||||||
|
$this->assertTrue(S::isDue($moment, 0, $moment - 1, 0, null, $moment + 20), 'it started before this moment: one more');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAfterAFailureOnlyAFewAttemptsAnHourApart(): void {
|
||||||
|
$moment = 1000000;
|
||||||
|
$finished = $moment + 60;
|
||||||
|
$this->assertFalse(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER - 1));
|
||||||
|
$this->assertTrue(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER));
|
||||||
|
$this->assertFalse(S::isDue($moment, 0, null, S::MAX_ATTEMPTS, $finished, $finished + 86400));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -25,6 +25,17 @@ class RetentionPolicyTest extends TestCase {
|
|||||||
$this->assertCount(3, $months);
|
$this->assertCount(3, $months);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testAPinnedSnapshotStaysAndDoesNotCount(): void {
|
||||||
|
$policy = new RetentionPolicy(1, 0, 0, 0);
|
||||||
|
$snaps = ['old' => 100, 'pinned' => 300, 'new' => 200];
|
||||||
|
$this->assertSame(['pinned'], array_keys($policy->keep($snaps)), 'without the pin it would take the place of "last"');
|
||||||
|
$keep = $policy->keepWithPinned($snaps, ['pinned', 'gone-already']);
|
||||||
|
$this->assertSame(['pinned'], $keep['pinned']);
|
||||||
|
$this->assertSame(['last'], $keep['new'], 'the newest one that is not pinned');
|
||||||
|
$this->assertArrayNotHasKey('old', $keep);
|
||||||
|
$this->assertArrayNotHasKey('gone-already', $keep);
|
||||||
|
}
|
||||||
|
|
||||||
public function testKeepsEverythingWhenFewSnapshots(): void {
|
public function testKeepsEverythingWhenFewSnapshots(): void {
|
||||||
$keep = (new RetentionPolicy())->keep(['a' => 100, 'b' => 200]);
|
$keep = (new RetentionPolicy())->keep(['a' => 100, 'b' => 200]);
|
||||||
$this->assertSame(['a', 'b'], array_values(array_intersect(['a', 'b'], array_keys($keep))));
|
$this->assertSame(['a', 'b'], array_values(array_intersect(['a', 'b'], array_keys($keep))));
|
||||||
|
|||||||
@@ -0,0 +1,22 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\AlertService;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** A location without a schedule: a weekly reminder while another is backed up, else a daily one. */
|
||||||
|
class ScheduleOffReminderTest extends TestCase {
|
||||||
|
public function testOnceAWeekWhileAnotherLocationIsActive(): void {
|
||||||
|
$this->assertFalse(AlertService::offReminderDue(0, 6 * 86400, true));
|
||||||
|
$this->assertTrue(AlertService::offReminderDue(0, 7 * 86400, true));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testEveryDayWhenNoLocationIsActive(): void {
|
||||||
|
$this->assertFalse(AlertService::offReminderDue(0, 3600, false));
|
||||||
|
$this->assertTrue(AlertService::offReminderDue(0, 86400, false));
|
||||||
|
$this->assertTrue(AlertService::offReminderDue(0, 86400 - 60, false), 'a cron run a minute earlier than yesterday');
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -57,15 +57,4 @@ class ScheduleTest extends TestCase {
|
|||||||
$this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get());
|
$this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get());
|
||||||
}
|
}
|
||||||
|
|
||||||
public function testTodayAtIsTheMomentInTheSchedulesZone(): void {
|
|
||||||
$this->values = [Schedule::TIME_KEY => '03:00', Schedule::ZONE_KEY => 'Europe/Berlin'];
|
|
||||||
$at = $this->schedule()->todayAt(strtotime('2026-10-22 12:00 UTC'));
|
|
||||||
$this->assertSame(strtotime('2026-10-22 01:00 UTC'), $at, '03:00 in Berlin (summer time) is 01:00 UTC');
|
|
||||||
}
|
|
||||||
|
|
||||||
public function testAnUnknownZoneOrABadTimeIsRefused(): void {
|
|
||||||
$s = $this->schedule();
|
|
||||||
$this->expectException(\InvalidArgumentException::class);
|
|
||||||
$s->set('03:00', 'Mars/Olympus');
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,8 +2,10 @@
|
|||||||
# Disaster recovery: occ backup:key:import + backup:target:add + backup:restore:full on a
|
# Disaster recovery: occ backup:key:import + backup:target:add + backup:restore:full on a
|
||||||
# SECOND, fresh Nextcloud instance that never saw the first one, sharing only the backup
|
# SECOND, fresh Nextcloud instance that never saw the first one, sharing only the backup
|
||||||
# location (a local directory, bind-mounted into both containers) -- see PLAN.md 3.7.
|
# location (a local directory, bind-mounted into both containers) -- see PLAN.md 3.7.
|
||||||
# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|salvage|salvage-files|to-pgsql|to-mysql|to-sqlite]
|
# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|salvage|salvage-files|to-pgsql|to-mysql|to-sqlite|no-kit]
|
||||||
# e.g. full-restore-roundtrip.sh 35-apache pgsql
|
# e.g. full-restore-roundtrip.sh 35-apache pgsql
|
||||||
|
# With "no-kit" the recovery kit is gone: backup:target:add takes the key from the location with
|
||||||
|
# the passphrase, and backups wait until a new kit is confirmed.
|
||||||
# With "salvage-files" one pack is damaged and another is gone from the location. The restore must
|
# With "salvage-files" one pack is damaged and another is gone from the location. The restore must
|
||||||
# stop and point at --salvage; run again as the guide says, it restores the database and every file
|
# stop and point at --salvage; run again as the guide says, it restores the database and every file
|
||||||
# that can be read, and lists the rest (exit 2). NGB_FRT_DATA_MB sets the amount of data (default 200).
|
# that can be read, and lists the rest (exit 2). NGB_FRT_DATA_MB sets the amount of data (default 200).
|
||||||
@@ -93,7 +95,10 @@ run_on ngb-frt-origin app:enable ng_backup >/dev/null
|
|||||||
ok "origin instance up ($DB${LEGACY:+, NG Backup 0.9.7})"
|
ok "origin instance up ($DB${LEGACY:+, NG Backup 0.9.7})"
|
||||||
|
|
||||||
docker exec ngb-frt-origin sh -c "printf 'full restore roundtrip passphrase 2026\n' > /tmp/pass"
|
docker exec ngb-frt-origin sh -c "printf 'full restore roundtrip passphrase 2026\n' > /tmp/pass"
|
||||||
run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI >/dev/null || fail "key init"
|
DELAY_OPT=()
|
||||||
|
[ "$MODE" = no-kit ] && DELAY_OPT=(--delete-delay-days=21)
|
||||||
|
run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI "${DELAY_OPT[@]}" >/dev/null || fail "key init"
|
||||||
|
ORIGIN_FP=$(run_on ngb-frt-origin backup:status | grep -o 'Key: [0-9a-f-]*' | cut -d' ' -f2)
|
||||||
run_on ngb-frt-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
run_on ngb-frt-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||||
CODE=$(run_on ngb-frt-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
CODE=$(run_on ngb-frt-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
run_on ngb-frt-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
run_on ngb-frt-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||||
@@ -102,7 +107,7 @@ ok "key, kit and location on the origin"
|
|||||||
|
|
||||||
NC_PASS=$(openssl rand -hex 16)
|
NC_PASS=$(openssl rand -hex 16)
|
||||||
docker exec -u www-data -e OC_PASS="$NC_PASS" ngb-frt-origin php occ user:add --password-from-env frtest >/dev/null
|
docker exec -u www-data -e OC_PASS="$NC_PASS" ngb-frt-origin php occ user:add --password-from-env frtest >/dev/null
|
||||||
docker exec ngb-frt-origin sh -c "mkdir -p /var/www/html/data/frtest/files/Docs && head -c 200000 /dev/urandom > /var/www/html/data/frtest/files/Docs/report.bin && echo 'hello disaster recovery' > /var/www/html/data/frtest/files/Docs/note.txt && chown -R www-data:www-data /var/www/html/data/frtest"
|
docker exec ngb-frt-origin sh -c "mkdir -p /var/www/html/data/frtest/files/Docs /var/www/html/data/frtest/files/Empty/Nested && head -c 200000 /dev/urandom > /var/www/html/data/frtest/files/Docs/report.bin && echo 'hello disaster recovery' > /var/www/html/data/frtest/files/Docs/note.txt && chown -R www-data:www-data /var/www/html/data/frtest"
|
||||||
if [ "$MODE" = sftp ]; then
|
if [ "$MODE" = sftp ]; then
|
||||||
# about three packs of 32 MiB
|
# about three packs of 32 MiB
|
||||||
docker exec ngb-frt-origin sh -c "head -c 90000000 /dev/urandom > /var/www/html/data/frtest/files/Docs/large.bin && chown www-data:www-data /var/www/html/data/frtest/files/Docs/large.bin"
|
docker exec ngb-frt-origin sh -c "head -c 90000000 /dev/urandom > /var/www/html/data/frtest/files/Docs/large.bin && chown www-data:www-data /var/www/html/data/frtest/files/Docs/large.bin"
|
||||||
@@ -187,8 +192,29 @@ docker exec ngb-frt-restore chmod 644 /tmp/kit.json
|
|||||||
docker cp /tmp/ngb-frt-pass ngb-frt-restore:/tmp/pass
|
docker cp /tmp/ngb-frt-pass ngb-frt-restore:/tmp/pass
|
||||||
ok "fresh restore instance up"
|
ok "fresh restore instance up"
|
||||||
|
|
||||||
|
if [ "$MODE" = no-kit ]; then
|
||||||
|
# No recovery kit: backup:target:add takes the key from the location with a passphrase
|
||||||
|
docker exec ngb-frt-restore rm -f /tmp/kit.json
|
||||||
|
docker exec ngb-frt-restore sh -c "printf 'not the passphrase of the old server\n' > /tmp/wrongpass && chmod 644 /tmp/wrongpass"
|
||||||
|
OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo --passphrase-file=/tmp/wrongpass 2>&1) \
|
||||||
|
&& { printf '%s\n' "$OUT"; fail "a wrong passphrase was accepted"; }
|
||||||
|
printf '%s\n' "$OUT" | grep -q "Nothing was changed" || { printf '%s\n' "$OUT"; fail "a wrong passphrase did not say that nothing was changed"; }
|
||||||
|
STATUS_OUT=$(run_on ngb-frt-restore backup:status 2>&1 || true)
|
||||||
|
printf '%s\n' "$STATUS_OUT" | grep -q "Key: not created" || { printf '%s\n' "$STATUS_OUT"; fail "a wrong passphrase left a key behind"; }
|
||||||
|
[ -z "$(run_on ngb-frt-restore backup:target:list 2>&1 | grep offsite)" ] || fail "a wrong passphrase left a location behind"
|
||||||
|
OUT=$(run_on ngb-frt-restore backup:target:add other "${LOCATION[@]}" --path=nothing-here --passphrase-file=/tmp/pass 2>&1) \
|
||||||
|
&& { printf '%s\n' "$OUT"; fail "a folder without backups was added without a key"; }
|
||||||
|
printf '%s\n' "$OUT" | grep -q "no backups at this location" || { printf '%s\n' "$OUT"; fail "a folder without backups did not say so"; }
|
||||||
|
ok "without a kit: a wrong passphrase and a folder without backups change nothing"
|
||||||
|
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo --passphrase-file=/tmp/pass 2>&1) \
|
||||||
|
|| { printf '%s\n' "$ADD_OUT"; fail "target add with the passphrase on the restore instance"; }
|
||||||
|
printf '%s\n' "$ADD_OUT" | grep -q "fingerprint $ORIGIN_FP" || { printf '%s\n' "$ADD_OUT"; fail "the key from the location is not the origin's ($ORIGIN_FP)"; }
|
||||||
|
printf '%s\n' "$ADD_OUT" | grep -q "21 days (from the location)" || { printf '%s\n' "$ADD_OUT"; fail "the deletion delay did not come from the location"; }
|
||||||
|
ok "without a kit: the key and the deletion delay came from the location"
|
||||||
|
else
|
||||||
run_on ngb-frt-restore backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "key import"
|
run_on ngb-frt-restore backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "key import"
|
||||||
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo 2>&1) || { printf '%s\n' "$ADD_OUT"; fail "target add on the restore instance"; }
|
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo 2>&1) || { printf '%s\n' "$ADD_OUT"; fail "target add on the restore instance"; }
|
||||||
|
fi
|
||||||
LIST_OUT=$(run_on ngb-frt-restore backup:list offsite 2>&1)
|
LIST_OUT=$(run_on ngb-frt-restore backup:list offsite 2>&1)
|
||||||
printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
|
printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
|
||||||
printf 'target:add said:\n%s\nbackup:list said:\n%s\n' "$ADD_OUT" "$LIST_OUT"
|
printf 'target:add said:\n%s\nbackup:list said:\n%s\n' "$ADD_OUT" "$LIST_OUT"
|
||||||
@@ -196,7 +222,8 @@ printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
|
|||||||
docker exec ngb-frt-restore sh -c 'ls -la /mnt/shared/repo /mnt/shared/repo/snapshots' || true
|
docker exec ngb-frt-restore sh -c 'ls -la /mnt/shared/repo /mnt/shared/repo/snapshots' || true
|
||||||
fail "the origin's snapshot is not listed after connecting the location"
|
fail "the origin's snapshot is not listed after connecting the location"
|
||||||
}
|
}
|
||||||
ok "recovery kit imported, existing repository adopted, snapshot visible"
|
if [ "$MODE" = no-kit ]; then KEY_FROM="key from the location"; else KEY_FROM="recovery kit imported"; fi
|
||||||
|
ok "$KEY_FROM, existing repository adopted, snapshot visible"
|
||||||
|
|
||||||
if [ "$MODE" = salvage ]; then
|
if [ "$MODE" = salvage ]; then
|
||||||
set +e
|
set +e
|
||||||
@@ -214,6 +241,7 @@ if [ "$MODE" = salvage ]; then
|
|||||||
printf '%s\n' "$OUT" | grep "$SALVAGE" >/dev/null || { printf '%s\n' "$OUT" | tail -30; fail "the output does not name the salvage folder"; }
|
printf '%s\n' "$OUT" | grep "$SALVAGE" >/dev/null || { printf '%s\n' "$OUT" | tail -30; fail "the output does not name the salvage folder"; }
|
||||||
[ "$(docker exec ngb-frt-restore cat "$SALVAGE/data/frtest/files/Docs/note.txt")" = "hello disaster recovery" ] || fail "the user's file is not in the salvage folder"
|
[ "$(docker exec ngb-frt-restore cat "$SALVAGE/data/frtest/files/Docs/note.txt")" = "hello disaster recovery" ] || fail "the user's file is not in the salvage folder"
|
||||||
docker exec ngb-frt-restore test -f "$SALVAGE/config/config.php" || fail "config.php is not in the salvage folder"
|
docker exec ngb-frt-restore test -f "$SALVAGE/config/config.php" || fail "config.php is not in the salvage folder"
|
||||||
|
docker exec ngb-frt-restore test -d "$SALVAGE/data/frtest/files/Empty/Nested" || fail "the empty folder is not in the salvage folder"
|
||||||
run_on ngb-frt-restore user:list | grep frtest >/dev/null && fail "the fresh database was changed (the backed-up user is there)"
|
run_on ngb-frt-restore user:list | grep frtest >/dev/null && fail "the fresh database was changed (the backed-up user is there)"
|
||||||
run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode is still on after the salvage"
|
run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode is still on after the salvage"
|
||||||
run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after the salvage"
|
run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after the salvage"
|
||||||
@@ -254,8 +282,8 @@ if [ "$MODE" = salvage-files ]; then
|
|||||||
if (!file_exists($root . $p)) { $bad[] = "not listed and not there: $p"; continue; }
|
if (!file_exists($root . $p)) { $bad[] = "not listed and not there: $p"; continue; }
|
||||||
hash_file("sha256", $root . $p) === $sum ? $same++ : $bad[] = "different: $p";
|
hash_file("sha256", $root . $p) === $sum ? $same++ : $bad[] = "different: $p";
|
||||||
}
|
}
|
||||||
$temps = glob($root . "*/*.ngb-restore-*") ?: [];
|
$walk = new RecursiveIteratorIterator(new RecursiveDirectoryIterator("/var/www/html/data", FilesystemIterator::SKIP_DOTS), RecursiveIteratorIterator::SELF_FIRST);
|
||||||
foreach ($temps as $t) { $bad[] = "temporary left: $t"; }
|
foreach ($walk as $f) { if (str_contains($f->getFilename(), ".ngb-restore-")) { $bad[] = "temporary left: " . $f->getPathname(); } }
|
||||||
echo "same=$same gone=$gone bad=", count($bad), "\n", implode("\n", array_slice($bad, 0, 10)), "\n";
|
echo "same=$same gone=$gone bad=", count($bad), "\n", implode("\n", array_slice($bad, 0, 10)), "\n";
|
||||||
echo "reasons: ", implode(" | ", array_unique(array_map(fn ($r) => substr(preg_replace("#[0-9a-f]{16,}#", "<id>", $r), 0, 70), $reasons))), "\n";')
|
echo "reasons: ", implode(" | ", array_unique(array_map(fn ($r) => substr(preg_replace("#[0-9a-f]{16,}#", "<id>", $r), 0, 70), $reasons))), "\n";')
|
||||||
printf '%s\n' "$CHECK"
|
printf '%s\n' "$CHECK"
|
||||||
@@ -265,6 +293,8 @@ if [ "$MODE" = salvage-files ]; then
|
|||||||
ok "with --salvage: every file listed and absent, or there and identical"
|
ok "with --salvage: every file listed and absent, or there and identical"
|
||||||
run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode is still on after the salvage"
|
run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode is still on after the salvage"
|
||||||
run_on ngb-frt-restore user:list | grep frtest >/dev/null || fail "the backed-up user is missing: the database was not restored"
|
run_on ngb-frt-restore user:list | grep frtest >/dev/null || fail "the backed-up user is missing: the database was not restored"
|
||||||
|
run_on ngb-frt-restore user:list | grep admin >/dev/null || fail "the admin is missing: the database was not restored"
|
||||||
|
docker exec ngb-frt-restore test -d /var/www/html/data/frtest/files/Empty/Nested || fail "the empty folder did not come back with --salvage"
|
||||||
[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt 2>/dev/null)" = "hello disaster recovery" ] \
|
[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt 2>/dev/null)" = "hello disaster recovery" ] \
|
||||||
|| printf '%s\n' "$OUT" | grep "Docs/note.txt" >/dev/null || fail "note.txt is neither restored nor listed"
|
|| printf '%s\n' "$OUT" | grep "Docs/note.txt" >/dev/null || fail "note.txt is neither restored nor listed"
|
||||||
run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after the salvage"
|
run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after the salvage"
|
||||||
@@ -323,7 +353,16 @@ run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails aft
|
|||||||
run_on ngb-frt-restore user:list | grep frtest >/dev/null || fail "the backed-up user is missing from the restored database"
|
run_on ngb-frt-restore user:list | grep frtest >/dev/null || fail "the backed-up user is missing from the restored database"
|
||||||
docker exec ngb-frt-restore test -f /var/www/html/data/frtest/files/Docs/note.txt || fail "restored file is missing"
|
docker exec ngb-frt-restore test -f /var/www/html/data/frtest/files/Docs/note.txt || fail "restored file is missing"
|
||||||
[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt)" = "hello disaster recovery" ] || fail "restored file content differs"
|
[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt)" = "hello disaster recovery" ] || fail "restored file content differs"
|
||||||
|
if [ "$LEGACY" = legacy ]; then
|
||||||
|
# Backups made before 1.0.3 do not record empty folders
|
||||||
|
docker exec ngb-frt-restore test -e /var/www/html/data/frtest/files/Empty/Nested && fail "an empty folder came back from a backup that does not record them"
|
||||||
run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after restore"
|
run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after restore"
|
||||||
|
else
|
||||||
|
docker exec ngb-frt-restore test -d /var/www/html/data/frtest/files/Empty/Nested || fail "the empty folder did not come back"
|
||||||
|
run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after restore"
|
||||||
|
docker exec ngb-frt-restore test -d /var/www/html/data/frtest/files/Empty/Nested || fail "files:scan removed the empty folder"
|
||||||
|
ok "the empty folder came back and stays after files:scan"
|
||||||
|
fi
|
||||||
# New rows after the restore: a new file (file cache) and a new user must not collide with
|
# New rows after the restore: a new file (file cache) and a new user must not collide with
|
||||||
# restored rows (ID counters after the data).
|
# restored rows (ID counters after the data).
|
||||||
docker exec ngb-frt-restore sh -c "echo new > /var/www/html/data/frtest/files/Docs/after-restore.txt && chown www-data /var/www/html/data/frtest/files/Docs/after-restore.txt"
|
docker exec ngb-frt-restore sh -c "echo new > /var/www/html/data/frtest/files/Docs/after-restore.txt && chown www-data /var/www/html/data/frtest/files/Docs/after-restore.txt"
|
||||||
@@ -359,6 +398,13 @@ if [ "$MODE" = sftp ]; then
|
|||||||
ok "over SFTP: read ahead on connections of its own ($LOGINS SFTP logins during the restore), large file identical, no spool left"
|
ok "over SFTP: read ahead on connections of its own ($LOGINS SFTP logins during the restore), large file identical, no spool left"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
if [ "$MODE" = no-kit ]; then
|
||||||
|
OUT=$(run_on ngb-frt-restore backup:run offsite -l post-restore-check 2>&1) && { printf '%s\n' "$OUT"; fail "a backup ran without a confirmed recovery kit"; }
|
||||||
|
printf '%s\n' "$OUT" | grep -q "Confirm the recovery kit" || { printf '%s\n' "$OUT"; fail "the refused backup does not point at the recovery kit"; }
|
||||||
|
CODE=$(run_on ngb-frt-restore backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
run_on ngb-frt-restore backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm a new kit after the restore"
|
||||||
|
ok "without a kit: backups wait for a new recovery kit to be confirmed"
|
||||||
|
fi
|
||||||
run_on ngb-frt-restore backup:run offsite -l post-restore-check >/dev/null || fail "a new backup after the restore fails"
|
run_on ngb-frt-restore backup:run offsite -l post-restore-check >/dev/null || fail "a new backup after the restore fails"
|
||||||
ok "the restored instance can run a new backup of its own"
|
ok "the restored instance can run a new backup of its own"
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,311 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Getting the backup key onto a new server, with and without the recovery kit, and what the
|
||||||
|
# administrator is told in each case. Every scenario starts on a fresh Nextcloud (SQLite) with a
|
||||||
|
# copy of the origin's location; nothing is restored (full-restore-roundtrip.sh does that).
|
||||||
|
# Usage: recovery-scenarios.sh [nextcloud-image-tag] e.g. recovery-scenarios.sh 35-apache
|
||||||
|
#
|
||||||
|
# kit kit + passphrase: backup:key:import, then backup:target:add asks nothing
|
||||||
|
# kit-wrong kit with a wrong passphrase: refused, no key left behind
|
||||||
|
# no-kit passphrase only: target:add asks for it (or --passphrase-file), a wrong one
|
||||||
|
# changes nothing; backups wait for a new kit
|
||||||
|
# no-kit-old as no-kit, on a location backed up only before 1.1.0 (no settings file):
|
||||||
|
# the deletion delay is the default, and the admin is told so
|
||||||
|
# no-kit-damaged as no-kit, with a damaged settings file: the longest deletion delay, and why
|
||||||
|
# no-config-no-kit the location's config is gone and there is no kit: told that only the kit helps
|
||||||
|
# no-config-kit config gone, kit there: --repair-config writes it again, the backups open
|
||||||
|
# other-kit the kit of another server: refused, told to use the old server's kit
|
||||||
|
# empty nothing at the location: told to check the details, not to make a new key
|
||||||
|
# unused-key-kit a new key was made by mistake: key:import refuses and names
|
||||||
|
# --replace-unused-key; with it the kit's key takes its place
|
||||||
|
# unused-key-no-kit the same without a kit: target:add with --replace-unused-key and a passphrase
|
||||||
|
# of the old server; a wrong one keeps the unused key
|
||||||
|
# used-key a key that protected a location is never replaced, also after the location was removed
|
||||||
|
# slot-catch-up a passphrase added while a location was unreachable reaches it at its next
|
||||||
|
# backup; a new server then gets the key from it with that passphrase
|
||||||
|
set -euo pipefail
|
||||||
|
IMAGE="nextcloud:${1:-35-apache}"
|
||||||
|
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
||||||
|
SHARED=$(mktemp -d)
|
||||||
|
chmod 777 "$SHARED"
|
||||||
|
PASS='recovery scenarios passphrase 2026'
|
||||||
|
fail() { echo "FAIL: $*"; exit 1; }
|
||||||
|
ok() { echo "ok $*"; }
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
docker rm -fv ngb-rs-origin ngb-rs-other ngb-rs-new >/dev/null 2>&1 || true
|
||||||
|
docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
|
||||||
|
rmdir "$SHARED" 2>/dev/null || true
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
|
||||||
|
# Output of a command that must fail; the scenario fails when it succeeds.
|
||||||
|
must_fail() { # container what args...
|
||||||
|
local out
|
||||||
|
if out=$(run_on "$1" "${@:3}" 2>&1); then printf '%s\n' "$out"; fail "$2: the command succeeded"; fi
|
||||||
|
printf '%s\n' "$out"
|
||||||
|
}
|
||||||
|
expect() { # output text what
|
||||||
|
printf '%s\n' "$1" | grep -qF -- "$2" || { printf '%s\n' "$1"; fail "$3: the output does not say \"$2\""; }
|
||||||
|
}
|
||||||
|
not_expect() {
|
||||||
|
printf '%s\n' "$1" | grep -qF -- "$2" && { printf '%s\n' "$1"; fail "$3: the output says \"$2\""; }
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
start() { # name
|
||||||
|
docker rm -fv "$1" >/dev/null 2>&1 || true
|
||||||
|
docker run -d --name "$1" -e SQLITE_DATABASE=nextcloud -e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin \
|
||||||
|
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
|
||||||
|
for i in $(seq 1 90); do run_on "$1" status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
|
||||||
|
docker exec "$1" chown www-data:www-data /var/www/html/custom_apps
|
||||||
|
run_on "$1" app:enable ng_backup >/dev/null
|
||||||
|
docker exec "$1" sh -c "printf '%s\n' '$PASS' > /tmp/pass; printf 'not the passphrase at all\n' > /tmp/wrongpass; chmod 644 /tmp/pass /tmp/wrongpass"
|
||||||
|
}
|
||||||
|
key_of() { run_on "$1" backup:status 2>&1 | sed -n 's/^Key: \([0-9a-f-]*\),.*/\1/p'; }
|
||||||
|
no_key() { # container what
|
||||||
|
local s
|
||||||
|
s=$(run_on "$1" backup:status 2>&1 || true)
|
||||||
|
expect "$s" "Key: not created" "$2: a key was left behind"
|
||||||
|
[ -z "$(run_on "$1" backup:target:list 2>&1 | grep offsite || true)" ] || fail "$2: a location was left behind"
|
||||||
|
}
|
||||||
|
location() { echo "local -a null::null -o datadir=/mnt/shared/$1 --path=repo"; }
|
||||||
|
# A fresh server with its own copy of the origin's location
|
||||||
|
scenario() { # name
|
||||||
|
docker exec ngb-rs-origin sh -c "rm -rf /mnt/shared/$1 && mkdir -p /mnt/shared/$1 && cp -a /mnt/shared/origin/repo /mnt/shared/$1/ && chown -R www-data /mnt/shared/$1"
|
||||||
|
start ngb-rs-new
|
||||||
|
docker exec -i ngb-rs-new sh -c 'cat > /tmp/kit.json && chmod 644 /tmp/kit.json' < "$SHARED/kit.json"
|
||||||
|
}
|
||||||
|
|
||||||
|
# The origin: a key with a deletion delay of its own, a kit, one backup
|
||||||
|
start ngb-rs-origin
|
||||||
|
run_on ngb-rs-origin backup:key:init --passphrase-file=/tmp/pass --label=Safe --delete-delay-days=21 >/dev/null || fail "key init"
|
||||||
|
run_on ngb-rs-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||||
|
CODE=$(run_on ngb-rs-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
run_on ngb-rs-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||||
|
docker exec ngb-rs-origin sh -c 'mkdir -p /mnt/shared/origin && chown www-data /mnt/shared/origin'
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
run_on ngb-rs-origin backup:target:add offsite $(location origin) >/dev/null || fail "target add on the origin"
|
||||||
|
run_on ngb-rs-origin backup:run offsite >/dev/null || fail "backup on the origin"
|
||||||
|
SNAP=$(run_on ngb-rs-origin backup:list offsite | grep -o '[0-9a-f]\{32\}' | head -1)
|
||||||
|
FP=$(key_of ngb-rs-origin)
|
||||||
|
docker exec ngb-rs-origin cat /tmp/kit.json > "$SHARED/kit.json"
|
||||||
|
docker exec ngb-rs-origin test -f /mnt/shared/origin/repo/settings || fail "the origin's location has no settings file"
|
||||||
|
ok "origin: key $FP, kit, snapshot $SNAP, settings on the location"
|
||||||
|
|
||||||
|
# kit: the passphrase once, for the kit; target:add asks nothing more
|
||||||
|
scenario kit
|
||||||
|
OUT=$(run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "kit: key import"; }
|
||||||
|
expect "$OUT" "fingerprint $FP" "kit"
|
||||||
|
expect "$OUT" "21 days (from the kit)" "kit"
|
||||||
|
expect "$OUT" "backup:target:add" "kit: the next step"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location kit) --no-interaction 2>&1) || { printf '%s\n' "$OUT"; fail "kit: target add"; }
|
||||||
|
not_expect "$OUT" "Passphrase" "kit: target:add asked for a passphrase"
|
||||||
|
expect "$OUT" "found and adopted" "kit"
|
||||||
|
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "kit: the snapshot is not listed"
|
||||||
|
run_on ngb-rs-new backup:run offsite >/dev/null || fail "kit: a backup after the kit import"
|
||||||
|
ok "kit: imported with the passphrase, location added without asking again, backups run"
|
||||||
|
|
||||||
|
# kit-wrong
|
||||||
|
scenario kit-wrong
|
||||||
|
OUT=$(must_fail ngb-rs-new "kit-wrong" backup:key:import /tmp/kit.json --passphrase-file=/tmp/wrongpass)
|
||||||
|
expect "$OUT" "Wrong passphrase" "kit-wrong"
|
||||||
|
no_key ngb-rs-new "kit-wrong"
|
||||||
|
ok "kit-wrong: refused, nothing left behind"
|
||||||
|
|
||||||
|
# no-kit
|
||||||
|
scenario no-kit
|
||||||
|
docker exec ngb-rs-new rm /tmp/kit.json
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "no-kit without a passphrase" backup:target:add offsite $(location no-kit) --no-interaction)
|
||||||
|
expect "$OUT" "this location holds backups" "no-kit: explains where the key comes from"
|
||||||
|
expect "$OUT" "--passphrase-file" "no-kit: says how to give the passphrase"
|
||||||
|
no_key ngb-rs-new "no-kit without a passphrase"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "no-kit wrong passphrase" backup:target:add offsite $(location no-kit) --passphrase-file=/tmp/wrongpass)
|
||||||
|
expect "$OUT" "Wrong passphrase" "no-kit wrong passphrase"
|
||||||
|
expect "$OUT" "Nothing was changed" "no-kit wrong passphrase"
|
||||||
|
no_key ngb-rs-new "no-kit wrong passphrase"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit: target add"; }
|
||||||
|
expect "$OUT" "fingerprint $FP" "no-kit"
|
||||||
|
expect "$OUT" "21 days (from the location)" "no-kit"
|
||||||
|
expect "$OUT" "download a new recovery kit" "no-kit: asks for a new kit"
|
||||||
|
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "no-kit: the snapshot is not listed"
|
||||||
|
OUT=$(must_fail ngb-rs-new "no-kit backup before a new kit" backup:run offsite)
|
||||||
|
expect "$OUT" "Confirm the recovery kit" "no-kit: a backup before a new kit"
|
||||||
|
CODE=$(run_on ngb-rs-new backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
run_on ngb-rs-new backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "no-kit: confirm the new kit"
|
||||||
|
run_on ngb-rs-new backup:run offsite >/dev/null || fail "no-kit: a backup after the new kit"
|
||||||
|
ok "no-kit: passphrase asked, wrong one changes nothing, key and delay from the location, backups wait for a new kit"
|
||||||
|
|
||||||
|
# no-kit-old: a location from before 1.1.0 has no settings file
|
||||||
|
scenario no-kit-old
|
||||||
|
docker exec ngb-rs-new rm /tmp/kit.json /mnt/shared/no-kit-old/repo/settings
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit-old) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit-old: target add"; }
|
||||||
|
expect "$OUT" "fingerprint $FP" "no-kit-old"
|
||||||
|
expect "$OUT" "7 days (the default" "no-kit-old: the default delay and why"
|
||||||
|
docker exec ngb-rs-new test -f /mnt/shared/no-kit-old/repo/settings || fail "no-kit-old: the settings were not written when the location was added"
|
||||||
|
ok "no-kit-old: key from the location, default delay with the reason, settings written now"
|
||||||
|
|
||||||
|
# no-kit-damaged: the settings file is there but damaged: the longest delay, and the admin is told
|
||||||
|
scenario no-kit-damaged
|
||||||
|
docker exec ngb-rs-new sh -c 'rm /tmp/kit.json && printf garbage >> /mnt/shared/no-kit-damaged/repo/settings'
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit-damaged) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit-damaged: target add"; }
|
||||||
|
expect "$OUT" "fingerprint $FP" "no-kit-damaged"
|
||||||
|
expect "$OUT" "365 days (the longest" "no-kit-damaged: the longest delay and why"
|
||||||
|
ok "no-kit-damaged: key from the location, the longest delay with the reason"
|
||||||
|
|
||||||
|
# no-config-no-kit
|
||||||
|
scenario no-config-no-kit
|
||||||
|
docker exec ngb-rs-new rm /tmp/kit.json /mnt/shared/no-config-no-kit/repo/config
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "no-config-no-kit" backup:target:add offsite $(location no-config-no-kit) --passphrase-file=/tmp/pass)
|
||||||
|
expect "$OUT" "its config file is gone" "no-config-no-kit"
|
||||||
|
expect "$OUT" "only the recovery kit has the key" "no-config-no-kit"
|
||||||
|
expect "$OUT" "--repair-config" "no-config-no-kit"
|
||||||
|
no_key ngb-rs-new "no-config-no-kit"
|
||||||
|
docker exec ngb-rs-new test ! -f /mnt/shared/no-config-no-kit/repo/config || fail "no-config-no-kit: a config was written"
|
||||||
|
ok "no-config-no-kit: told the kit is needed, nothing written"
|
||||||
|
|
||||||
|
# no-config-kit
|
||||||
|
scenario no-config-kit
|
||||||
|
docker exec ngb-rs-new rm /mnt/shared/no-config-kit/repo/config
|
||||||
|
run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "no-config-kit: key import"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "no-config-kit without --repair-config" backup:target:add offsite $(location no-config-kit))
|
||||||
|
expect "$OUT" "--repair-config" "no-config-kit: points at --repair-config"
|
||||||
|
docker exec ngb-rs-new test ! -f /mnt/shared/no-config-kit/repo/config || fail "no-config-kit: a config was written without --repair-config"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-config-kit) --repair-config 2>&1) || { printf '%s\n' "$OUT"; fail "no-config-kit: --repair-config"; }
|
||||||
|
expect "$OUT" "written again" "no-config-kit"
|
||||||
|
expect "$OUT" "found and adopted" "no-config-kit"
|
||||||
|
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "no-config-kit: the snapshot is not listed after the repair"
|
||||||
|
run_on ngb-rs-new backup:browse offsite "$SNAP" >/dev/null 2>&1 || fail "no-config-kit: the snapshot cannot be read after the repair"
|
||||||
|
ok "no-config-kit: refused without --repair-config, config written again with it, the backups open"
|
||||||
|
|
||||||
|
# other-kit: the kit of another server
|
||||||
|
start ngb-rs-other
|
||||||
|
run_on ngb-rs-other backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "other: key init"
|
||||||
|
run_on ngb-rs-other backup:key:kit --output=/tmp/other.json >/dev/null
|
||||||
|
docker exec ngb-rs-other cat /tmp/other.json > "$SHARED/other.json"
|
||||||
|
docker rm -fv ngb-rs-other >/dev/null
|
||||||
|
scenario other-kit
|
||||||
|
docker exec ngb-rs-new rm /tmp/kit.json
|
||||||
|
docker exec -i ngb-rs-new sh -c 'cat > /tmp/other.json && chmod 644 /tmp/other.json' < "$SHARED/other.json"
|
||||||
|
run_on ngb-rs-new backup:key:import /tmp/other.json --passphrase-file=/tmp/pass >/dev/null || fail "other-kit: key import"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "other-kit" backup:target:add offsite $(location other-kit))
|
||||||
|
expect "$OUT" "different key" "other-kit"
|
||||||
|
expect "$OUT" "must be the old server's" "other-kit: points at the right kit"
|
||||||
|
[ -z "$(run_on ngb-rs-new backup:target:list 2>&1 | grep offsite || true)" ] || fail "other-kit: a location was left behind"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "other-kit repair" backup:target:add offsite $(location other-kit) --repair-config)
|
||||||
|
expect "$OUT" "leave out --repair-config" "other-kit: a config that is there is not repaired"
|
||||||
|
ok "other-kit: refused, told to use the old server's kit, the config untouched"
|
||||||
|
|
||||||
|
# empty
|
||||||
|
scenario empty
|
||||||
|
docker exec ngb-rs-new sh -c 'mkdir -p /mnt/shared/nothing && chown www-data /mnt/shared/nothing'
|
||||||
|
OUT=$(must_fail ngb-rs-new "empty" backup:target:add offsite local -a null::null -o datadir=/mnt/shared/nothing --path=repo --passphrase-file=/tmp/pass)
|
||||||
|
expect "$OUT" "no backups at this location" "empty"
|
||||||
|
expect "$OUT" "Check the connection details and --path" "empty"
|
||||||
|
no_key ngb-rs-new "empty"
|
||||||
|
ok "empty: told to check the details, no key made"
|
||||||
|
|
||||||
|
# unused-key-kit: a key made by mistake, then the kit
|
||||||
|
scenario unused-key-kit
|
||||||
|
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "unused-key-kit: key init"
|
||||||
|
MISTAKE=$(key_of ngb-rs-new)
|
||||||
|
OUT=$(must_fail ngb-rs-new "unused-key-kit without the option" backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass)
|
||||||
|
expect "$OUT" "never used" "unused-key-kit"
|
||||||
|
expect "$OUT" "--replace-unused-key" "unused-key-kit: names the option"
|
||||||
|
[ "$(key_of ngb-rs-new)" = "$MISTAKE" ] || fail "unused-key-kit: the key changed without the option"
|
||||||
|
OUT=$(must_fail ngb-rs-new "unused-key-kit wrong passphrase" backup:key:import /tmp/kit.json --passphrase-file=/tmp/wrongpass --replace-unused-key)
|
||||||
|
expect "$OUT" "Wrong passphrase" "unused-key-kit wrong passphrase"
|
||||||
|
[ "$(key_of ngb-rs-new)" = "$MISTAKE" ] || fail "unused-key-kit: a wrong passphrase changed the key"
|
||||||
|
OUT=$(run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass --replace-unused-key 2>&1) || { printf '%s\n' "$OUT"; fail "unused-key-kit: replace"; }
|
||||||
|
expect "$OUT" "The unused key was replaced" "unused-key-kit"
|
||||||
|
expect "$OUT" "fingerprint $FP" "unused-key-kit"
|
||||||
|
expect "$OUT" "21 days (from the kit)" "unused-key-kit"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location unused-key-kit) --no-interaction 2>&1) || { printf '%s\n' "$OUT"; fail "unused-key-kit: target add"; }
|
||||||
|
expect "$OUT" "found and adopted" "unused-key-kit"
|
||||||
|
run_on ngb-rs-new backup:run offsite >/dev/null || fail "unused-key-kit: a backup after the replacement"
|
||||||
|
ok "unused-key-kit: refused without the option, a wrong passphrase changes nothing, replaced with it, backups run"
|
||||||
|
|
||||||
|
# unused-key-no-kit: a key made by mistake, no kit
|
||||||
|
scenario unused-key-no-kit
|
||||||
|
docker exec ngb-rs-new rm /tmp/kit.json
|
||||||
|
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "unused-key-no-kit: key init"
|
||||||
|
MISTAKE=$(key_of ngb-rs-new)
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "unused-key-no-kit without the option" backup:target:add offsite $(location unused-key-no-kit) --no-interaction)
|
||||||
|
expect "$OUT" "different key" "unused-key-no-kit"
|
||||||
|
expect "$OUT" "never used" "unused-key-no-kit: says the key can make way"
|
||||||
|
expect "$OUT" "--replace-unused-key" "unused-key-no-kit: names the option"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "unused-key-no-kit wrong passphrase" backup:target:add offsite $(location unused-key-no-kit) --replace-unused-key --passphrase-file=/tmp/wrongpass)
|
||||||
|
expect "$OUT" "Nothing was changed" "unused-key-no-kit wrong passphrase"
|
||||||
|
[ "$(key_of ngb-rs-new)" = "$MISTAKE" ] || fail "unused-key-no-kit: a wrong passphrase changed the key"
|
||||||
|
[ -z "$(run_on ngb-rs-new backup:target:list 2>&1 | grep offsite || true)" ] || fail "unused-key-no-kit: a location was left behind"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location unused-key-no-kit) --replace-unused-key --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "unused-key-no-kit: replace"; }
|
||||||
|
expect "$OUT" "The unused key was replaced" "unused-key-no-kit"
|
||||||
|
expect "$OUT" "fingerprint $FP" "unused-key-no-kit"
|
||||||
|
expect "$OUT" "download a new recovery kit" "unused-key-no-kit: asks for a new kit"
|
||||||
|
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "unused-key-no-kit: the snapshot is not listed"
|
||||||
|
OUT=$(must_fail ngb-rs-new "unused-key-no-kit backup before a new kit" backup:run offsite)
|
||||||
|
expect "$OUT" "Confirm the recovery kit" "unused-key-no-kit: the old key's kit confirmation does not count"
|
||||||
|
ok "unused-key-no-kit: hint on the refusal, a wrong passphrase changes nothing, replaced with a passphrase, a new kit is needed"
|
||||||
|
|
||||||
|
# used-key: a key that protected a location stays, also after that location is removed
|
||||||
|
scenario used-key
|
||||||
|
docker exec ngb-rs-new sh -c 'mkdir -p /mnt/shared/own && chown www-data /mnt/shared/own'
|
||||||
|
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "used-key: key init"
|
||||||
|
OWN=$(key_of ngb-rs-new)
|
||||||
|
run_on ngb-rs-new backup:target:add own local -a null::null -o datadir=/mnt/shared/own --path=repo >/dev/null || fail "used-key: own location"
|
||||||
|
run_on ngb-rs-new backup:target:remove own >/dev/null || fail "used-key: remove the own location"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "used-key target:add" backup:target:add offsite $(location used-key) --replace-unused-key --passphrase-file=/tmp/pass)
|
||||||
|
expect "$OUT" "never replaced" "used-key target:add"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(must_fail ngb-rs-new "used-key target:add without the option" backup:target:add offsite $(location used-key) --no-interaction)
|
||||||
|
not_expect "$OUT" "--replace-unused-key" "used-key: offers a replacement for a used key"
|
||||||
|
OUT=$(must_fail ngb-rs-new "used-key key:import" backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass --replace-unused-key)
|
||||||
|
expect "$OUT" "never replaced" "used-key key:import"
|
||||||
|
[ "$(key_of ngb-rs-new)" = "$OWN" ] || fail "used-key: the key changed"
|
||||||
|
ok "used-key: never replaced, by kit or passphrase, also after its location was removed"
|
||||||
|
|
||||||
|
# slot-catch-up: a passphrase change that missed a location reaches it at the next backup
|
||||||
|
start ngb-rs-new
|
||||||
|
docker exec ngb-rs-new sh -c "printf '%s\n' 'catch up passphrase of the cto' > /tmp/pass2 && chmod 644 /tmp/pass2 && rm -rf /mnt/shared/catchup && mkdir -p /mnt/shared/catchup && chown www-data /mnt/shared/catchup"
|
||||||
|
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass --label=Safe >/dev/null || fail "slot-catch-up: key init"
|
||||||
|
CODE=$(run_on ngb-rs-new backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
run_on ngb-rs-new backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "slot-catch-up: confirm"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
run_on ngb-rs-new backup:target:add offsite $(location catchup) >/dev/null || fail "slot-catch-up: target add"
|
||||||
|
run_on ngb-rs-new backup:run offsite >/dev/null || fail "slot-catch-up: first backup"
|
||||||
|
BEFORE=$(docker exec ngb-rs-new md5sum /mnt/shared/catchup/repo/config | cut -d' ' -f1)
|
||||||
|
docker exec ngb-rs-new chmod 000 /mnt/shared/catchup/repo
|
||||||
|
OUT=$(must_fail ngb-rs-new "slot-catch-up: slot add with the location unreachable" backup:key:slot:add --label=CTO --passphrase-file=/tmp/pass2)
|
||||||
|
expect "$OUT" "tried again at their next backup" "slot-catch-up: says the change follows"
|
||||||
|
docker exec ngb-rs-new chmod 755 /mnt/shared/catchup/repo
|
||||||
|
[ "$(docker exec ngb-rs-new md5sum /mnt/shared/catchup/repo/config | cut -d' ' -f1)" = "$BEFORE" ] || fail "slot-catch-up: the location changed while unreachable"
|
||||||
|
CODE=$(run_on ngb-rs-new backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
run_on ngb-rs-new backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "slot-catch-up: confirm the new kit"
|
||||||
|
run_on ngb-rs-new backup:run offsite >/dev/null || fail "slot-catch-up: backup after the location came back"
|
||||||
|
[ "$(docker exec ngb-rs-new md5sum /mnt/shared/catchup/repo/config | cut -d' ' -f1)" != "$BEFORE" ] || fail "slot-catch-up: the backup did not bring the new passphrase to the location"
|
||||||
|
CATCHFP=$(key_of ngb-rs-new)
|
||||||
|
start ngb-rs-other
|
||||||
|
docker exec ngb-rs-other sh -c "printf '%s\n' 'catch up passphrase of the cto' > /tmp/pass2 && chmod 644 /tmp/pass2"
|
||||||
|
# shellcheck disable=SC2046
|
||||||
|
OUT=$(run_on ngb-rs-other backup:target:add offsite $(location catchup) --passphrase-file=/tmp/pass2 2>&1) || { printf '%s\n' "$OUT"; fail "slot-catch-up: a new server with the new passphrase"; }
|
||||||
|
expect "$OUT" "fingerprint $CATCHFP" "slot-catch-up: the new server"
|
||||||
|
ok "slot-catch-up: a passphrase that missed a location reaches it at the next backup and opens it on a new server"
|
||||||
|
|
||||||
|
echo "RECOVERY SCENARIOS PASS"
|
||||||
@@ -23,12 +23,15 @@ ok "key, kit code and confirmation"
|
|||||||
occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add"
|
occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add"
|
||||||
ok "location added"
|
ok "location added"
|
||||||
|
|
||||||
# Test data: a user with files.
|
# Test data: a user with files. No skeleton files on the first login, below, in the middle of the checks.
|
||||||
|
occ config:system:set skeletondirectory --value='' >/dev/null
|
||||||
|
occ config:system:set templatedirectory --value='' >/dev/null
|
||||||
$X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
|
$X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
|
||||||
# large.bin (40 MB) fills more than one pack: a restore to a directory reads ahead over several ranges
|
# large.bin (40 MB) fills more than one pack: a restore to a directory reads ahead over several ranges
|
||||||
$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin; head -c 40000000 /dev/urandom > $DATA/$U/files/large.bin"
|
$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs/Empty/Nested $DATA/$U/files/Hollow && touch -d @1700000000 $DATA/$U/files/Hollow && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin; head -c 40000000 /dev/urandom > $DATA/$U/files/large.bin"
|
||||||
|
$X sh -c "touch -d @1600000000 $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs"
|
||||||
occ files:scan "$U" >/dev/null
|
occ files:scan "$U" >/dev/null
|
||||||
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||||
|
|
||||||
occ backup:run ci -l first >/dev/null || fail "first backup"
|
occ backup:run ci -l first >/dev/null || fail "first backup"
|
||||||
SECOND=$(occ backup:run ci -l second)
|
SECOND=$(occ backup:run ci -l second)
|
||||||
@@ -39,14 +42,29 @@ SNAP=$(occ backup:list ci | awk -F'|' '!f && / second /{gsub(/ /,"",$3); print $
|
|||||||
[ -n "$SNAP" ] || fail "snapshot not listed"
|
[ -n "$SNAP" ] || fail "snapshot not listed"
|
||||||
|
|
||||||
# Damage, then restore in each mode.
|
# Damage, then restore in each mode.
|
||||||
$X sh -c "cd $DATA/$U/files && rm Docs/Sub/note.txt && echo broken > Docs/f1.bin && echo extra > Docs/extra.txt"
|
$X sh -c "cd $DATA/$U/files && rm Docs/Sub/note.txt && echo broken > Docs/f1.bin && echo extra > Docs/extra.txt && rm -r Docs/Empty && rmdir Hollow"
|
||||||
occ files:scan "$U" >/dev/null
|
occ files:scan "$U" >/dev/null
|
||||||
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m new-folder >/dev/null || fail "restore new-folder"
|
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m new-folder >/dev/null || fail "restore new-folder"
|
||||||
$X sh -c "ls $DATA/$U/files | grep -q '^Restored '" || fail "no Restored folder"
|
$X sh -c "ls $DATA/$U/files | grep -q '^Restored '" || fail "no Restored folder"
|
||||||
|
$X sh -c "test -d $DATA/$U/files/Restored\ */Docs/Empty/Nested" || fail "the empty folder is not in the Restored folder"
|
||||||
|
# Folders with files: on disk and in what Nextcloud shows, the parent as well as the child.
|
||||||
|
RDIR=$($X sh -c "cd $DATA/$U/files && ls -d Restored\ *")
|
||||||
|
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs/Sub")" = 1600000000 ] || fail "a folder with files did not get its time back"
|
||||||
|
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs")" = 1600000000 ] || fail "the parent folder did not get its time back"
|
||||||
|
DAV=$($X curl -sf -u "$U:Roundtrip-pw-2026" -X PROPFIND -H 'Depth: 1' "http://localhost/remote.php/dav/files/$U/$(php -r 'echo rawurlencode($argv[1]);' "$RDIR")/Docs/") || fail "PROPFIND"
|
||||||
|
[ "$(echo "$DAV" | grep -o 'Sun, 13 Sep 2020 12:26:40 GMT' | wc -l)" = 2 ] || fail "Nextcloud does not show the time of the folders: $(echo "$DAV" | grep -o '<d:getlastmodified>[^<]*' | sort | uniq -c)"
|
||||||
|
occ backup:restore ci "$SNAP" "data/$U/files/Docs/Sub" -m merge >/dev/null || fail "restore into a folder that is there"
|
||||||
|
[ "$($X stat -c %Y "$DATA/$U/files/Docs/Sub")" != 1600000000 ] || fail "a folder that was already there got the old time"
|
||||||
|
occ backup:browse ci "$SNAP" "data/$U/files" | grep Hollow >/dev/null || fail "an empty folder is not shown when browsing the snapshot"
|
||||||
|
occ backup:restore ci "$SNAP" "data/$U/files/Hollow" -m merge >/dev/null || fail "restore of a folder that was empty"
|
||||||
|
$X sh -c "test -d $DATA/$U/files/Hollow" || fail "the empty folder did not come back by itself"
|
||||||
|
[ "$($X stat -c %Y "$DATA/$U/files/Hollow")" = 1700000000 ] || fail "the empty folder did not get its time back"
|
||||||
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m replace >/dev/null || fail "restore replace"
|
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m replace >/dev/null || fail "restore replace"
|
||||||
|
$X sh -c "test -d $DATA/$U/files/Docs/Empty/Nested" || fail "replace did not bring back the empty folder, or moved it to the trash"
|
||||||
|
ok "empty folders: restored in each mode, by themselves and shown when browsing"
|
||||||
$X sh -c "rm -rf '$DATA/$U/files/Restored '*" ; occ files:scan "$U" >/dev/null
|
$X sh -c "rm -rf '$DATA/$U/files/Restored '*" ; occ files:scan "$U" >/dev/null
|
||||||
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore"
|
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore: $(diff <(echo "$SUM_BEFORE") <(echo "$SUM_AFTER") | cut -c1-120)"
|
||||||
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
||||||
ok "restore new-folder and replace: files identical, extra file in the trash"
|
ok "restore new-folder and replace: files identical, extra file in the trash"
|
||||||
|
|
||||||
@@ -60,9 +78,10 @@ ok "restore of one file to a directory; the temporary of a stopped restore remov
|
|||||||
|
|
||||||
# All of the user's files to a directory: over S3 and SFTP the packs are read ahead
|
# All of the user's files to a directory: over S3 and SFTP the packs are read ahead
|
||||||
occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-dir >/dev/null || fail "restore of a folder to a directory"
|
occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-dir >/dev/null || fail "restore of a folder to a directory"
|
||||||
SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||||
[ "$SUM_BEFORE" = "$SUM_DIR" ] || fail "the files restored to a directory differ"
|
[ "$SUM_BEFORE" = "$SUM_DIR" ] || fail "the files restored to a directory differ"
|
||||||
$X sh -c "ls -A /tmp/ngb-rt-dir | grep -q ngb-restore-spool" && fail "the read-ahead spool was left in the directory"
|
$X sh -c "ls -A /tmp/ngb-rt-dir | grep -q ngb-restore-spool" && fail "the read-ahead spool was left in the directory"
|
||||||
|
$X sh -c "test -d /tmp/ngb-rt-dir/data/$U/files/Docs/Empty/Nested" || fail "the empty folder is not in the directory"
|
||||||
$X sh -c "rm -rf /tmp/ngb-rt-dir"
|
$X sh -c "rm -rf /tmp/ngb-rt-dir"
|
||||||
ok "restore of all the user's files to a directory: identical, no spool left"
|
ok "restore of all the user's files to a directory: identical, no spool left"
|
||||||
# not grep -q: it stops reading at the first match, and occ, still writing, then fails (pipefail)
|
# not grep -q: it stops reading at the first match, and occ, still writing, then fails (pipefail)
|
||||||
@@ -90,13 +109,101 @@ done
|
|||||||
[ "$RS" = done ] || fail "the backup started through the API ended as '$RS' after $i ticks"
|
[ "$RS" = done ] || fail "the backup started through the API ended as '$RS' after $i ticks"
|
||||||
occ backup:list ci | grep " page " >/dev/null || fail "the snapshot of the ticked backup is not listed"
|
occ backup:list ci | grep " page " >/dev/null || fail "the snapshot of the ticked backup is not listed"
|
||||||
ok "without system cron: warning in backup:status and the API, backup finished by $i ticks of the open page"
|
ok "without system cron: warning in backup:status and the API, backup finished by $i ticks of the open page"
|
||||||
|
# The Bin reads in parts, as the page does (what is in the parts: CatalogTest)
|
||||||
|
api "$API/targets/$TID/trash" | php -r '$p = json_decode(stream_get_contents(STDIN), true); exit(is_int($p["total"] ?? null) && is_array($p["items"] ?? null) && count($p["items"]) <= $p["total"] ? 0 : 1);' || fail "the Bin API does not give a total and items"
|
||||||
|
ok "the Bin API reads in parts, with a total"
|
||||||
|
# Locations are backed up one after the other: the second one started is queued until the first is done.
|
||||||
|
for L in 2 3; do
|
||||||
|
$X sh -c "mkdir -p /tmp/ngb-rt-target$L"
|
||||||
|
occ backup:target:add ci$L local -a null::null -o datadir=/tmp/ngb-rt-target$L --path=repo >/dev/null || fail "target add ci$L"
|
||||||
|
done
|
||||||
|
IDS=$(api "$API/status" | php -r 'foreach (json_decode(stream_get_contents(STDIN), true)["targets"] as $t) { if ($t["name"] === "ci2" || $t["name"] === "ci3") { echo $t["id"], " "; } }')
|
||||||
|
for ID in $IDS; do api -X POST -d "{\"targetId\": $ID, \"label\": \"turn\"}" "$API/backups" >/dev/null || fail "start a backup to location $ID"; done
|
||||||
|
# Never two runs to one location: a second start gets the run that is there.
|
||||||
|
ID=${IDS%% *}
|
||||||
|
RUNS=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||||
|
api -X POST -d "{\"targetId\": $ID, \"label\": \"again\"}" "$API/backups" >/dev/null || fail "second start"
|
||||||
|
AGAIN=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||||
|
[ "$RUNS" = 2 ] && [ "$AGAIN" = 2 ] || fail "a second start to one location: $RUNS, then $AGAIN running"
|
||||||
|
occ backup:status | grep "queued" >/dev/null || fail "the second backup is not queued"
|
||||||
|
QUEUED=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["queued"]));')
|
||||||
|
[ "$QUEUED" = 1 ] || fail "$QUEUED backups queued instead of 1"
|
||||||
|
for i in $(seq 1 60); do
|
||||||
|
api -X POST "$API/tick" >/dev/null || fail "tick"
|
||||||
|
RUNNING=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||||
|
[ "$RUNNING" != 0 ] || break
|
||||||
|
done
|
||||||
|
[ "$RUNNING" = 0 ] || fail "the queued backups did not finish"
|
||||||
|
occ backup:list ci3 | grep " turn " >/dev/null || fail "the queued backup made no snapshot"
|
||||||
|
ok "locations in turn: the second backup queued, and done after the first"
|
||||||
|
# Heavy background jobs of other apps wait during a backup, and only then. Ticks keep this one moving.
|
||||||
|
HEAVY='OCA\Recognize\BackgroundJobs\NgbTestJob'
|
||||||
|
reserved() { $X php -r 'require "lib/base.php"; $q = \OC::$server->get(\OCP\IDBConnection::class)->getQueryBuilder(); echo (int)$q->select("reserved_at")->from("jobs")->where($q->expr()->eq("class", $q->createNamedParameter($argv[1])))->executeQuery()->fetchOne();' "$HEAVY"; }
|
||||||
|
tick_until_done() {
|
||||||
|
for i in $(seq 1 60); do
|
||||||
|
api -X POST "$API/tick" >/dev/null || fail "tick"
|
||||||
|
RUNNING=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||||
|
[ "$RUNNING" != 0 ] || return 0
|
||||||
|
done
|
||||||
|
fail "the backup did not finish"
|
||||||
|
}
|
||||||
|
$X php -r 'require "lib/base.php"; \OC::$server->get(\OCP\BackgroundJob\IJobList::class)->add($argv[1]);' "$HEAVY" || fail "add a job"
|
||||||
|
[ "$(reserved)" = 0 ] || fail "a new job is reserved"
|
||||||
|
L3=$(echo "$IDS" | awk '{print $2}')
|
||||||
|
api -X POST -d "{\"targetId\": $L3, \"label\": \"heavy\"}" "$API/backups" >/dev/null || fail "start a backup"
|
||||||
|
tick_until_done
|
||||||
|
[ "$(reserved)" != 0 ] || fail "the job was not paused during the backup"
|
||||||
|
api -X POST "$API/tick" >/dev/null
|
||||||
|
[ "$(reserved)" = 0 ] || fail "the job stays paused after the backup"
|
||||||
|
occ backup:pause-jobs off >/dev/null || fail "turn off"
|
||||||
|
sleep 8 # APCu app config cache of the web server
|
||||||
|
api -X POST -d "{\"targetId\": $L3, \"label\": \"heavy\"}" "$API/backups" >/dev/null || fail "start a backup"
|
||||||
|
tick_until_done
|
||||||
|
[ "$(reserved)" = 0 ] || fail "the job was paused with the setting off"
|
||||||
|
occ backup:pause-jobs on >/dev/null
|
||||||
|
$X php -r 'require "lib/base.php"; \OC::$server->get(\OCP\BackgroundJob\IJobList::class)->remove($argv[1]);' "$HEAVY"
|
||||||
|
ok "heavy background jobs: paused during a backup, back after it, left alone with the setting off"
|
||||||
occ background:cron >/dev/null
|
occ background:cron >/dev/null
|
||||||
$X php -f /var/www/html/cron.php >/dev/null 2>&1 || true
|
$X php -f /var/www/html/cron.php >/dev/null 2>&1 || true
|
||||||
occ backup:status | grep "system cron" >/dev/null && fail "backup:status still warns after cron ran"
|
occ backup:status | grep "system cron" >/dev/null && fail "backup:status still warns after cron ran"
|
||||||
sleep 8 # APCu app config cache of the web server
|
sleep 8 # APCu app config cache of the web server
|
||||||
api "$API/status" | php -r 'exit(json_decode(stream_get_contents(STDIN), true)["cron"]["ok"] ? 0 : 1);' || fail "the status API still reports a missing cron after cron ran"
|
api "$API/status" | php -r 'exit(json_decode(stream_get_contents(STDIN), true)["cron"]["ok"] ? 0 : 1);' || fail "the status API still reports a missing cron after cron ran"
|
||||||
ok "with a system cron that ran: no warning in backup:status or the API"
|
ok "with a system cron that ran: no warning in backup:status or the API"
|
||||||
|
|
||||||
|
# A schedule per location. Choosing a time starts nothing; the moment itself does, and only there.
|
||||||
|
JOB=$(occ background-job:list --class='OCA\NgBackup\BackgroundJob\BackupJob' --output=json | php -r 'echo json_decode(stream_get_contents(STDIN), true)[0]["id"];')
|
||||||
|
job() { occ background-job:execute "$JOB" --force-execute >/dev/null; }
|
||||||
|
occ backup:target:schedule ci off >/dev/null
|
||||||
|
occ backup:target:schedule ci3 off >/dev/null
|
||||||
|
occ backup:target:schedule ci2 00:00 --timezone=UTC >/dev/null || fail "set a schedule"
|
||||||
|
occ backup:target:schedule ci2 | grep "00:00 (UTC)" >/dev/null || fail "schedule not shown"
|
||||||
|
job
|
||||||
|
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 0 ] || fail "choosing a time started a backup"
|
||||||
|
occ backup:target:schedule ci2 "$($X date -u -d '+1 minute' +%H:%M)/6" --timezone=UTC >/dev/null || fail "set every 6 hours"
|
||||||
|
sleep 65
|
||||||
|
job
|
||||||
|
occ backup:list ci2 | grep " scheduled " >/dev/null || fail "the scheduled moment started no backup"
|
||||||
|
[ "$(occ backup:list ci3 | grep -c ' scheduled ')" = 0 ] || fail "a location that is off was backed up"
|
||||||
|
job
|
||||||
|
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 1 ] || fail "one moment started more than one backup"
|
||||||
|
occ backup:target:schedule ci2 nonsense >/dev/null 2>&1 && fail "a bad schedule was accepted"
|
||||||
|
ok "schedule per location: nothing on choosing a time, one backup at the moment, none when off"
|
||||||
|
|
||||||
|
# What administrators are told, as Nextcloud shows it.
|
||||||
|
notes() { api "http://localhost/ocs/v2.php/apps/notifications/api/v2/notifications?format=json"; }
|
||||||
|
occ backup:target:schedule ci2 off >/dev/null
|
||||||
|
$X sh -c "touch '$DATA/ngb-rt-bad\\name.txt'"
|
||||||
|
occ backup:run ci3 -l names | grep "their name is not supported" >/dev/null || fail "occ does not report the name"
|
||||||
|
$X sh -c "rm '$DATA/ngb-rt-bad\\name.txt'"
|
||||||
|
NOTES=$(notes) || fail "read the notifications"
|
||||||
|
echo "$NOTES" | grep "turned off the schedule of backup location ci2" >/dev/null || fail "no notification about the schedule that was turned off"
|
||||||
|
echo "$NOTES" | grep "is missing because of its name" >/dev/null || fail "no notification about the name"
|
||||||
|
echo "$NOTES" | grep "Rename them" >/dev/null || fail "the notification about the name does not ask to rename"
|
||||||
|
job
|
||||||
|
notes | grep "has no schedule" >/dev/null && fail "a reminder right after turning it off"
|
||||||
|
for L in 2 3; do occ backup:target:remove ci$L >/dev/null || fail "target remove ci$L"; done
|
||||||
occ user:delete "$ADM" >/dev/null
|
occ user:delete "$ADM" >/dev/null
|
||||||
|
ok "notifications: schedule turned off, name not supported, no early reminder"
|
||||||
|
|
||||||
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
|
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
|
||||||
occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify"
|
occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify"
|
||||||
@@ -109,6 +216,17 @@ ok "database dump and restore"
|
|||||||
|
|
||||||
# Retention, trash and catalog.
|
# Retention, trash and catalog.
|
||||||
occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null
|
occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null
|
||||||
|
# A pinned restore point stays, and a note can change afterwards.
|
||||||
|
FIRST=$(occ backup:list ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
|
||||||
|
occ backup:pin ci "$FIRST" >/dev/null || fail "pin"
|
||||||
|
occ backup:note ci "$FIRST" "a note made later" >/dev/null || fail "note"
|
||||||
|
occ backup:list ci | grep " first " | grep " yes " | grep "a note made later" >/dev/null || fail "pin and note not listed"
|
||||||
|
occ backup:prune ci --dry-run | grep "would forget 1 " >/dev/null || fail "a pinned snapshot would be removed"
|
||||||
|
occ backup:pin ci "$FIRST" --unpin >/dev/null || fail "unpin"
|
||||||
|
occ backup:prune ci --dry-run | grep "would forget 2 " >/dev/null || fail "an unpinned snapshot would stay"
|
||||||
|
occ backup:note ci "$FIRST" >/dev/null || fail "remove note"
|
||||||
|
occ backup:list ci | grep -q "a note made later" && fail "note not removed"
|
||||||
|
ok "pin and note on a restore point"
|
||||||
occ backup:prune ci | grep "forgot 2" >/dev/null || fail "prune" # first and second; "page" (above) is the last
|
occ backup:prune ci | grep "forgot 2" >/dev/null || fail "prune" # first and second; "page" (above) is the last
|
||||||
occ backup:trash ci | grep " first " >/dev/null || fail "trash"
|
occ backup:trash ci | grep " first " >/dev/null || fail "trash"
|
||||||
TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
|
TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
|
||||||
|
|||||||
@@ -0,0 +1,109 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# The update from a released version to this working tree, on a server that was backing up.
|
||||||
|
# Usage: upgrade-from-release.sh <release package .tar.gz> [nextcloud-image-tag] [sqlite|pgsql] [new-version]
|
||||||
|
# e.g. upgrade-from-release.sh ~/dev-local/release/ng_backup-1.0.3/ng_backup-1.0.3.tar.gz 34-apache pgsql 1.1.0
|
||||||
|
#
|
||||||
|
# The old version gets a key, a confirmed kit, a location with a daily time and one backup. Then the
|
||||||
|
# app files are replaced by this working tree (version raised to new-version when given) and
|
||||||
|
# occ upgrade runs. After it: the location keeps its time as its own schedule, the old restore point
|
||||||
|
# is listed, verifies and restores a file byte for byte, a new backup runs, the location gets its
|
||||||
|
# settings file, and the key counts as used.
|
||||||
|
set -euo pipefail
|
||||||
|
PKG=${1:?release package}
|
||||||
|
IMAGE="nextcloud:${2:-34-apache}"
|
||||||
|
DB=${3:-sqlite}
|
||||||
|
NEWVER=${4:-}
|
||||||
|
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
||||||
|
N=ngb-up; NET=ngb-up-net
|
||||||
|
SHARED=$(mktemp -d)
|
||||||
|
NEWAPP=$(mktemp -d)
|
||||||
|
chmod 777 "$SHARED"
|
||||||
|
PASS='upgrade test passphrase 2026'
|
||||||
|
fail() { echo "FAIL: $*"; exit 1; }
|
||||||
|
ok() { echo "ok $*"; }
|
||||||
|
cleanup() {
|
||||||
|
docker rm -fv $N $N-db >/dev/null 2>&1 || true
|
||||||
|
docker network rm $NET >/dev/null 2>&1 || true
|
||||||
|
docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
|
||||||
|
rm -rf "$NEWAPP"
|
||||||
|
rmdir "$SHARED" 2>/dev/null || true
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
occ() { docker exec -u www-data $N php occ "$@"; }
|
||||||
|
|
||||||
|
docker network create $NET >/dev/null
|
||||||
|
env=(-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin-upgrade-pw)
|
||||||
|
case $DB in
|
||||||
|
sqlite) env+=(-e SQLITE_DATABASE=nextcloud) ;;
|
||||||
|
pgsql) docker run -d --name $N-db --network $NET -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nc -e POSTGRES_PASSWORD=nc postgres:16-alpine >/dev/null
|
||||||
|
env+=(-e POSTGRES_HOST=$N-db -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nc -e POSTGRES_PASSWORD=nc)
|
||||||
|
sleep 8 ;;
|
||||||
|
*) fail "unknown database $DB" ;;
|
||||||
|
esac
|
||||||
|
docker run -d --name $N --network $NET "${env[@]}" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
|
||||||
|
for i in $(seq 1 90); do occ status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
|
||||||
|
docker exec $N chown www-data:www-data /var/www/html/custom_apps
|
||||||
|
docker exec -i -u www-data $N tar xz -C /var/www/html/custom_apps < "$PKG"
|
||||||
|
OLDVER=$(docker exec $N sed -n 's#.*<version>\(.*\)</version>.*#\1#p' /var/www/html/custom_apps/ng_backup/appinfo/info.xml)
|
||||||
|
occ app:enable ng_backup >/dev/null || fail "enable $OLDVER"
|
||||||
|
ok "NG Backup $OLDVER on $(occ -V) / $DB"
|
||||||
|
|
||||||
|
# The old version at work: key, kit, a location with a daily time, a user file, one backup
|
||||||
|
docker exec $N sh -c "printf '%s\n' '$PASS' > /tmp/pass; chmod 644 /tmp/pass; mkdir -p /mnt/shared/loc; chown www-data /mnt/shared/loc"
|
||||||
|
occ backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "key init"
|
||||||
|
occ backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||||
|
CODE=$(occ backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||||
|
occ backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||||
|
occ config:app:set ng_backup schedule_time --value=03:15 >/dev/null
|
||||||
|
occ config:app:set ng_backup schedule_timezone --value=America/Toronto >/dev/null
|
||||||
|
occ backup:target:add offsite local -a null::null -o datadir=/mnt/shared/loc --path=repo >/dev/null || fail "target add"
|
||||||
|
docker exec -u www-data $N sh -c 'mkdir -p data/admin/files/Docs && head -c 3000000 /dev/urandom > data/admin/files/Docs/before.bin'
|
||||||
|
occ files:scan admin >/dev/null
|
||||||
|
SUM=$(docker exec $N sha256sum /var/www/html/data/admin/files/Docs/before.bin | cut -d' ' -f1)
|
||||||
|
occ backup:run offsite >/dev/null || fail "backup with $OLDVER"
|
||||||
|
OLDSNAP=$(occ backup:list offsite | grep -o '[0-9a-f]\{32\}' | head -1)
|
||||||
|
[ -n "$OLDSNAP" ] || fail "no restore point with $OLDVER"
|
||||||
|
ok "$OLDVER: key, kit, location at 03:15, restore point $OLDSNAP"
|
||||||
|
|
||||||
|
# The update: this working tree's runtime files in place of the old ones
|
||||||
|
mkdir -p "$NEWAPP/ng_backup/js"
|
||||||
|
cp -r "$APP/appinfo" "$APP/lib" "$APP/templates" "$APP/img" "$APP/css" "$NEWAPP/ng_backup/"
|
||||||
|
cp "$APP"/js/*.mjs "$NEWAPP/ng_backup/js/"
|
||||||
|
[ -n "$NEWVER" ] && sed -i "s#<version>.*</version>#<version>$NEWVER</version>#" "$NEWAPP/ng_backup/appinfo/info.xml"
|
||||||
|
docker exec -u www-data $N rm -rf /var/www/html/custom_apps/ng_backup
|
||||||
|
tar c -C "$NEWAPP" ng_backup | docker exec -i -u www-data $N tar x -C /var/www/html/custom_apps
|
||||||
|
OUT=$(occ upgrade 2>&1) || { printf '%s\n' "$OUT"; fail "occ upgrade"; }
|
||||||
|
VER=$(occ app:list --output=json | php -r '$a = json_decode(stream_get_contents(STDIN), true); echo $a["enabled"]["ng_backup"] ?? "";')
|
||||||
|
[ -z "$NEWVER" ] || [ "$VER" = "$NEWVER" ] || fail "after the upgrade the app has version '$VER'"
|
||||||
|
ok "occ upgrade to $VER"
|
||||||
|
# Before anything else touches the key: only the migration can have marked it as used
|
||||||
|
[ "$(occ config:app:get ng_backup master_key_used 2>/dev/null)" = 1 ] || [ "$(occ config:app:get ng_backup master_key_used 2>/dev/null)" = true ] || fail "the migration did not mark the key from before the upgrade as used"
|
||||||
|
ok "the key from before the upgrade counts as used"
|
||||||
|
|
||||||
|
SCHED=$(occ backup:target:schedule offsite)
|
||||||
|
printf '%s\n' "$SCHED" | grep -qx "offsite: 03:15 (America/Toronto)" || { printf '%s\n' "$SCHED"; fail "the location did not keep 03:15 as its schedule"; }
|
||||||
|
occ backup:list offsite | grep -q "$OLDSNAP" || fail "the old restore point is not listed"
|
||||||
|
occ backup:verify offsite "$OLDSNAP" >/dev/null || fail "the old restore point does not verify"
|
||||||
|
occ backup:status >/dev/null || fail "backup:status"
|
||||||
|
docker exec $N sh -c 'rm -rf /tmp/rst && mkdir /tmp/rst && chown www-data /tmp/rst'
|
||||||
|
occ backup:restore offsite "$OLDSNAP" data/admin/files/Docs/before.bin --to-directory=/tmp/rst >/dev/null || fail "restore from the old restore point"
|
||||||
|
GOT=$(docker exec $N sh -c 'find /tmp/rst -name before.bin -exec sha256sum {} \;' | cut -d' ' -f1)
|
||||||
|
[ "$GOT" = "$SUM" ] || fail "the restored file differs"
|
||||||
|
ok "after the upgrade: schedule kept, old restore point listed, verified, a file restored byte for byte"
|
||||||
|
|
||||||
|
docker exec -u www-data $N sh -c 'head -c 2000000 /dev/urandom > data/admin/files/Docs/after.bin'
|
||||||
|
occ files:scan admin >/dev/null
|
||||||
|
occ backup:run offsite >/dev/null || fail "backup after the upgrade"
|
||||||
|
NEWSNAP=$(occ backup:list offsite | grep -o '[0-9a-f]\{32\}' | grep -v "$OLDSNAP" | head -1)
|
||||||
|
[ -n "$NEWSNAP" ] || fail "no new restore point after the upgrade"
|
||||||
|
SUM2=$(docker exec $N sha256sum /var/www/html/data/admin/files/Docs/after.bin | cut -d' ' -f1)
|
||||||
|
docker exec $N sh -c 'rm -rf /tmp/rst && mkdir /tmp/rst && chown www-data /tmp/rst'
|
||||||
|
occ backup:restore offsite "$NEWSNAP" data/admin/files/Docs/after.bin --to-directory=/tmp/rst >/dev/null || fail "restore from the new restore point"
|
||||||
|
GOT=$(docker exec $N sh -c 'find /tmp/rst -name after.bin -exec sha256sum {} \;' | cut -d' ' -f1)
|
||||||
|
[ "$GOT" = "$SUM2" ] || fail "the file restored from the new restore point differs"
|
||||||
|
docker exec $N test -f /mnt/shared/loc/repo/settings || fail "the location got no settings file"
|
||||||
|
OUT=$(occ backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass --replace-unused-key 2>&1) && fail "a used key from before the upgrade was offered for replacement"
|
||||||
|
printf '%s\n' "$OUT" | grep -q "never replaced" || { printf '%s\n' "$OUT"; fail "the refusal does not say the key is never replaced"; }
|
||||||
|
ok "a new backup runs and restores, the location got its settings, the key counts as used"
|
||||||
|
echo "UPGRADE $OLDVER -> ${VER:-working tree} PASS"
|
||||||