Merge pull request #31 from dillardblom/1.1.0
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
Release 1.1.0
This commit is contained in:
Dillard Blom
@@ -11,6 +11,60 @@ Each heading starts with that `<version>`, once: the app store shows a release t
|
||||
whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
|
||||
shown under 1.0.0. The git tag of a release candidate goes after the date.
|
||||
|
||||
## [1.1.0] - 2026-11-09
|
||||
|
||||
### Added
|
||||
- Pin a restore point, on the restore page or with `occ backup:pin`: the retention policy leaves it
|
||||
alone, for example the one from before an upgrade. Unpinning notifies all administrators.
|
||||
- A note on a restore point that you can change afterwards, on the restore page or with
|
||||
`occ backup:note`. Pins and notes are stored encrypted on the location, so a new server sees them.
|
||||
An older version of NG Backup can still read the location, but drops them when it writes to it.
|
||||
- A schedule per location: off, once a day, or every N hours from a given time, on the page under
|
||||
Backups or with `occ backup:target:schedule`. After the update every location keeps the time that
|
||||
held for all of them; a new location starts with the schedule of the first one. A schedule is
|
||||
shown and changed in the time zone it was saved in, which the page names next to the time.
|
||||
- All folders get their original modification time back on a restore, not only the empty ones, from
|
||||
the first backup made with this version. An older version of NG Backup can still read new restore
|
||||
points. A restore through Nextcloud into a folder that is already there leaves its time alone.
|
||||
- Heavy background jobs (Memories, Recognize, Preview Generator, full text search, the antivirus
|
||||
background scan) pause while a backup runs, at most 6 hours in a row, then they get an hour. No app
|
||||
is turned off, and a job that is running goes on. On by default; turn it off under Backups or with
|
||||
`occ backup:pause-jobs off`.
|
||||
- Disaster recovery without the recovery kit: on a new server without a backup key,
|
||||
`occ backup:target:add` finds the backups on the location and takes their key from there with one
|
||||
of the old server's passphrases. Backups then wait until a new recovery kit is confirmed. The
|
||||
deletion delay is now also stored on the location, sealed with the key, from its first backup
|
||||
with this version; a location backed up only by an older version gives the default of 7 days.
|
||||
When the settings on the location cannot be read, the longest delay (365 days) is taken, so
|
||||
nothing leaves the Bin sooner than on the old server; the command says so.
|
||||
- A passphrase change that cannot reach a location (offline at the time) now reaches it at the
|
||||
location's next backup, so a new server can open it with the new passphrase.
|
||||
- `occ backup:target:add --repair-config`: when the config file of a location is gone, the recovery
|
||||
kit writes it again (for locations backed up by this version or newer). Before, the message said
|
||||
this was possible, but it was not.
|
||||
- The key page warns against creating a new key when you are restoring a server. A key made there
|
||||
by mistake can still make way for the key of the backups, as long as it was never used (no
|
||||
location, no backup): `occ backup:key:import --replace-unused-key` with the kit, or
|
||||
`occ backup:target:add --replace-unused-key` with a passphrase of the old server. A key that
|
||||
protected a location is never replaced, and neither is a key from before this version. All
|
||||
administrators are notified of a replacement.
|
||||
|
||||
### Changed
|
||||
- The list of backups stays within the page: a long label, note or error wraps onto more lines
|
||||
instead of making the table scroll sideways.
|
||||
- Reading files for a backup is much faster on disks where every request waits, such as network and
|
||||
cloud volumes: files are read in pieces of 4 MiB instead of 8 KiB. On a test server the disk
|
||||
delivered 27 MiB/s instead of 5 MiB/s.
|
||||
- Locations are backed up one after the other: a backup that has to wait shows as queued. A restore
|
||||
point has the time its backup really began, not the time it was queued.
|
||||
- Choosing a time no longer starts a backup right away when that time has already passed today.
|
||||
- A location without a schedule no longer gets the notification that it has had no recent backup.
|
||||
Instead, all administrators are told when a schedule is turned off, and reminded once a week while
|
||||
another location is still backed up, or every day when none is.
|
||||
- A file or folder that is left out because of its name now gets a notification of its own, in the
|
||||
log and in `occ backup:run`. It says the item is missing from the backup, gives example paths and
|
||||
asks to rename it; before, it was reported as unreadable, with advice about permissions.
|
||||
|
||||
## [1.0.3] - 2026-11-06
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -472,9 +472,12 @@ in README "Known limitations":
|
||||
tijd terug. Hoort zo, vooral bij oudere verzamelingen: de maptijd wordt gebruikt om in te schatten
|
||||
wanneer een map is aangemaakt. De backup moet dan de tijd van alle mappen vastleggen en de restore
|
||||
die aan het eind terugzetten; werkt pas vanaf de eerste backup met die versie. Op de roadmap onder 1.1.0.
|
||||
**1.0.3 uitgebracht (2026-11-06)**, ook op nc01 geïnstalleerd.
|
||||
- **Branches (besluit 2026-11-06):** werk voor een versie op een branch met die naam (`1.0.3`);
|
||||
`polish` alleen voor kleine cosmetische zaken, per geval bekijken of en wanneer die gemerged
|
||||
worden. SMB staat uit op sub5 en sub6; sub6 is het testaccount, eerst aanzetten voor een SMB-test.
|
||||
Na 1.0.3 gaat het werk naar `1.1.0`. Een spoedfix die niet kan wachten gaat op `1.0.4` vanaf `main`,
|
||||
krijgt een eigen merge en release; daarna `main` in `1.1.0` mergen, zodat daar met de fix getest wordt.
|
||||
- **1.0.1** (uitgebracht 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
|
||||
1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november,
|
||||
niet aangekondigd en los van de release. Voor 1.0.2 ook: nieuwe screenshots (het labelveld bij
|
||||
@@ -569,11 +572,33 @@ in README "Known limitations":
|
||||
- **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3,
|
||||
4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan
|
||||
een paar bestanden.
|
||||
- **1.1.0:**
|
||||
- **1.1.0** (geen harde datum: een week was een voorzet, langer mag; de releasefrequentie mag omlaag,
|
||||
besluit 2026-11-06):
|
||||
**Betere melding voor namen die een snapshot niet kan bevatten** (backslash, stuurteken, geen UTF-8):
|
||||
een eigen melding in plaats van "unreadable" met het advies over rechten. Zegt dat deze bestanden of
|
||||
mappen in de backup ontbreken, geeft voorbeelden van de paden en vraagt ze te hernoemen; de volgende
|
||||
backup neemt ze mee. Alleen als Nextcloud-melding: de app mailt niet zelf, of meldingen gemaild worden
|
||||
is de keuze van de beheerder (besluit 2026-11-06).
|
||||
**Stand op branch `1.1.0` (2026-11-07):** klaar: betere melding, punten 2, 4, 5, 6 en 8, en de maptijden
|
||||
(`files:scan` na een volledige restore: 0 "Updated" in de simulatie; opslag gepakt in het metarecord).
|
||||
Punt 2 reserveert de jobs in `oc_jobs` met één tijdstempel, alleen wat Nextcloud op dat moment zou kunnen
|
||||
starten; 6 uur aan, dan 1 uur vrij; bij een crash geeft Nextcloud de reservering na 12 uur zelf vrij;
|
||||
de logregels staan op niveau info, dus standaard niet zichtbaar. Pin en notitie staan
|
||||
in de catalogus, met een kopie in de database voor de lijsten. Een moment van vóór het instellen van
|
||||
een schema start niets. Geen schema: geen melding "geen recente backup", wel een melding aan alle
|
||||
beheerders bij het uitzetten en een herinnering, wekelijks zolang een andere locatie een schema heeft,
|
||||
anders dagelijks (besluit 2026-11-07). Nog niet getest: de update vanaf 1.0.3, de pagina in een browser.
|
||||
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
|
||||
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
||||
`target:add` en sleutelimport (het gevoeligste pad), daarom niet in rc1. Daarna de
|
||||
DR-test op PVE02 herhalen zonder kit.
|
||||
**Gebouwd (2026-11-07):** `backup:target:add` op een server zonder sleutel kijkt eerst of de
|
||||
locatie een repository heeft, vraagt dan een wachtwoordzin (of `--passphrase-file`), opent de
|
||||
slots uit `config`, controleert de sleutel tegen de `check` en slaat hem pas dan op; geen
|
||||
kitbevestiging, dus backups wachten op een nieuwe kit. Nieuw bestand `settings` op de locatie,
|
||||
versleuteld met de hoofdsleutel en eenmalig geschreven (bij `target:add` en bij het begin van
|
||||
een backup): de verwijdertermijn, anders de standaard 7 dagen. Getest: unit-tests en
|
||||
`full-restore-roundtrip.sh 35-apache pgsql no-kit`. Nog open: de DR-test op PVE02 zonder kit.
|
||||
2. Instelling "zware achtergrondjobs pauzeren tijdens een backup" (standaard aan, uit te zetten):
|
||||
Memories, files_antivirus BackgroundScanner, Recognize, Preview Generator, fulltextsearch.
|
||||
Via de reservering/last_run van die jobs (geen apps uitzetten), crashbestendig, maximaal
|
||||
@@ -581,6 +606,14 @@ in README "Known limitations":
|
||||
3. Snelheid: lezen en uploaden laten overlappen, meerdere bestanden tegelijk lezen, eventueel
|
||||
een eigen worker (`occ background-job:worker`). Gemeten op nc01: 55% wachten op de schijf,
|
||||
33% op de SMB-upload, ~7% CPU. Vergt extra PHP-processen; botst mogelijk met uitgangspunt 1.
|
||||
**Gemeten 2026-11-07 op nc01 (alleen lezen):** de schijf (sdb, 2 TB, ROTA=1) haalt met `dd`
|
||||
21 MiB/s met 1 stroom, 37 met 4, 55 met 8. PHP las 8 KiB per verzoek: 5,2 MiB/s. Met
|
||||
`stream_set_read_buffer(0)` en blokken van 4 MiB: 26,8 MiB/s. Hulpprocessen die vooruitlezen
|
||||
gaven daarbovenop niets (20 MiB/s, ze concurreren). Gebouwd: grote leesverzoeken (commit
|
||||
"Read files ... 4 MiB"). Niet gebouwd: meerdere uploads tegelijk; SMB bundelt al per 1 MB
|
||||
(icewind/smb), dus winst over een lange lijn vraagt meerdere packs tegelijk.
|
||||
**Besluit 2026-11-08: meerdere uploads tegelijk in 1.2.0**, met een instelling (standaard 2)
|
||||
zodat de beheerder het afstemt op zijn server en de omvang van zijn backup.
|
||||
4. **Schema per locatie** (besluit 2026-10-19): in de instellingen van een locatie "uit",
|
||||
"dagelijks om HH:MM" of "elke N uur (vanaf HH:MM)". Nu is er één `schedule_time` voor alle
|
||||
locaties, één keer per dag. Bij de update wordt de huidige tijd het schema van elke
|
||||
@@ -601,11 +634,10 @@ in README "Known limitations":
|
||||
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
||||
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
||||
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
||||
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
|
||||
9. **Feedbackvraag weghalen in 1.2.0** (besluit 2026-11-07, niet in 1.1.0): de regel "We would like
|
||||
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
||||
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
|
||||
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
|
||||
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
|
||||
dan staan. Er komt weinig feedback bij zo'n 150 downloads per versie; geen bericht is goed
|
||||
bericht. Het formulier (`experience.yml`) in de issues blijft.
|
||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||
|
||||
@@ -7,7 +7,7 @@ installations and hosted Nextcloud. (Choosing SMB as a storage backend is the on
|
||||
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
||||
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
||||
|
||||
**Status: 1.0.3, stable.** Disaster recovery has been tested on real installations (see "Tested
|
||||
**Status: 1.1.0, stable.** Disaster recovery has been tested on real installations (see "Tested
|
||||
disaster recoveries" below). See "Known limitations" below, `PLAN.md` for the design and
|
||||
`docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
|
||||
`docs/DISASTER-RECOVERY.md`.
|
||||
@@ -87,8 +87,9 @@ All backups of an installation are encrypted with one master key.
|
||||
- The **recovery kit** is your own copy of that encrypted key. Still keep it off the server and
|
||||
off the location: an attacker who gets write access to the location can damage or delete its
|
||||
`config` file on purpose, and without the kit that makes every backup there unreadable.
|
||||
- For a **restore on a new server** (disaster recovery) you currently need the **recovery kit**
|
||||
and **one passphrase**.
|
||||
- For a **restore on a new server** (disaster recovery) **one passphrase** is enough: the new
|
||||
server takes the key from the location. With the **recovery kit** you can restore even when the
|
||||
location's `config` file is damaged or gone.
|
||||
- Planned: an option to not store the key on the location at all. Then the kit is the only copy,
|
||||
and losing it means losing every backup; that option will come with an explicit warning.
|
||||
- Up to **three passphrases** ("slots") can open the key, so one person leaving or dying does not
|
||||
@@ -170,6 +171,7 @@ Full restores onto a fresh server, following only [the guide](docs/DISASTER-RECO
|
||||
| Production, about 1 TB, 150,000 files (Nextcloud All-in-One) | Hetzner Storage Box (SFTP), Germany → Canada | PostgreSQL → PostgreSQL | 150,622 files restored in about 25 h (13-14 MiB/s), sample identical |
|
||||
| The same, 1.0.2 | Hetzner Storage Box (SMB), Germany → Canada | PostgreSQL → PostgreSQL | 150,755 files restored in about 29 h (7-13 MiB/s), going on after a lost connection; sample identical, empty folders not restored ([#27](https://github.com/dillardblom/ng_backup/issues/27)) |
|
||||
| 1.0.3, small simulations of 2 GB: as planned, with the restore killed halfway, and with a disk that runs full | Local SFTP server | PostgreSQL → PostgreSQL and MariaDB | all 3,101 user files and all empty folders identical; users, a share and a calendar back |
|
||||
| 1.1.0, the same three simulations | Local SFTP server | PostgreSQL → PostgreSQL and MariaDB | all 3,101 user files identical in content, size and time, all empty folders back; users, a share and a calendar back |
|
||||
| 138 GB simulation: 20 users, calendars, contacts, mail, shares, team folders with an ACL, versions, trash | OVH S3, Warsaw → Canada | MariaDB → PostgreSQL | all 15,772 files and every database count identical |
|
||||
| The same | OVH S3 | MariaDB → MariaDB | all 15,772 files and every database count identical |
|
||||
| The same | Hetzner Storage Box (SMB), Germany → Canada | MariaDB → SQLite | all 15,772 files and every database count identical (27.6 h, about 1.5 MB/s) |
|
||||
@@ -266,18 +268,9 @@ What is planned, without dates. Ideas and votes are welcome in the
|
||||
[issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
|
||||
a 👍 on its issue helps decide what comes first.
|
||||
|
||||
**1.1.0**
|
||||
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
||||
your passphrases.
|
||||
- A schedule per location, also "every N hours".
|
||||
- Locations backed up one after the other instead of at the same time.
|
||||
- All folders get their original modification time back on a restore, not only the empty ones.
|
||||
- A note on a restore point that you can change afterwards.
|
||||
- Pinning a restore point, so the retention policy never removes it (for example the one from
|
||||
before an upgrade).
|
||||
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
||||
Generator) while a backup runs.
|
||||
- Faster backups: reading and uploading at the same time.
|
||||
**1.2.0**
|
||||
- Faster backups over a long line: several uploads at the same time, with a setting
|
||||
(2 by default) to suit the server and the size of the backup.
|
||||
|
||||
**Later**
|
||||
- A full restore onto the existing installation, without first installing a fresh Nextcloud, in a
|
||||
@@ -293,8 +286,9 @@ On the backlog; know them before you rely on NG Backup:
|
||||
|
||||
- **Empty folders only come back from backups made with 1.0.3 or newer.** Older restore points do
|
||||
not contain them.
|
||||
- **Folders with files in them show the time of the restore**, not their original modification time.
|
||||
The files keep theirs. On the roadmap.
|
||||
- **Folders with files in them only get their original modification time back from backups made with
|
||||
1.1.0 or newer.** From older restore points they show the time of the restore; the files keep theirs.
|
||||
A restore through Nextcloud into a folder that is already there leaves that folder's time alone.
|
||||
- **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
|
||||
`config.php`) is not supported: a backup refuses to start there. Folders mounted through the
|
||||
External storage app are not in the data directory and are **not included**, without a warning.
|
||||
@@ -329,10 +323,13 @@ On the backlog; know them before you rely on NG Backup:
|
||||
| `backup:key:init`, `backup:key:kit`, `backup:key:confirm` | create the key, recovery kit, confirmation |
|
||||
| `backup:key:slots`, `backup:key:slot:add/replace/remove` | passphrase slots |
|
||||
| `backup:target:backends/add/edit/list/remove/append-only` | backup locations |
|
||||
| `backup:target:schedule` | when a location is backed up: off, daily, or every N hours |
|
||||
| `backup:run`, `backup:status`, `backup:list` | back up, status (runs of the last 7 days; `--all` for all), restore points |
|
||||
| `backup:cancel <run>` | stop a running backup |
|
||||
| `backup:browse`, `backup:restore` | browse and restore |
|
||||
| `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash |
|
||||
| `backup:pin`, `backup:note` | keep a restore point, a note you can change later |
|
||||
| `backup:pause-jobs` | whether heavy background jobs of other apps pause during a backup |
|
||||
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
||||
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
||||
| `backup:key:import` | import a recovery kit on a fresh installation |
|
||||
|
||||
+9
-5
@@ -45,7 +45,7 @@ welcome: use the "Share your experience" link on the admin page.
|
||||
|
||||
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
||||
you need help with restoring a backup after a crash.]]></description>
|
||||
<version>1.0.3</version>
|
||||
<version>1.1.0</version>
|
||||
<licence>agpl</licence>
|
||||
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
||||
<namespace>NgBackup</namespace>
|
||||
@@ -56,10 +56,10 @@ you need help with restoring a backup after a crash.]]></description>
|
||||
<website>https://github.com/dillardblom/ng_backup</website>
|
||||
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
||||
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.3/screenshots/1-backups.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.3/screenshots/2-restore.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.3/screenshots/3-settings.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.3/screenshots/4-bin.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/1-backups.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/2-restore.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/3-settings.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.1.0/screenshots/4-bin.png</screenshot>
|
||||
<dependencies>
|
||||
<php min-version="8.1"/>
|
||||
<lib>sodium</lib>
|
||||
@@ -83,11 +83,15 @@ you need help with restoring a backup after a crash.]]></description>
|
||||
<command>OCA\NgBackup\Command\ListSnapshots</command>
|
||||
<command>OCA\NgBackup\Command\Browse</command>
|
||||
<command>OCA\NgBackup\Command\Restore</command>
|
||||
<command>OCA\NgBackup\Command\PauseJobs</command>
|
||||
<command>OCA\NgBackup\Command\Pin</command>
|
||||
<command>OCA\NgBackup\Command\Note</command>
|
||||
<command>OCA\NgBackup\Command\Prune</command>
|
||||
<command>OCA\NgBackup\Command\Retention</command>
|
||||
<command>OCA\NgBackup\Command\TargetAppendOnly</command>
|
||||
<command>OCA\NgBackup\Command\TargetTrustCurrent</command>
|
||||
<command>OCA\NgBackup\Command\TargetLimit</command>
|
||||
<command>OCA\NgBackup\Command\TargetSchedule</command>
|
||||
<command>OCA\NgBackup\Command\Trash</command>
|
||||
<command>OCA\NgBackup\Command\TrashRestore</command>
|
||||
<command>OCA\NgBackup\Command\KeySlots</command>
|
||||
|
||||
+4
-1
@@ -25,8 +25,11 @@ return [
|
||||
['name' => 'AdminApi#setTargetLimit', 'url' => '/api/targets/{id}/limit', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#startBackup', 'url' => '/api/backups', 'verb' => 'POST'],
|
||||
['name' => 'AdminApi#tick', 'url' => '/api/tick', 'verb' => 'POST'],
|
||||
['name' => 'AdminApi#setSchedule', 'url' => '/api/schedule', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#setPauseHeavyJobs', 'url' => '/api/settings/pause-heavy-jobs', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#setTargetSchedule', 'url' => '/api/targets/{id}/schedule', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#snapshots', 'url' => '/api/targets/{targetId}/snapshots', 'verb' => 'GET'],
|
||||
['name' => 'AdminApi#setSnapshotNote', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/note', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#setSnapshotPinned', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/pin', 'verb' => 'PUT'],
|
||||
['name' => 'AdminApi#browse', 'url' => '/api/targets/{targetId}/snapshots/{snapshotId}/browse', 'verb' => 'GET'],
|
||||
['name' => 'AdminApi#startRestore', 'url' => '/api/restores', 'verb' => 'POST'],
|
||||
],
|
||||
|
||||
@@ -5,7 +5,8 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
||||
|
||||
## What you need
|
||||
|
||||
- [ ] The **recovery kit** file (`ng_backup-kit.json`) and **one passphrase** of it.
|
||||
- [ ] **One passphrase** of the old server, and if you have it, the **recovery kit** file
|
||||
(`ng_backup-kit.json`).
|
||||
- [ ] The **connection details of the backup location**: host, user, password or keys, bucket, folder.
|
||||
- [ ] A new server with the **same Nextcloud major version** as the old one (for example 34),
|
||||
a database (PostgreSQL, MySQL/MariaDB or SQLite; it may differ from the old one) and room for
|
||||
@@ -15,8 +16,10 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
||||
- [ ] A list of the apps that are **not in the app store** (your own builds, apps copied in by
|
||||
hand). Keep their packages somewhere safe beforehand.
|
||||
|
||||
No kit? If the location still holds its `config` file, the key is there too, but this version
|
||||
can only import it from a kit. Keep the kit off the server, next to the passphrase.
|
||||
No kit? Since version 1.1.0 the passphrase is enough: the key is also stored on the location (in
|
||||
its `config` file), locked with the same passphrases. Skip step 4. The kit is still the safer
|
||||
copy: it helps when that file on the location is damaged. Keep it off the server, next to the
|
||||
passphrase.
|
||||
|
||||
## Steps
|
||||
|
||||
@@ -40,7 +43,8 @@ This keeps the new server quiet (no mail syncs, no notifications, no backups) un
|
||||
|
||||
**3. Install NG Backup** (the same version or newer) and enable it.
|
||||
|
||||
**4. Import the backup key.** Copy the kit to the new server so the web server user can read it.
|
||||
**4. Import the backup key** (with a kit; without one, go to step 5). Copy the kit to the new
|
||||
server so the web server user can read it.
|
||||
|
||||
From your own computer:
|
||||
|
||||
@@ -67,13 +71,29 @@ the index and shows how far it is, for example
|
||||
|
||||
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=... -o root=... --path=...
|
||||
|
||||
If it says "No backup key yet", the key is not imported. Run step 4:
|
||||
Without a kit (step 4 skipped), it finds the backups and asks for a passphrase of the old server;
|
||||
you type it yourself. It then shows the key fingerprint and the deletion delay of the old server.
|
||||
A wrong passphrase changes nothing: run the command again. Backups made before version 1.1.0 did
|
||||
not store the deletion delay on the location; it is then set to 7 days.
|
||||
|
||||
occ backup:key:import /tmp/ng_backup-kit.json
|
||||
If it says that the location's **config file is gone**, the key is only in the recovery kit:
|
||||
import it (step 4), then add the location with `--repair-config` at the end of the same command.
|
||||
That writes the config again from the kit; it works for locations backed up by version 1.1.0 or
|
||||
newer.
|
||||
|
||||
Do **not** run `backup:key:init`, even though the message suggests it: that makes a brand new key,
|
||||
and your backups were encrypted with the old one. NG Backup then refuses the location ("created
|
||||
with a different key").
|
||||
If it says "No backup key yet, and no backups at this location", the connection details or
|
||||
`--path` differ from the old server's. Check them; do **not** run `backup:key:init`: that makes a
|
||||
brand new key, and your backups were encrypted with the old one. NG Backup then refuses the
|
||||
location ("created with a different key").
|
||||
|
||||
**Made a new key by mistake** (with `backup:key:init` or on the page) before adding the location?
|
||||
As long as that key was never used (no location added with it, no backup), it can make way for the
|
||||
key of your backups. NG Backup says so when it refuses the location. Add `--replace-unused-key` to
|
||||
`occ backup:key:import` (with the kit) or to `occ backup:target:add` (without the kit; it asks for
|
||||
a passphrase of the old server). The passphrases and kit of the unused key then stop working. A key
|
||||
that was used for a location is never replaced: its own backups depend on it. A key made with a
|
||||
version before 1.1.0 always counts as used (those versions did not keep track), also when it never
|
||||
protected a location; recover onto a fresh Nextcloud installation instead.
|
||||
|
||||
**6. List the restore points** and pick one, normally the newest:
|
||||
|
||||
@@ -164,6 +184,10 @@ The command then ends with exit code 2 instead of 0. The same option exists for
|
||||
**11. Make a backup of the new server** to the same location (`occ backup:run offsite`). The old
|
||||
server must stay off: two servers must never write to the same location.
|
||||
|
||||
Restored without a kit? Then backups wait until you have a new one: download it on the backup
|
||||
page (or `occ backup:key:kit --output=...`), store it off the server and confirm it
|
||||
(`occ backup:key:confirm`).
|
||||
|
||||
## Never
|
||||
|
||||
- Never let the old server write to the location again if it comes back. Switch its backups off
|
||||
|
||||
+12
-1
@@ -79,7 +79,9 @@ occ backup:list
|
||||
Later runs upload only changed data. An unchanged run uploads (almost) nothing; the output shows
|
||||
the number of uploaded bytes, so check that value when you test incremental behaviour.
|
||||
|
||||
Schedule `occ backup:run` from system cron (or use the Administration settings, if you prefer).
|
||||
Set when each location is backed up in the Administration settings, or with
|
||||
`occ backup:target:schedule offsite 02:00` (once a day), `02:00/6` (every 6 hours from 02:00) or `off`.
|
||||
Locations are backed up one after the other.
|
||||
|
||||
## 4. Check the backup regularly
|
||||
|
||||
@@ -155,6 +157,10 @@ occ backup:list offsite # the snapshots must be lis
|
||||
occ backup:restore:full offsite <snapshot>
|
||||
```
|
||||
|
||||
Without the kit, leave out `backup:key:import`: `backup:target:add` finds the backups and takes
|
||||
the key from the location with a passphrase (`--passphrase-file=/path/passphrase`, or it asks).
|
||||
Backups then wait until a new kit is downloaded and confirmed.
|
||||
|
||||
The command refuses on an installation that already has more than one user, and asks before it
|
||||
starts; `--force` skips both (also needed to rerun it after it stopped halfway).
|
||||
|
||||
@@ -173,8 +179,13 @@ occ backup:prune offsite --dry-run # show first
|
||||
occ backup:prune offsite # apply the policy; forgotten snapshots go to the trash first
|
||||
occ backup:trash # list the trash
|
||||
occ backup:trash:restore offsite <id>
|
||||
occ backup:pin offsite <id> # keep this restore point; --unpin to undo
|
||||
occ backup:note offsite <id> "text" # a note you can change later; without text it is removed
|
||||
```
|
||||
|
||||
The retention policy leaves a pinned restore point alone, and a pinned one does not take the place of
|
||||
a daily, weekly or monthly one. Pin and note are stored on the location, so a new server sees them too.
|
||||
|
||||
Snapshots are kept according to the policy, and their data is deleted only after the deletion delay
|
||||
(set with `backup:key:init --delete-delay-days`). `occ backup:prune offsite --dry-run` shows what would be
|
||||
removed without changing anything.
|
||||
|
||||
@@ -67,6 +67,9 @@ final class LocalBackend implements IBackend {
|
||||
if ($fh === false) {
|
||||
throw new BackendException('Not found: ' . $path);
|
||||
}
|
||||
// Packs are read in large pieces: one request each, not 8 KiB at a time (see Repository::openForReading()).
|
||||
stream_set_read_buffer($fh, 0);
|
||||
stream_set_chunk_size($fh, 4 * 1048576);
|
||||
return $fh;
|
||||
}
|
||||
|
||||
|
||||
@@ -33,6 +33,7 @@ final class BackupJob extends TimedJob {
|
||||
private IAppConfig $appConfig,
|
||||
private LoggerInterface $logger,
|
||||
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||
private \OCA\NgBackup\Service\HeavyJobPause $heavyJobs,
|
||||
) {
|
||||
parent::__construct($time);
|
||||
// Well below the usual 5-minute cron interval: the last run is stamped a moment after cron
|
||||
@@ -43,25 +44,28 @@ final class BackupJob extends TimedJob {
|
||||
$this->setTimeSensitivity(IJob::TIME_SENSITIVE);
|
||||
}
|
||||
|
||||
private const MAX_DAILY_ATTEMPTS = 3;
|
||||
|
||||
protected function run($argument): void {
|
||||
$budget = PHP_SAPI === 'cli' ? 240.0 : 20.0;
|
||||
$deadline = microtime(true) + $budget;
|
||||
$this->startScheduled();
|
||||
foreach ($this->runs->findRunning(null, BackupService::KIND_FULL) as $run) {
|
||||
$this->heavyJobs->sync($this->backups->runsInTurn() !== []);
|
||||
// One after the other: runs that read the same disks at the same time are not faster together.
|
||||
foreach ($this->backups->runsInTurn() as $run) {
|
||||
while (microtime(true) < $deadline && $run->getStatus() === 'running') {
|
||||
$next = $this->backups->step($run, min($deadline, microtime(true) + $budget));
|
||||
if ($next === $run) {
|
||||
break; // step() hands back the same object when another process holds the run's lease
|
||||
break 2; // step() hands back the same object when another process holds the run's lease
|
||||
}
|
||||
$run = $next;
|
||||
}
|
||||
}
|
||||
$this->failStaleRestores();
|
||||
$this->weeklyPrune();
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$targets = $this->targets->list();
|
||||
foreach ($targets as $t) {
|
||||
$this->alerts->checkStale($t);
|
||||
$othersActive = array_filter($targets, static fn ($o) => $o->getId() !== $t->getId() && (string)$o->getSchedule() !== '') !== [];
|
||||
$this->alerts->checkOff($t, $othersActive);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -102,21 +106,18 @@ final class BackupJob extends TimedJob {
|
||||
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
|
||||
}
|
||||
|
||||
/** Daily schedule: "HH:MM" in the schedule's time zone (see Schedule); empty = no schedule. */
|
||||
/** Start the backups the schedules of the locations ask for; they run in turn (see LocationSchedule). */
|
||||
private function startScheduled(): void {
|
||||
$todayAt = $this->schedule->todayAt();
|
||||
if ($todayAt === null || time() < $todayAt) {
|
||||
return;
|
||||
}
|
||||
$now = time();
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$last = $this->runs->lastFinished($t->getId(), BackupService::KIND_FULL);
|
||||
if ($this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== [] || ($last !== null && $last->getStartedAt() >= $todayAt)) {
|
||||
$moment = \OCA\NgBackup\Service\LocationSchedule::lastMoment((string)$t->getSchedule(), $this->schedule->zoneOr($t->getScheduleZone()), $now);
|
||||
if ($moment === null || $this->runs->findRunning($t->getId(), BackupService::KIND_FULL) !== []) {
|
||||
continue;
|
||||
}
|
||||
// After a failure: retry at most MAX_DAILY_ATTEMPTS times a day, an hour apart, instead
|
||||
// of starting (and notifying about) a new full run every five minutes all day.
|
||||
$today = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $todayAt);
|
||||
if (count($today) >= self::MAX_DAILY_ATTEMPTS || ($today !== [] && time() - (int)$today[0]->getFinishedAt() < 3600)) {
|
||||
$attempts = $this->runs->startedSince($t->getId(), BackupService::KIND_FULL, $moment);
|
||||
if (!\OCA\NgBackup\Service\LocationSchedule::isDue($moment, (int)$t->getScheduleSince(),
|
||||
$this->dataStarted($this->runs->lastFinished($t->getId(), BackupService::KIND_FULL)),
|
||||
count($attempts), ($attempts[0] ?? null)?->getFinishedAt(), $now)) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
@@ -126,4 +127,15 @@ final class BackupJob extends TimedJob {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** When a backup began reading its data: a queued one can start well after it was asked for. */
|
||||
private function dataStarted(?\OCA\NgBackup\Db\Run $run): ?int {
|
||||
if ($run === null) {
|
||||
return null;
|
||||
}
|
||||
$state = json_decode($run->getState(), true);
|
||||
$started = is_array($state) ? ($state['files']['started'] ?? null) : null;
|
||||
$at = is_string($started) ? strtotime($started) : false;
|
||||
return $at !== false ? $at : $run->getStartedAt();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,6 +8,7 @@ declare(strict_types=1);
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\KeyService;
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
@@ -18,6 +19,7 @@ use Symfony\Component\Console\Question\Question;
|
||||
final class KeyImport extends Command {
|
||||
public function __construct(
|
||||
private KeyService $keys,
|
||||
private TargetService $targets,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
@@ -27,11 +29,19 @@ final class KeyImport extends Command {
|
||||
->setDescription('Disaster recovery: adopt the backup key from a recovery kit on a fresh installation')
|
||||
->addArgument('kit-file', InputArgument::REQUIRED, 'The recovery kit file (occ backup:key:kit on the original installation)')
|
||||
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Read the passphrase from this file instead of asking')
|
||||
->addOption('replace-unused-key', null, InputOption::VALUE_NONE, 'A new key was made here by mistake: replace it by the kit\'s key, if it was never used (no location, no backup)')
|
||||
->setHelp(<<<'HELP'
|
||||
For a fresh installation only (occ backup:key:init must not have been run here yet). Reads one
|
||||
of the kit's passphrase slots to recover the original master key, so occ backup:target:add opens
|
||||
the existing repository on the location instead of creating a new, empty one.
|
||||
|
||||
A new key was made here by mistake before the location was added? As long as it was never used
|
||||
(no location, no backup), --replace-unused-key replaces it by the kit's key. A key that was used
|
||||
is never replaced.
|
||||
|
||||
No kit? Skip this command: occ backup:target:add then takes the key from the location with one
|
||||
of the old server's passphrases. The kit still helps when the location's config file is damaged.
|
||||
|
||||
Example:
|
||||
occ backup:key:import /root/ng_backup-recovery-kit.json
|
||||
occ backup:target:add offsite sftp -a password::password -o host=... -o user=... -o password=...
|
||||
@@ -40,10 +50,24 @@ HELP);
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
if ($this->keys->isInitialized()) {
|
||||
$output->writeln('<error>A backup key already exists here (fingerprint ' . $this->keys->fingerprint() . '). occ backup:key:import is only for a fresh installation.</error>');
|
||||
$replace = (bool)$input->getOption('replace-unused-key');
|
||||
if ($replace && !$this->keys->isInitialized()) {
|
||||
$output->writeln('<error>There is no backup key here to replace: leave out --replace-unused-key.</error>');
|
||||
return 1;
|
||||
}
|
||||
if ($this->keys->isInitialized()) {
|
||||
if (!$this->targets->keyIsUnused()) {
|
||||
$output->writeln('<error>A backup key already exists here (fingerprint ' . $this->keys->fingerprint() . ') and it has been used for a backup location: it is never replaced.</error>');
|
||||
$output->writeln('occ backup:key:import is for a server that is being restored.');
|
||||
return 1;
|
||||
}
|
||||
if (!$replace) {
|
||||
$output->writeln('<error>A backup key already exists here (fingerprint ' . $this->keys->fingerprint() . '), but it was never used: no location, no backup.</error>');
|
||||
$output->writeln('Made by mistake on a server that is being restored? Run this command again with --replace-unused-key:');
|
||||
$output->writeln('the kit\'s key then takes its place, and the passphrases and recovery kit of the unused key stop working.');
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
$kitFile = (string)$input->getArgument('kit-file');
|
||||
$raw = @file_get_contents($kitFile);
|
||||
if ($raw === false) {
|
||||
@@ -70,12 +94,16 @@ HELP);
|
||||
$pass = (string)$helper->ask($input, $output, $q);
|
||||
}
|
||||
try {
|
||||
$this->keys->importRecoveryKit($kit, $pass);
|
||||
if ($replace) {
|
||||
$this->targets->importKitReplacingUnusedKey($kit, $pass);
|
||||
} else {
|
||||
$this->keys->importRecoveryKit($kit, $pass);
|
||||
}
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln('Backup key imported, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
||||
$output->writeln(($replace ? 'The unused key was replaced. ' : '') . 'Backup key imported, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
||||
// Read from the kit file, which is not authenticated beyond the wrapped key: show it so a
|
||||
// wrong or altered value stands out.
|
||||
$output->writeln('Deletion delay for forgotten snapshots: <info>' . $this->keys->deleteDelayDays() . '</info> days (from the kit).');
|
||||
|
||||
@@ -40,14 +40,14 @@ final class ListSnapshots extends Command {
|
||||
foreach ($targets as $t) {
|
||||
foreach ($this->snapshots->findForTarget($t->getId()) as $s) {
|
||||
$rows[] = [$t->getName(), $s->getSnapshotId(), Format::time($s->getCreatedAt(), $zone), $s->getKind(),
|
||||
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getLabel() ?? ''];
|
||||
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getPinned() ? 'yes' : '', $s->getLabel() ?? '', $s->getNote() ?? ''];
|
||||
}
|
||||
}
|
||||
if ($rows === []) {
|
||||
$output->writeln('No snapshots yet (occ backup:run).');
|
||||
return 0;
|
||||
}
|
||||
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'label'])->setRows($rows)->render();
|
||||
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'pinned', 'label', 'note'])->setRows($rows)->render();
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\PruneService;
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class Note extends Command {
|
||||
public function __construct(
|
||||
private TargetService $targets,
|
||||
private PruneService $prune,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:note')
|
||||
->setDescription('Set or remove the note on a snapshot')
|
||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||
->addArgument('note', InputArgument::OPTIONAL, 'The note, at most ' . PruneService::NOTE_MAX . ' characters; without it the note is removed', '');
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$note = (string)$input->getArgument('note');
|
||||
try {
|
||||
$this->prune->setNote($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), $note);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln(trim($note) === '' ? 'Note removed.' : 'Note saved.');
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\HeavyJobPause;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class PauseJobs extends Command {
|
||||
public function __construct(
|
||||
private HeavyJobPause $pause,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:pause-jobs')
|
||||
->setDescription('Show or set whether heavy background jobs (Memories, Recognize, Preview Generator, ...) pause while a backup runs')
|
||||
->addArgument('setting', InputArgument::OPTIONAL, '"on" or "off"; omit to show the current one');
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$setting = $input->getArgument('setting');
|
||||
if ($setting !== null) {
|
||||
if ($setting !== 'on' && $setting !== 'off') {
|
||||
$output->writeln('<error>Use "on" or "off".</error>');
|
||||
return 1;
|
||||
}
|
||||
$this->pause->setEnabled($setting === 'on');
|
||||
}
|
||||
$status = $this->pause->status();
|
||||
$output->writeln('Pausing heavy background jobs during a backup: ' . ($status['enabled'] ? 'on' : 'off'));
|
||||
if ($status['jobs'] !== []) {
|
||||
$output->writeln('Paused now: ' . implode(', ', $status['jobs']));
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\PruneService;
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Input\InputOption;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class Pin extends Command {
|
||||
public function __construct(
|
||||
private TargetService $targets,
|
||||
private PruneService $prune,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:pin')
|
||||
->setDescription('Keep a snapshot: the retention policy does not remove a pinned one')
|
||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||
->addArgument('snapshot', InputArgument::REQUIRED, 'Snapshot id (see backup:list)')
|
||||
->addOption('unpin', null, InputOption::VALUE_NONE, 'Leave the snapshot to the retention policy again');
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$unpin = (bool)$input->getOption('unpin');
|
||||
try {
|
||||
$this->prune->setPinned($this->targets->get($input->getArgument('target')), $input->getArgument('snapshot'), !$unpin);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln($unpin ? 'Snapshot unpinned; the retention policy applies to it again.' : 'Snapshot pinned; the retention policy leaves it alone.');
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -79,6 +79,9 @@ HELP);
|
||||
if (($f['unreadable'] ?? 0) > 0) {
|
||||
$output->writeln(sprintf('<comment> warning: %d unreadable files or folders are missing from this backup (see the server log)</comment>', $f['unreadable']));
|
||||
}
|
||||
if (($f['badNames'] ?? 0) > 0) {
|
||||
$output->writeln(sprintf('<comment> warning: %d files or folders are missing from this backup because their name is not supported; rename them and the next backup includes them (see the server log)</comment>', $f['badNames']));
|
||||
}
|
||||
if (($f['changing'] ?? 0) > 0) {
|
||||
$output->writeln(sprintf('<comment> warning: %d files kept changing and were saved as read (see the server log)</comment>', $f['changing']));
|
||||
}
|
||||
|
||||
@@ -31,6 +31,7 @@ final class Status extends Command {
|
||||
private VerifyService $verify,
|
||||
private Schedule $schedule,
|
||||
private CronHealth $cron,
|
||||
private \OCA\NgBackup\Service\BackupService $backups,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
@@ -61,9 +62,11 @@ final class Status extends Command {
|
||||
$rows = [];
|
||||
$all = (bool)$input->getOption('all');
|
||||
$shown = $this->runs->findShown($all ? null : time() - self::RECENT_DAYS * 86400);
|
||||
$queued = array_map(static fn (\OCA\NgBackup\Db\Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1));
|
||||
foreach ($shown as $r) {
|
||||
$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
|
||||
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus(),
|
||||
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(),
|
||||
in_array($r->getId(), $queued, true) ? 'queued' : ($r->getStatus() === 'running' ? self::runningStatus($r) : $r->getStatus()),
|
||||
Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
|
||||
$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
|
||||
isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
|
||||
|
||||
+120
-9
@@ -13,7 +13,9 @@ use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Input\InputOption;
|
||||
use Symfony\Component\Console\Helper\QuestionHelper;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
use Symfony\Component\Console\Question\Question;
|
||||
|
||||
final class TargetAdd extends Command {
|
||||
public function __construct(
|
||||
@@ -31,10 +33,25 @@ final class TargetAdd extends Command {
|
||||
->addOption('auth', 'a', InputOption::VALUE_REQUIRED, 'Auth mechanism id (see backup:target:backends)')
|
||||
->addOption('option', 'o', InputOption::VALUE_REQUIRED | InputOption::VALUE_IS_ARRAY, 'Backend/auth option key=value (repeatable)')
|
||||
->addOption('path', 'p', InputOption::VALUE_REQUIRED, 'Folder on the location for this repository', 'ng_backup')
|
||||
->addOption('passphrase-file', null, InputOption::VALUE_REQUIRED, 'Disaster recovery without the recovery kit: read the passphrase from this file instead of asking')
|
||||
->addOption('repair-config', null, InputOption::VALUE_NONE, 'The location\'s config file is gone: write it again with the key of this server or of the imported recovery kit (locations backed up by 1.1.0 or newer)')
|
||||
->addOption('replace-unused-key', null, InputOption::VALUE_NONE, 'Disaster recovery after a new key was made here by mistake: replace that key, if it was never used, by the key of the backups at this location (asks for a passphrase of the old server)')
|
||||
->setHelp(<<<'HELP'
|
||||
The location is reached with files_external's own connection code, but it is never added as an
|
||||
external storage: it does not appear in Files for anyone. Credentials are stored encrypted.
|
||||
|
||||
Disaster recovery without the recovery kit: on a server without a backup key, a location that
|
||||
holds backups gives its key with one of the old server's passphrases. Run this command first
|
||||
(no occ backup:key:import), then occ backup:list and occ backup:restore:full.
|
||||
|
||||
A new key was made here by mistake (occ backup:key:init or the web page) before the location
|
||||
was added? As long as that key was never used (no location, no backup), --replace-unused-key
|
||||
replaces it by the key of the backups. A key that was used is never replaced.
|
||||
|
||||
With the recovery kit: occ backup:key:import first; this command then asks for nothing more.
|
||||
If the location's config file is gone, only the kit has the key: import it, then add the
|
||||
location with --repair-config.
|
||||
|
||||
Examples:
|
||||
occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket=backups -o hostname=s3.example.com -o region=eu-central-1 -o use_ssl=true -o use_path_style=true -o key=AKIA... -o secret=... --path=nextcloud
|
||||
|
||||
@@ -43,13 +60,6 @@ HELP);
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
if (!$this->keys->isInitialized()) {
|
||||
$output->writeln('<error>No backup key yet.</error>');
|
||||
$output->writeln('Restoring from a location that already has backups (disaster recovery)? Import the key from the recovery kit first:');
|
||||
$output->writeln(' occ backup:key:import /path/to/ng_backup-kit.json');
|
||||
$output->writeln('Only for a new location without backups: create a new key with occ backup:key:init. A new key cannot read existing backups.');
|
||||
return 1;
|
||||
}
|
||||
$options = [];
|
||||
foreach ($input->getOption('option') as $pair) {
|
||||
if (!str_contains($pair, '=')) {
|
||||
@@ -63,24 +73,125 @@ HELP);
|
||||
$output->writeln('<error>--auth is required (see occ backup:target:backends)</error>');
|
||||
return 1;
|
||||
}
|
||||
$name = (string)$input->getArgument('name');
|
||||
$backend = (string)$input->getArgument('backend');
|
||||
$auth = (string)$input->getOption('auth');
|
||||
$path = (string)$input->getOption('path');
|
||||
$passphrase = null;
|
||||
$repair = (bool)$input->getOption('repair-config');
|
||||
$replace = (bool)$input->getOption('replace-unused-key');
|
||||
if ($replace) {
|
||||
if ($repair) {
|
||||
$output->writeln('<error>--replace-unused-key needs the config at the location; with the config gone only the kit helps: occ backup:key:import <kit file> --replace-unused-key, then --repair-config.</error>');
|
||||
return 1;
|
||||
}
|
||||
if (!$this->keys->isInitialized()) {
|
||||
$output->writeln('<error>There is no backup key here to replace: leave out --replace-unused-key.</error>');
|
||||
return 1;
|
||||
}
|
||||
if (!$this->targets->keyIsUnused()) {
|
||||
$output->writeln('<error>The backup key of this server (fingerprint ' . $this->keys->fingerprint() . ') has been used for a backup location: it is never replaced.</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln('The backup key of this server (fingerprint ' . $this->keys->fingerprint() . ') was never used. It is replaced by the key of the backups at');
|
||||
$output->writeln('this location, once a passphrase of the old server opens it. Its own passphrases and recovery kit then stop working.');
|
||||
$passphrase = $this->passphrase($input, $output);
|
||||
if ($passphrase === null) {
|
||||
return 1;
|
||||
}
|
||||
} elseif (!$this->keys->isInitialized()) {
|
||||
try {
|
||||
$state = $this->targets->locationState($backend, $auth, $options, $path);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
if ($state === TargetService::LOCATION_EMPTY) {
|
||||
$output->writeln('<error>No backup key yet, and no backups at this location.</error>');
|
||||
$output->writeln('Restoring a server (disaster recovery)? Check the connection details and --path: they must be the same as on the old server.');
|
||||
$output->writeln('Only for a new location: create a new key first with occ backup:key:init. A new key cannot read existing backups.');
|
||||
return 1;
|
||||
}
|
||||
if ($state === TargetService::LOCATION_DATA_WITHOUT_CONFIG) {
|
||||
$output->writeln('<error>This location holds backups, but its config file is gone, and the key was stored in it.</error>');
|
||||
$output->writeln('Now only the recovery kit has the key. Import it, then run this command again with --repair-config:');
|
||||
$output->writeln(' occ backup:key:import /path/to/ng_backup-kit.json');
|
||||
$output->writeln('Do not run occ backup:key:init: a new key cannot read these backups.');
|
||||
return 1;
|
||||
}
|
||||
if ($repair) {
|
||||
$output->writeln('<error>The config at this location is there: leave out --repair-config.</error>');
|
||||
return 1;
|
||||
}
|
||||
$output->writeln('No backup key on this server yet, and this location holds backups. Their key is stored there too,');
|
||||
$output->writeln('locked with the passphrases of the old server: one of them unlocks it (the recovery kit is not needed).');
|
||||
$passphrase = $this->passphrase($input, $output);
|
||||
if ($passphrase === null) {
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
// Adopting an existing repository reads its whole index first, which takes a while on a
|
||||
// large backup or a slow line; show how far it is instead of looking stuck.
|
||||
$output->writeln('Connecting to the location; with existing backups this reads their index first ...');
|
||||
try {
|
||||
$r = $this->targets->add($input->getArgument('name'), $input->getArgument('backend'), $input->getOption('auth'), $options,
|
||||
(string)$input->getOption('path'), self::progress($output));
|
||||
$r = $passphrase === null
|
||||
? $this->targets->add($name, $backend, $auth, $options, $path, self::progress($output), $repair)
|
||||
: $this->targets->addWithPassphrase($name, $backend, $auth, $options, $path, $passphrase, self::progress($output), $replace);
|
||||
} catch (\OCA\NgBackup\Crypto\CryptoException $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
if (!$this->keys->isInitialized() || $replace) {
|
||||
$output->writeln('Nothing was changed. Run the command again to try another passphrase, or use the recovery kit (occ backup:key:import).');
|
||||
}
|
||||
return 1;
|
||||
} catch (\Throwable $e) {
|
||||
if ($output->isDecorated()) {
|
||||
$output->write("\r\033[K"); // clear an unfinished progress line
|
||||
}
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
if (str_contains($e->getMessage(), 'different key') && !$replace) {
|
||||
$output->writeln('The backup key here (fingerprint ' . $this->keys->fingerprint() . ') is not the one these backups were made with.');
|
||||
$output->writeln('Restoring a server? The recovery kit must be the old server\'s: its fingerprint is in the kit file. Nothing was changed.');
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
if ($repair) {
|
||||
$output->writeln('The config of the location was written again from the recovery kit.');
|
||||
}
|
||||
$output->writeln(sprintf('Location <info>%s</info> added (id %d), repository %s %s.', $r['target']->getName(), $r['target']->getId(),
|
||||
$r['target']->getRepositoryId(), $r['created'] ? 'created' : 'found and adopted'));
|
||||
if ($passphrase !== null) {
|
||||
$output->writeln(($replace ? 'The unused key was replaced. ' : '') . 'Backup key taken from the location, fingerprint <info>' . $this->keys->fingerprint() . '</info>.');
|
||||
$days = 'Deletion delay for forgotten snapshots: <info>' . $this->keys->deleteDelayDays() . '</info> days';
|
||||
$output->writeln(match (true) {
|
||||
($r['deleteDelayDays'] ?? null) !== null => $days . ' (from the location).',
|
||||
$r['settingsUnreadable'] ?? false => '<comment>' . $days . ' (the longest: the settings on the location could not be read or hold no valid delay, so nothing leaves the Bin sooner than on the old server; removed restore points take space for that long).</comment>',
|
||||
default => $days . ' (the default: the location has none stored, its backups were made before NG Backup 1.1.0).',
|
||||
});
|
||||
$output->writeln('Next: <info>occ backup:list</info> and <info>occ backup:restore:full</info>. Before backups run again, download a new recovery kit and confirm it (<info>occ backup:key:kit</info>, <info>occ backup:key:confirm</info>).');
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
private function passphrase(InputInterface $input, OutputInterface $output): ?string {
|
||||
$file = $input->getOption('passphrase-file');
|
||||
if ($file !== null) {
|
||||
$raw = @file_get_contents((string)$file);
|
||||
if ($raw === false) {
|
||||
$output->writeln('<error>Cannot read the passphrase file</error>');
|
||||
return null;
|
||||
}
|
||||
return rtrim($raw, "\r\n");
|
||||
}
|
||||
if (!$input->isInteractive()) {
|
||||
$output->writeln('<error>Give the passphrase with --passphrase-file, or run the command without --no-interaction to type it.</error>');
|
||||
return null;
|
||||
}
|
||||
$q = (new Question('Passphrase of the old server (one of its slots): '))->setHidden(true)->setHiddenFallback(false);
|
||||
/** @var QuestionHelper $helper */
|
||||
$helper = $this->getHelper('question');
|
||||
return (string)$helper->ask($input, $output, $q);
|
||||
}
|
||||
|
||||
/**
|
||||
* Progress of reading the index: on a terminal one line that is updated in place, otherwise
|
||||
* (a log, a pipe) a line every 30 seconds. The time left is estimated from the rate so far.
|
||||
|
||||
@@ -31,7 +31,8 @@ final class TargetList extends Base {
|
||||
$rows = [];
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$row = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
||||
'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt())];
|
||||
'repository' => $t->getRepositoryId(), 'added' => date('Y-m-d H:i', $t->getCreatedAt()),
|
||||
'schedule' => (string)$t->getSchedule() !== '' ? $t->getSchedule() : 'off'];
|
||||
if ($input->getOption('test')) {
|
||||
try {
|
||||
$repo = $this->targets->repository($t);
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\Schedule;
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Input\InputOption;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class TargetSchedule extends Command {
|
||||
public function __construct(
|
||||
private TargetService $targets,
|
||||
private Schedule $schedule,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:target:schedule')
|
||||
->setDescription('Show or set when a location is backed up')
|
||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||
->addArgument('schedule', InputArgument::OPTIONAL, '"off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM; omit to show the current one')
|
||||
->addOption('timezone', null, InputOption::VALUE_REQUIRED, 'Time zone of the time, e.g. Europe/Amsterdam');
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$t = $this->targets->get($input->getArgument('target'));
|
||||
$schedule = $input->getArgument('schedule');
|
||||
if ($schedule === null && $input->getOption('timezone') !== null) {
|
||||
$output->writeln('<error>--timezone goes with a schedule, e.g. backup:target:schedule ' . $t->getName() . ' 02:30 --timezone=Europe/Amsterdam</error>');
|
||||
return 1;
|
||||
}
|
||||
$zone = $input->getOption('timezone') ?? $this->schedule->zoneOr($t->getScheduleZone());
|
||||
if ($schedule !== null) {
|
||||
try {
|
||||
$t = $this->targets->setSchedule($t, $schedule, (string)$zone);
|
||||
} catch (\InvalidArgumentException $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
$current = (string)$t->getSchedule();
|
||||
$output->writeln($t->getName() . ': ' . ($current === '' ? 'off' : $current . ' (' . $zone . ')'));
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -48,6 +48,7 @@ final class AdminApiController extends Controller {
|
||||
private \OCA\NgBackup\Service\KeySlotService $slots,
|
||||
private \OCA\NgBackup\Service\PruneService $prune,
|
||||
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||
private \OCA\NgBackup\Service\HeavyJobPause $heavyJobs,
|
||||
private \OCA\NgBackup\Service\CronHealth $cron,
|
||||
) {
|
||||
parent::__construct(Application::APP_ID, $request);
|
||||
@@ -59,10 +60,12 @@ final class AdminApiController extends Controller {
|
||||
foreach ($this->targets->list() as $t) {
|
||||
$targets[] = ['id' => $t->getId(), 'name' => $t->getName(), 'backend' => $t->getBackend(), 'path' => $t->getBasePath(),
|
||||
'repositoryId' => $t->getRepositoryId(), 'createdAt' => $t->getCreatedAt(), 'appendOnly' => (bool)$t->getAppendOnly(),
|
||||
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t)];
|
||||
'maxBytes' => $t->getMaxBytes(), 'storedBytes' => $this->alerts->lastStoredBytes($t),
|
||||
'schedule' => (string)$t->getSchedule(), 'scheduleZone' => $this->schedule->zoneOr($t->getScheduleZone())];
|
||||
}
|
||||
$shown = $this->runs->findShown($allRuns ? null : time() - \OCA\NgBackup\Command\Status::RECENT_DAYS * 86400);
|
||||
$runs = array_map(fn (Run $r) => $this->runJson($r), $shown);
|
||||
$queued = array_flip(array_map(static fn (Run $r): int => $r->getId(), array_slice($this->backups->runsInTurn(), 1)));
|
||||
$runs = array_map(fn (Run $r) => $this->runJson($r) + ['queued' => isset($queued[$r->getId()])], $shown);
|
||||
return new JSONResponse([
|
||||
'key' => ['initialized' => $this->keys->isInitialized(), 'fingerprint' => $this->keys->fingerprint(),
|
||||
'confirmation' => $this->keys->recoveryKitConfirmation(), 'everConfirmed' => $this->keys->isInitialized() && $this->keys->everConfirmed(),
|
||||
@@ -74,12 +77,16 @@ final class AdminApiController extends Controller {
|
||||
'runs' => $runs,
|
||||
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
|
||||
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
|
||||
'schedule' => $this->schedule->get()['time'],
|
||||
'scheduleTimezone' => $this->schedule->get()['timezone'],
|
||||
'cron' => $this->cron->check(),
|
||||
'heavyJobs' => $this->heavyJobs->status(),
|
||||
]);
|
||||
}
|
||||
|
||||
public function setPauseHeavyJobs(bool $enabled): JSONResponse {
|
||||
$this->heavyJobs->setEnabled($enabled);
|
||||
return new JSONResponse(['enabled' => $enabled]);
|
||||
}
|
||||
|
||||
/** Seconds of backup work one tick() does: well within a normal web request. */
|
||||
public const TICK_SECONDS = 10.0;
|
||||
/** Seconds one request for the Bin reads, before the page shows how far it is and asks for more. */
|
||||
@@ -97,16 +104,17 @@ final class AdminApiController extends Controller {
|
||||
}
|
||||
$deadline = microtime(true) + self::TICK_SECONDS;
|
||||
$stepped = 0;
|
||||
$running = $this->runs->findRunning(null, BackupService::KIND_FULL);
|
||||
// Longest wait first: a run that takes all of a tick does not keep another from its turn.
|
||||
usort($running, static fn ($a, $b) => $a->getUpdatedAt() <=> $b->getUpdatedAt());
|
||||
foreach ($running as $run) {
|
||||
$this->heavyJobs->sync($this->backups->runsInTurn() !== []);
|
||||
foreach ($this->backups->runsInTurn() as $run) {
|
||||
if (microtime(true) >= $deadline) {
|
||||
break;
|
||||
}
|
||||
// step() returns at once when the background job (or another tab) holds the run.
|
||||
$this->backups->step($run, $deadline);
|
||||
$next = $this->backups->step($run, $deadline);
|
||||
$stepped++;
|
||||
if ($next->getStatus() === Run::RUNNING) {
|
||||
break; // not finished: the next one waits for its turn
|
||||
}
|
||||
}
|
||||
return new JSONResponse(['stepped' => $stepped]);
|
||||
}
|
||||
@@ -278,25 +286,43 @@ final class AdminApiController extends Controller {
|
||||
return new JSONResponse($this->runJson($run));
|
||||
}
|
||||
|
||||
/** @param string $timezone the administrator's zone (the browser's); empty: keep the current one */
|
||||
public function setSchedule(string $time, string $timezone = ''): JSONResponse {
|
||||
/** @param string $timezone the administrator's zone (the browser's); empty: the default one */
|
||||
public function setTargetSchedule(int $id, string $schedule, string $timezone = ''): JSONResponse {
|
||||
try {
|
||||
$this->schedule->set($time, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
||||
} catch (\InvalidArgumentException $e) {
|
||||
$t = $this->targets->setSchedule($this->targets->get((string)$id), $schedule, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
||||
} catch (\Throwable $e) {
|
||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||
}
|
||||
return new JSONResponse(['schedule' => $time, 'scheduleTimezone' => $this->schedule->get()['timezone']]);
|
||||
return new JSONResponse(['schedule' => $t->getSchedule(), 'scheduleZone' => $t->getScheduleZone()]);
|
||||
}
|
||||
|
||||
public function snapshots(int $targetId): JSONResponse {
|
||||
$out = [];
|
||||
foreach ($this->snapshots->findForTarget($targetId) as $s) {
|
||||
$out[] = ['id' => $s->getSnapshotId(), 'kind' => $s->getKind(), 'label' => $s->getLabel(), 'createdAt' => $s->getCreatedAt(),
|
||||
'files' => $s->getFiles(), 'bytes' => $s->getBytes()];
|
||||
'files' => $s->getFiles(), 'bytes' => $s->getBytes(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
|
||||
}
|
||||
return new JSONResponse($out);
|
||||
}
|
||||
|
||||
public function setSnapshotNote(int $targetId, string $snapshotId, string $note = ''): JSONResponse {
|
||||
try {
|
||||
$this->prune->setNote($this->targets->get((string)$targetId), $snapshotId, $note);
|
||||
} catch (\Throwable $e) {
|
||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||
}
|
||||
return new JSONResponse(['ok' => true]);
|
||||
}
|
||||
|
||||
public function setSnapshotPinned(int $targetId, string $snapshotId, bool $pinned = true): JSONResponse {
|
||||
try {
|
||||
$this->prune->setPinned($this->targets->get((string)$targetId), $snapshotId, $pinned);
|
||||
} catch (\Throwable $e) {
|
||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||
}
|
||||
return new JSONResponse(['ok' => true]);
|
||||
}
|
||||
|
||||
public function browse(int $targetId, string $snapshotId, string $path = ''): JSONResponse {
|
||||
try {
|
||||
return new JSONResponse($this->restore->browse($this->targets->get((string)$targetId), $snapshotId, $path));
|
||||
@@ -341,7 +367,7 @@ final class AdminApiController extends Controller {
|
||||
if ($r->getSnapshotId() !== null) {
|
||||
try {
|
||||
$s = $this->snapshots->findOne($r->getTargetId(), $r->getSnapshotId());
|
||||
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel()];
|
||||
$snapshot = ['createdAt' => $s->getCreatedAt(), 'label' => $s->getLabel(), 'note' => $s->getNote(), 'pinned' => (bool)$s->getPinned()];
|
||||
} catch (\Throwable) {
|
||||
// Snapshot already pruned or trashed: the run keeps its id, just not the date.
|
||||
}
|
||||
|
||||
+24
-6
@@ -87,14 +87,20 @@ final class KeyRing {
|
||||
foreach ($field['slots'] as $slot) {
|
||||
try {
|
||||
return self::unwrap($slot['key'], $passphrase);
|
||||
} catch (CryptoException) {
|
||||
} catch (CryptoException|\SodiumException|\TypeError|\ValueError) {
|
||||
// a wrong passphrase for this slot, or a damaged slot: the next one may still open
|
||||
}
|
||||
}
|
||||
throw new CryptoException('Wrong passphrase or damaged key');
|
||||
}
|
||||
|
||||
/** @param array{v:int, salt:string, ops:int, mem:int, nonce:string, key:string} $wrapped */
|
||||
/** @param array<string, mixed> $wrapped from a kit file or a location's config: checked here, not trusted */
|
||||
public static function unwrap(array $wrapped, #[\SensitiveParameter] string $passphrase): self {
|
||||
if (!isset($wrapped['salt'], $wrapped['nonce'], $wrapped['key'], $wrapped['ops'], $wrapped['mem'])
|
||||
|| !is_string($wrapped['salt']) || !is_string($wrapped['nonce']) || !is_string($wrapped['key'])
|
||||
|| !is_int($wrapped['ops']) || !is_int($wrapped['mem'])) {
|
||||
throw new CryptoException('Damaged key file: a part of the key is missing');
|
||||
}
|
||||
$b64 = static fn (string $s): string => sodium_base642bin($s, SODIUM_BASE64_VARIANT_ORIGINAL);
|
||||
// ops/mem come from unauthenticated JSON (kit file, repository config): bound them, so a
|
||||
// damaged or crafted file cannot make the KDF use unbounded memory or time.
|
||||
@@ -102,10 +108,22 @@ final class KeyRing {
|
||||
|| $wrapped['mem'] < SODIUM_CRYPTO_PWHASH_MEMLIMIT_INTERACTIVE || $wrapped['mem'] > SODIUM_CRYPTO_PWHASH_MEMLIMIT_SENSITIVE) {
|
||||
throw new CryptoException('Unsupported key derivation parameters (damaged or foreign key file)');
|
||||
}
|
||||
$kek = sodium_crypto_pwhash(SODIUM_CRYPTO_SECRETBOX_KEYBYTES, $passphrase, $b64($wrapped['salt']),
|
||||
$wrapped['ops'], $wrapped['mem'], SODIUM_CRYPTO_PWHASH_ALG_ARGON2ID13);
|
||||
$master = sodium_crypto_secretbox_open($b64($wrapped['key']), $b64($wrapped['nonce']), $kek);
|
||||
sodium_memzero($kek);
|
||||
$kek = null;
|
||||
try {
|
||||
$salt = $b64($wrapped['salt']);
|
||||
$sealed = $b64($wrapped['key']);
|
||||
$nonce = $b64($wrapped['nonce']);
|
||||
$kek = sodium_crypto_pwhash(SODIUM_CRYPTO_SECRETBOX_KEYBYTES, $passphrase, $salt,
|
||||
$wrapped['ops'], $wrapped['mem'], SODIUM_CRYPTO_PWHASH_ALG_ARGON2ID13);
|
||||
$master = sodium_crypto_secretbox_open($sealed, $nonce, $kek);
|
||||
} catch (\SodiumException|\ValueError $e) {
|
||||
// invalid base64 or a salt/nonce of the wrong length: damaged, so the caller shows its recovery options
|
||||
throw new CryptoException('Damaged key file: ' . $e->getMessage(), 0, $e);
|
||||
} finally {
|
||||
if ($kek !== null) {
|
||||
sodium_memzero($kek);
|
||||
}
|
||||
}
|
||||
if ($master === false) {
|
||||
throw new CryptoException('Wrong passphrase or damaged key');
|
||||
}
|
||||
|
||||
@@ -24,6 +24,10 @@ use OCP\AppFramework\Db\Entity;
|
||||
* @method void setFiles(?int $v)
|
||||
* @method int|null getBytes()
|
||||
* @method void setBytes(?int $v)
|
||||
* @method string|null getNote()
|
||||
* @method void setNote(?string $v)
|
||||
* @method bool|null getPinned()
|
||||
* @method void setPinned(?bool $v)
|
||||
*/
|
||||
final class Snapshot extends Entity {
|
||||
protected int $targetId = 0;
|
||||
@@ -33,8 +37,11 @@ final class Snapshot extends Entity {
|
||||
protected int $createdAt = 0;
|
||||
protected ?int $files = null;
|
||||
protected ?int $bytes = null;
|
||||
protected ?string $note = null;
|
||||
protected ?bool $pinned = null;
|
||||
|
||||
public function __construct() {
|
||||
$this->addType('pinned', 'boolean');
|
||||
$this->addType('targetId', 'integer');
|
||||
$this->addType('createdAt', 'integer');
|
||||
$this->addType('files', 'integer');
|
||||
|
||||
@@ -28,6 +28,12 @@ use OCP\AppFramework\Db\Entity;
|
||||
* @method void setMaxBytes(?int $v)
|
||||
* @method int getCreatedAt()
|
||||
* @method void setCreatedAt(int $v)
|
||||
* @method string|null getSchedule()
|
||||
* @method void setSchedule(?string $v)
|
||||
* @method string|null getScheduleZone()
|
||||
* @method void setScheduleZone(?string $v)
|
||||
* @method int|null getScheduleSince()
|
||||
* @method void setScheduleSince(?int $v)
|
||||
*/
|
||||
final class Target extends Entity {
|
||||
protected string $name = '';
|
||||
@@ -39,8 +45,12 @@ final class Target extends Entity {
|
||||
protected int $createdAt = 0;
|
||||
protected ?bool $appendOnly = null;
|
||||
protected ?int $maxBytes = null;
|
||||
protected ?string $schedule = null;
|
||||
protected ?string $scheduleZone = null;
|
||||
protected ?int $scheduleSince = null;
|
||||
|
||||
public function __construct() {
|
||||
$this->addType('scheduleSince', 'integer');
|
||||
$this->addType('createdAt', 'integer');
|
||||
$this->addType('appendOnly', 'boolean');
|
||||
$this->addType('maxBytes', 'integer');
|
||||
|
||||
+47
-6
@@ -52,19 +52,35 @@ final class BackupRun {
|
||||
$stats = $state['stats'];
|
||||
$walker = new TreeWalker($state['roots'], $state['exclude'], function (string $logical) use (&$state, &$stats): void {
|
||||
self::recordUnreadable($state, $stats, $logical);
|
||||
});
|
||||
}, true);
|
||||
$parent = new ParentCursor($repo, $state['parent'], $state['parentPos']);
|
||||
$lines = [];
|
||||
$emptyDirs = [];
|
||||
$folders = [];
|
||||
$finishedWalk = true;
|
||||
|
||||
foreach ($walker->walk($state['after']) as $logical => $path) {
|
||||
if (!mb_check_encoding($logical, 'UTF-8') || !Repository::isSafePath($logical)) {
|
||||
$safe = mb_check_encoding($logical, 'UTF-8') && Repository::isSafePath($logical);
|
||||
if ($walker->atFolderWithContent()) {
|
||||
// Only its time: what is in it is reported by itself, and a bad name in the path of
|
||||
// each of those is what gets reported. The cursor stays for such a name, see below.
|
||||
$mtime = $safe ? @filemtime($path) : false;
|
||||
if ($safe && $mtime !== false) {
|
||||
$folders[] = json_encode([$logical, $mtime], JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
|
||||
$state['after'] = $logical;
|
||||
if (microtime(true) >= $deadline) {
|
||||
$finishedWalk = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (!$safe) {
|
||||
// A name a snapshot cannot hold (a backslash, a control character, not UTF-8). One
|
||||
// such path must not make a backup fail, or the whole snapshot unreadable later.
|
||||
// The cursor stays where it is: the state is JSON, which cannot hold such a name. A
|
||||
// next step meets it again, and it is reported once.
|
||||
self::recordUnreadable($state, $stats, mb_scrub($logical, 'UTF-8'));
|
||||
self::recordBadName($state, $stats, $logical);
|
||||
continue;
|
||||
}
|
||||
if (is_dir($path)) {
|
||||
@@ -104,7 +120,7 @@ final class BackupRun {
|
||||
if ($cur === null || $cur['path'] !== $logical) {
|
||||
$cur = ['path' => $logical, 'offset' => 0, 'blobs' => [], 'version' => $version];
|
||||
}
|
||||
$fh = @fopen($path, 'rb');
|
||||
$fh = Repository::openForReading($path);
|
||||
if ($fh === false) {
|
||||
self::recordUnreadable($state, $stats, $logical);
|
||||
$state['cur'] = null;
|
||||
@@ -164,6 +180,11 @@ final class BackupRun {
|
||||
$writer->write(implode("\n", $emptyDirs) . "\n");
|
||||
$state['dirSegments'][] = $writer->finish();
|
||||
}
|
||||
if ($folders !== []) {
|
||||
$writer = $repo->blobWriter($stats);
|
||||
$writer->write(implode("\n", $folders) . "\n");
|
||||
$state['folderSegments'][] = $writer->finish();
|
||||
}
|
||||
$repo->flushPacks();
|
||||
$state['parentPos'] = $parent->position();
|
||||
$state['stats'] = $stats;
|
||||
@@ -186,10 +207,14 @@ final class BackupRun {
|
||||
$emptyDirs[] = json_decode($line, true, 8, JSON_THROW_ON_ERROR);
|
||||
}
|
||||
}
|
||||
// The times of the other folders too, packed: every look at a snapshot's meta record reads
|
||||
// this, and a string of a few hundred KiB costs that far less than tens of thousands of arrays.
|
||||
$folders = isset($state['folderSegments']) ? implode("\n", iterator_to_array($repo->readLines(array_values(array_merge(...$state['folderSegments']))), false)) : '';
|
||||
$repo->putSnapshot($state['snapshot'],
|
||||
['time' => $state['started'], 'parent' => $state['parent'], 'label' => $state['label'],
|
||||
'roots' => array_keys($state['roots']), 'stats' => $state['stats'],
|
||||
'warnings' => self::warnings($state)] + ($emptyDirs === [] ? [] : ['emptyDirs' => $emptyDirs]) + $state['meta'],
|
||||
'warnings' => self::warnings($state)] + ($emptyDirs === [] ? [] : ['emptyDirs' => $emptyDirs])
|
||||
+ ($folders === '' ? [] : ['folderTimes' => base64_encode((string)gzdeflate($folders))]) + $state['meta'],
|
||||
$repo->readLines(array_values(array_merge(...$state['segments']))));
|
||||
$state['phase'] = 'commit';
|
||||
}
|
||||
@@ -250,10 +275,26 @@ final class BackupRun {
|
||||
$stats['unreadable'] = ($stats['unreadable'] ?? 0) + 1;
|
||||
}
|
||||
|
||||
/** @return array{unreadable: list<string>, changing: list<string>} examples for the report (first 50 each) */
|
||||
/**
|
||||
* A file or directory left out because of its name. Remembered like an unreadable one, in a form
|
||||
* that fits the JSON state and is safe to show: control characters and invalid bytes replaced.
|
||||
*/
|
||||
private static function recordBadName(array &$state, array &$stats, string $logical): void {
|
||||
$shown = (string)preg_replace('/[\x00-\x1f\x7f]/', '?', mb_scrub($logical, 'UTF-8'));
|
||||
if (isset($state['badNames'][$shown])) {
|
||||
return;
|
||||
}
|
||||
if (count($state['badNames'] ?? []) < 1000) {
|
||||
$state['badNames'][$shown] = true;
|
||||
}
|
||||
$stats['badNames'] = ($stats['badNames'] ?? 0) + 1;
|
||||
}
|
||||
|
||||
/** @return array{unreadable: list<string>, badNames: list<string>, changing: list<string>} examples for the report (first 50 each) */
|
||||
public static function warnings(array $state): array {
|
||||
return [
|
||||
'unreadable' => array_slice(array_map('strval', array_keys($state['unreadable'] ?? [])), 0, 50),
|
||||
'badNames' => array_slice(array_map('strval', array_keys($state['badNames'] ?? [])), 0, 50),
|
||||
'changing' => array_slice(array_map('strval', array_values($state['changing'] ?? [])), 0, 50),
|
||||
];
|
||||
}
|
||||
|
||||
+17
-1
@@ -17,6 +17,8 @@ namespace OCA\NgBackup\Job;
|
||||
* and be merge-joined with the previous snapshot without holding any list in memory.
|
||||
*/
|
||||
final class TreeWalker {
|
||||
private bool $atFolder = false;
|
||||
|
||||
/**
|
||||
* @param array<string, string> $roots name => absolute directory, walked in this order
|
||||
* @param list<string> $exclude path prefixes ("<root>/<relative>") that are skipped entirely;
|
||||
@@ -27,9 +29,16 @@ final class TreeWalker {
|
||||
private array $exclude = [],
|
||||
/** Called with the logical path of a directory that cannot be listed. */
|
||||
private ?\Closure $onUnreadable = null,
|
||||
/** Also yield directories with something in them, right before the first thing in them. */
|
||||
private bool $withFolders = false,
|
||||
) {
|
||||
}
|
||||
|
||||
/** Whether what was just yielded is a directory with something in it (see $withFolders). */
|
||||
public function atFolderWithContent(): bool {
|
||||
return $this->atFolder;
|
||||
}
|
||||
|
||||
/** @return \Generator<string, string> logical path => absolute path, strictly after $after */
|
||||
public function walk(?string $after = null): \Generator {
|
||||
foreach ($this->roots as $name => $dir) {
|
||||
@@ -75,7 +84,14 @@ final class TreeWalker {
|
||||
if (is_link($childAbs)) {
|
||||
continue; // never follow symlinks out of the data directory
|
||||
}
|
||||
if (is_dir($childAbs)) {
|
||||
$childIsDir = is_dir($childAbs);
|
||||
if (($childIsDir || is_file($childAbs)) && $empty && !$this->isExcluded($childLogical, $childIsDir)
|
||||
&& $this->withFolders && $logical !== '' && ($after === null || self::compare($logical, $after) > 0)) {
|
||||
$this->atFolder = true;
|
||||
yield $logical => $abs;
|
||||
$this->atFolder = false;
|
||||
}
|
||||
if ($childIsDir) {
|
||||
$empty = $empty && $this->isExcluded($childLogical, true);
|
||||
// Skip a directory entirely if everything in it sorts before the cursor.
|
||||
if ($after !== null && self::compare($childLogical, $after) < 0 && !self::isPrefix($childLogical, $after)) {
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Migration;
|
||||
|
||||
use Closure;
|
||||
use OCP\DB\ISchemaWrapper;
|
||||
use OCP\DB\Types;
|
||||
use OCP\Migration\IOutput;
|
||||
use OCP\Migration\SimpleMigrationStep;
|
||||
|
||||
/** Local copy of what the catalog on the location says about a snapshot: its note and pin. */
|
||||
final class Version000007Date20261107000000 extends SimpleMigrationStep {
|
||||
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
|
||||
/** @var ISchemaWrapper $schema */
|
||||
$schema = $schemaClosure();
|
||||
$t = $schema->getTable('ngb_snapshots');
|
||||
$changed = false;
|
||||
if (!$t->hasColumn('note')) {
|
||||
$t->addColumn('note', Types::STRING, ['notnull' => false, 'length' => 1000]);
|
||||
$changed = true;
|
||||
}
|
||||
if (!$t->hasColumn('pinned')) {
|
||||
$t->addColumn('pinned', Types::BOOLEAN, ['notnull' => false, 'default' => false]);
|
||||
$changed = true;
|
||||
}
|
||||
return $changed ? $schema : null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Migration;
|
||||
|
||||
use Closure;
|
||||
use OCA\NgBackup\Service\LocationSchedule;
|
||||
use OCA\NgBackup\Service\Schedule;
|
||||
use OCP\DB\ISchemaWrapper;
|
||||
use OCP\DB\Types;
|
||||
use OCP\IDBConnection;
|
||||
use OCP\Migration\IOutput;
|
||||
use OCP\Migration\SimpleMigrationStep;
|
||||
|
||||
/** A schedule per location. Each existing location gets the daily time that held for all of them. */
|
||||
final class Version000008Date20261107010000 extends SimpleMigrationStep {
|
||||
public function __construct(
|
||||
private IDBConnection $db,
|
||||
private Schedule $schedule,
|
||||
) {
|
||||
}
|
||||
|
||||
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
|
||||
/** @var ISchemaWrapper $schema */
|
||||
$schema = $schemaClosure();
|
||||
$t = $schema->getTable('ngb_targets');
|
||||
if ($t->hasColumn('schedule')) {
|
||||
return null;
|
||||
}
|
||||
$t->addColumn('schedule', Types::STRING, ['notnull' => false, 'length' => 16]);
|
||||
$t->addColumn('schedule_zone', Types::STRING, ['notnull' => false, 'length' => 64]);
|
||||
$t->addColumn('schedule_since', Types::BIGINT, ['notnull' => false, 'unsigned' => true]);
|
||||
return $schema;
|
||||
}
|
||||
|
||||
public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options): void {
|
||||
['time' => $time, 'timezone' => $zone] = $this->schedule->get();
|
||||
$qb = $this->db->getQueryBuilder();
|
||||
$qb->update('ngb_targets')
|
||||
->set('schedule', $qb->createNamedParameter(Schedule::validTime($time) ? LocationSchedule::normalize($time) : ''))
|
||||
->set('schedule_zone', $qb->createNamedParameter($zone))
|
||||
->where($qb->expr()->isNull('schedule'));
|
||||
$qb->executeStatement();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Migration;
|
||||
|
||||
use Closure;
|
||||
use OCA\NgBackup\Service\KeyService;
|
||||
use OCP\Migration\IOutput;
|
||||
use OCP\Migration\SimpleMigrationStep;
|
||||
|
||||
/**
|
||||
* A key from before 1.1.0 counts as used: those versions did not keep track, and it may protect
|
||||
* backups on a location that was removed from this server later (user exports included). Only a
|
||||
* key made with 1.1.0 or newer can be replaced as an unused one.
|
||||
*/
|
||||
final class Version000009Date20261108000000 extends SimpleMigrationStep {
|
||||
public function __construct(
|
||||
private KeyService $keys,
|
||||
) {
|
||||
}
|
||||
|
||||
public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options): void {
|
||||
if ($this->keys->isInitialized()) {
|
||||
$this->keys->markUsed();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -46,6 +46,10 @@ final class Notifier implements INotifier {
|
||||
$notification->setParsedSubject($l->t('Backup to %1$s finished with warnings: %2$s unreadable, %3$s changed while being read', [$p['target'], $p['unreadable'], $p['changing']]));
|
||||
$notification->setParsedMessage($l->t('Unreadable files and folders are missing from this backup (check their permissions). Files that kept changing were saved as they were while being read. Examples: %s', [$p['examples'] ?? '']));
|
||||
break;
|
||||
case 'run_bad_names':
|
||||
$notification->setParsedSubject($l->n('Backup to %s finished, but %n item is missing because of its name', 'Backup to %s finished, but %n items are missing because of their names', (int)$p['count'], [$p['target']]));
|
||||
$notification->setParsedMessage($l->t('Names with a backslash or a control character, or that are not valid UTF-8, are not supported, so these files and folders are not in this backup. Rename them; the next backup includes them. Examples: %s', [$p['examples'] ?? '']));
|
||||
break;
|
||||
case 'quota_warning':
|
||||
$notification->setParsedSubject($l->t('Backup location %1$s is at %2$s%% of its limit (%3$s of %4$s)', [$p['target'], $p['percent'], $p['used'], $p['limit']]));
|
||||
$notification->setParsedMessage($l->t('Backups stop when the limit is reached. Raise the limit, or let the retention policy remove older backups.'));
|
||||
@@ -57,13 +61,21 @@ final class Notifier implements INotifier {
|
||||
: $l->t('No successful backup to %1$s since %2$s', [$p['target'], $p['since']]));
|
||||
$notification->setParsedMessage($l->t('Check the backup settings and the server log.'));
|
||||
break;
|
||||
case 'schedule_off':
|
||||
$notification->setParsedSubject($l->t('Backup location %s has no schedule', [$p['target']]));
|
||||
$notification->setParsedMessage(($p['others'] ?? '') === '1'
|
||||
? $l->t('No backups run to this location until it gets a schedule or someone starts one. If that is what you want, you can ignore this reminder.')
|
||||
: $l->t('No location has a schedule, so no backups run at all unless someone starts one. Set a schedule under Backups.'));
|
||||
break;
|
||||
case 'security_event':
|
||||
$who = (string)($p['actor'] ?? '');
|
||||
$notification->setParsedSubject(match ($p['event'] ?? '') {
|
||||
'kit_downloaded' => $l->t('%s downloaded the backup recovery kit', [$who]),
|
||||
'key_replaced' => $l->t('%1$s replaced the unused backup key %2$s by the key of existing backups, %3$s; download and confirm the new recovery kit', [$who, $p['old'] ?? '', $p['new'] ?? '']),
|
||||
'target_removed' => $l->t('%1$s removed backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'target_changed' => $l->t('%1$s changed the connection settings of backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'append_only_off' => $l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'schedule_off' => $l->t('%1$s turned off the schedule of backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'append_only_off' =>$l->t('%1$s turned off append-only for backup location %2$s', [$who, $p['target'] ?? '']),
|
||||
'retention_changed' => $l->t('%1$s changed the backup retention to %2$s', [$who, $p['policy'] ?? '']),
|
||||
'snapshots_forgotten' => $l->t('%1$s moved %2$s snapshots of %3$s to the trash; they are deleted permanently after %4$s unless restored', [$who, $p['count'] ?? '', $p['target'] ?? '', $p['until'] ?? '']),
|
||||
'snapshot_untrashed' => $l->t('%1$s restored snapshot %2$s of %3$s from the trash', [$who, $p['snapshot'] ?? '', $p['target'] ?? '']),
|
||||
|
||||
@@ -30,6 +30,10 @@ final class Catalog {
|
||||
public array $trash = [];
|
||||
/** @var array<string, true> manifest path => true, for a per-user export (users/<uid>/<exportId>) */
|
||||
public array $userExports = [];
|
||||
/** @var array<string, true> snapshots the retention policy leaves alone */
|
||||
public array $pinned = [];
|
||||
/** @var array<string, string> snapshot id => note, also while the snapshot is in the trash */
|
||||
public array $notes = [];
|
||||
public string $hash = '';
|
||||
|
||||
public function __construct(
|
||||
@@ -82,6 +86,8 @@ final class Catalog {
|
||||
// Catalogs written before user exports existed have no such key: default to empty rather
|
||||
// than adopting the location's current users/ listing (that would skip verification).
|
||||
$this->userExports = array_fill_keys($data['userExports'] ?? [], true);
|
||||
$this->pinned = array_fill_keys($data['pinned'] ?? [], true);
|
||||
$this->notes = $data['notes'] ?? [];
|
||||
$this->anchor?->set($this->repositoryId, $this->gen, $this->hash);
|
||||
}
|
||||
|
||||
@@ -93,9 +99,12 @@ final class Catalog {
|
||||
if ($this->backend->exists($this->path($next))) {
|
||||
throw new RepositoryException("Catalog generation $next was written by another process at the same time; try again");
|
||||
}
|
||||
// A pin ends when its snapshot leaves the list, a note when the snapshot is gone for good.
|
||||
$this->pinned = array_intersect_key($this->pinned, $this->snapshots);
|
||||
$this->notes = array_intersect_key($this->notes, $this->snapshots + $this->trash);
|
||||
$plain = json_encode(['repository' => $this->repositoryId, 'gen' => $next, 'time' => gmdate('c'), 'prev' => $this->hash,
|
||||
'snapshots' => array_keys($this->snapshots), 'trash' => array_keys($this->trash),
|
||||
'userExports' => array_keys($this->userExports)], JSON_THROW_ON_ERROR);
|
||||
'userExports' => array_keys($this->userExports), 'pinned' => array_keys($this->pinned), 'notes' => $this->notes], JSON_THROW_ON_ERROR);
|
||||
$encrypted = $this->cipher->encryptString($plain, $this->ad($next));
|
||||
$stream = fopen('php://memory', 'w+b');
|
||||
fwrite($stream, $encrypted);
|
||||
|
||||
@@ -18,6 +18,7 @@ use OCA\NgBackup\Crypto\StreamCipher;
|
||||
*
|
||||
* Layout on the backend:
|
||||
* config plaintext JSON: format version, repository id, wrapped master key
|
||||
* settings encrypted JSON: settings fixed at installation (deletion delay)
|
||||
* packs/xx/<packId> encrypted blobs + encrypted pack header
|
||||
* index/<packId> encrypted list of the blobs in that pack
|
||||
* snapshots/<snapId> encrypted JSON lines: one meta line, then one line per file
|
||||
@@ -28,6 +29,7 @@ use OCA\NgBackup\Crypto\StreamCipher;
|
||||
final class Repository {
|
||||
public const FORMAT = 1;
|
||||
public const BLOB_SIZE = 4 * 1048576;
|
||||
private const SETTINGS = 'settings';
|
||||
private const FLAG_DEFLATE = 1;
|
||||
/** Blobs from this size on get a quick compression test before the full compression. */
|
||||
private const DEFLATE_PROBE_FROM = 256 * 1024;
|
||||
@@ -79,7 +81,7 @@ final class Repository {
|
||||
// opens this data; refuse instead.
|
||||
foreach (['index', 'snapshots', 'trash/snapshots', 'catalog'] as $dir) {
|
||||
if ($backend->listFiles($dir) !== []) {
|
||||
throw new RepositoryException("This location holds backup data ($dir/) but no repository config could be read: not creating a new repository over it. Check the connection and the folder; a missing config can be restored from the recovery kit.");
|
||||
throw new RepositoryException("This location holds backup data ($dir/) but no repository config could be read: not creating a new repository over it. Check the connection and the folder. If the config is really gone, occ backup:target:add with --repair-config writes it again with this server's key (on a new server: import the recovery kit first, occ backup:key:import).");
|
||||
}
|
||||
}
|
||||
$id = bin2hex(random_bytes(16));
|
||||
@@ -95,12 +97,7 @@ final class Repository {
|
||||
/** @param ?callable(int, int): void $progress index files downloaded so far, and in all (see BlobIndex::load()) */
|
||||
public static function openWithKey(IBackend $backend, KeyRing $keys, ?IIndexCache $cache = null, ?ICatalogAnchor $anchor = null, ?callable $progress = null): self {
|
||||
$config = self::readConfig($backend);
|
||||
try {
|
||||
$ok = isset($config['check']) && (new StreamCipher($keys))->decryptString((string)base64_decode($config['check'], true), 'config-check') === $config['id'];
|
||||
} catch (\OCA\NgBackup\Crypto\CryptoException) {
|
||||
$ok = false;
|
||||
}
|
||||
if (!$ok) {
|
||||
if (!self::configMatches($config, $keys)) {
|
||||
throw new RepositoryException('This repository was created with a different key');
|
||||
}
|
||||
$repo = new self($backend, $keys, $config['id']);
|
||||
@@ -134,6 +131,49 @@ final class Repository {
|
||||
return $this->catalog->gen;
|
||||
}
|
||||
|
||||
/** @return list<string> snapshots the retention policy leaves alone */
|
||||
public function pinnedSnapshots(): array {
|
||||
return array_map('strval', array_keys($this->catalog->pinned));
|
||||
}
|
||||
|
||||
/** @return array<string, string> snapshot id => note */
|
||||
public function snapshotNotes(): array {
|
||||
return $this->catalog->notes;
|
||||
}
|
||||
|
||||
public function setPinned(string $snapshotId, bool $pinned): void {
|
||||
$this->requireSnapshot($snapshotId);
|
||||
if (isset($this->catalog->pinned[$snapshotId]) === $pinned) {
|
||||
return;
|
||||
}
|
||||
if ($pinned) {
|
||||
$this->catalog->pinned[$snapshotId] = true;
|
||||
} else {
|
||||
unset($this->catalog->pinned[$snapshotId]);
|
||||
}
|
||||
$this->catalog->write();
|
||||
}
|
||||
|
||||
/** A note kept in the catalog: unlike the label inside the snapshot, it can change afterwards. */
|
||||
public function setSnapshotNote(string $snapshotId, string $note): void {
|
||||
$this->requireSnapshot($snapshotId);
|
||||
if (($this->catalog->notes[$snapshotId] ?? '') === $note) {
|
||||
return;
|
||||
}
|
||||
if ($note === '') {
|
||||
unset($this->catalog->notes[$snapshotId]);
|
||||
} else {
|
||||
$this->catalog->notes[$snapshotId] = $note;
|
||||
}
|
||||
$this->catalog->write();
|
||||
}
|
||||
|
||||
private function requireSnapshot(string $snapshotId): void {
|
||||
if (!isset($this->catalog->snapshots[$snapshotId])) {
|
||||
throw new \InvalidArgumentException('There is no snapshot ' . $snapshotId . ' on this location');
|
||||
}
|
||||
}
|
||||
|
||||
/** Index files downloaded when this repository was opened (0 when the local cache was complete). */
|
||||
public function indexDownloads(): int {
|
||||
return $this->index->downloaded;
|
||||
@@ -153,6 +193,114 @@ final class Repository {
|
||||
return $config;
|
||||
}
|
||||
|
||||
/**
|
||||
* Open a local file to read it in whole blobs. PHP reads a file in pieces of 8 KiB by default;
|
||||
* on a disk where each request waits (a network or cloud volume) that made reading five times
|
||||
* slower: 5 MiB/s instead of 27 MiB/s on nc01. Without PHP's read buffer every fread() of a
|
||||
* blob is one request to the disk.
|
||||
*
|
||||
* @return resource|false
|
||||
*/
|
||||
public static function openForReading(string $path) {
|
||||
$fh = @fopen($path, 'rb');
|
||||
if ($fh !== false) {
|
||||
stream_set_read_buffer($fh, 0);
|
||||
stream_set_chunk_size($fh, self::BLOB_SIZE);
|
||||
}
|
||||
return $fh;
|
||||
}
|
||||
|
||||
private static function configMatches(array $config, KeyRing $keys): bool {
|
||||
try {
|
||||
return isset($config['check'], $config['id'])
|
||||
&& (new StreamCipher($keys))->decryptString((string)base64_decode((string)$config['check'], true), 'config-check') === $config['id'];
|
||||
} catch (\OCA\NgBackup\Crypto\CryptoException) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The passphrase-wrapped key slots in the config at this location: disaster recovery without
|
||||
* the recovery kit, where any one of their passphrases gives back the master key.
|
||||
*/
|
||||
public static function wrappedKeyAt(IBackend $backend): array {
|
||||
$config = self::readConfig($backend);
|
||||
if (!isset($config['key']) || !is_array($config['key'])) {
|
||||
throw new RepositoryException('The repository config at this location holds no key');
|
||||
}
|
||||
return $config['key'];
|
||||
}
|
||||
|
||||
/** Whether the repository at this location was created with this key. Does not read the index. */
|
||||
public static function madeWithKey(IBackend $backend, KeyRing $keys): bool {
|
||||
return self::configMatches(self::readConfig($backend), $keys);
|
||||
}
|
||||
|
||||
/**
|
||||
* Settings of the installation that never change (the deletion delay), sealed with the
|
||||
* master key so that, unlike the config and the recovery kit, nobody can alter them unseen.
|
||||
* Empty when the location has none yet (made before 1.1.0, until its next backup); only
|
||||
* ['unreadable' => true] when they are there but cannot be read.
|
||||
*
|
||||
* @return array<string, mixed>
|
||||
*/
|
||||
public static function settingsAt(IBackend $backend, KeyRing $keys): array {
|
||||
try {
|
||||
if (!$backend->exists(self::SETTINGS)) {
|
||||
return [];
|
||||
}
|
||||
$fh = $backend->get(self::SETTINGS);
|
||||
$raw = (string)stream_get_contents($fh);
|
||||
fclose($fh);
|
||||
$settings = json_decode((new StreamCipher($keys))->decryptString($raw, 'settings'), true, 16, JSON_THROW_ON_ERROR);
|
||||
return is_array($settings) ? $settings : ['unreadable' => true];
|
||||
} catch (BackendException|\OCA\NgBackup\Crypto\CryptoException|\JsonException) {
|
||||
return ['unreadable' => true];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Write the settings once; a location that has them keeps them. The repository id goes with
|
||||
* them: with it and the key from the recovery kit, a lost config can be written again.
|
||||
*/
|
||||
public function recordSettings(array $settings): void {
|
||||
if ($this->backend->exists(self::SETTINGS)) {
|
||||
return;
|
||||
}
|
||||
$settings['repository'] = $this->repositoryId;
|
||||
self::putString($this->backend, self::SETTINGS, $this->cipher->encryptString(json_encode($settings, JSON_THROW_ON_ERROR), 'settings'));
|
||||
}
|
||||
|
||||
/** Whether the location holds backup data, with or without a config. */
|
||||
public static function holdsData(IBackend $backend): bool {
|
||||
foreach (['index', 'snapshots', 'catalog'] as $dir) {
|
||||
if ($backend->listFiles($dir) !== []) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Write a lost config again, from the key (recovery kit) and the repository id in the
|
||||
* sealed settings. Only where there is no config and the settings open with this key, so
|
||||
* nothing that works is ever overwritten and a wrong key writes nothing.
|
||||
*/
|
||||
public static function repairConfig(IBackend $backend, KeyRing $keys, array $wrappedKey): void {
|
||||
if ($backend->exists('config')) {
|
||||
throw new RepositoryException('This location has a config: nothing to repair, leave out --repair-config');
|
||||
}
|
||||
$id = self::settingsAt($backend, $keys)['repository'] ?? null;
|
||||
if (!is_string($id) || !preg_match('/^[0-9a-f]{32}$/', $id)) {
|
||||
throw new RepositoryException($backend->exists(self::SETTINGS)
|
||||
? 'The settings at this location do not open with this key: the recovery kit is of another server, or the settings are damaged. The config cannot be written again.'
|
||||
: 'This location has no settings file (it was backed up only by NG Backup before 1.1.0), so its config cannot be written again.');
|
||||
}
|
||||
$config = ['format' => self::FORMAT, 'id' => $id, 'created' => gmdate('c'), 'key' => $wrappedKey,
|
||||
'check' => base64_encode((new StreamCipher($keys))->encryptString($id, 'config-check')), 'repaired' => true];
|
||||
self::putString($backend, 'config', json_encode($config, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR));
|
||||
}
|
||||
|
||||
/** Replace the passphrase-wrapped key slots in this repository's config (after a slot change). */
|
||||
public function updateWrappedKey(array $wrappedKey): void {
|
||||
$config = self::readConfig($this->backend);
|
||||
@@ -392,9 +540,41 @@ final class Repository {
|
||||
self::restoreEmptyDir($target, $dest, $path, $mtime);
|
||||
}
|
||||
}
|
||||
// Very last, nothing is written in them any more. Also on a run that goes on after a stop, and
|
||||
// the deepest first. The time is not worth stopping for: all the files are there by now.
|
||||
foreach (array_reverse($this->folderTimes($snapshotId)) as [$path, $mtime]) {
|
||||
if (!self::underPrefix($path, $prefix) || ($stripPrefix && $path === rtrim($prefix, '/'))) {
|
||||
continue; // the target itself is not ours to change
|
||||
}
|
||||
if ($heartbeat !== null) {
|
||||
$heartbeat();
|
||||
}
|
||||
$dest = self::destination($target, $path, $prefix, $stripPrefix);
|
||||
@touch(self::directoryInTarget($target, $dest, $path), $mtime);
|
||||
}
|
||||
return $count;
|
||||
}
|
||||
|
||||
/**
|
||||
* The folders with something in them that a snapshot recorded, parents before children (none
|
||||
* before 1.1.0). Damage here is no reason to give up on the files of a restore.
|
||||
*
|
||||
* @return list<array{0: string, 1: int}> path, modification time
|
||||
*/
|
||||
public function folderTimes(string $snapshotId): array {
|
||||
$meta = $this->lastMeta[$snapshotId] ?? $this->snapshotMeta($snapshotId);
|
||||
$packed = is_string($meta['folderTimes'] ?? null) ? base64_decode($meta['folderTimes'], true) : false;
|
||||
$plain = $packed === false ? false : @gzinflate($packed);
|
||||
$folders = [];
|
||||
foreach ($plain === false ? [] : explode("\n", $plain) as $line) {
|
||||
$folder = json_decode($line, true);
|
||||
if (is_array($folder) && is_string($folder[0] ?? null) && self::isSafePath($folder[0]) && is_int($folder[1] ?? null)) {
|
||||
$folders[] = [$folder[0], $folder[1]];
|
||||
}
|
||||
}
|
||||
return $folders;
|
||||
}
|
||||
|
||||
/**
|
||||
* The folders without anything in them that a snapshot recorded (none before 1.0.3).
|
||||
*
|
||||
@@ -985,7 +1165,10 @@ final class Repository {
|
||||
for ($attempt = 1; $attempt <= 3; $attempt++) {
|
||||
$before = self::fileVersion($path);
|
||||
$blobs = [];
|
||||
$fh = fopen($path, 'rb');
|
||||
$fh = self::openForReading($path);
|
||||
if ($fh === false) {
|
||||
throw new RepositoryException('Cannot read ' . $path);
|
||||
}
|
||||
while (($data = self::readFull($fh, self::BLOB_SIZE)) !== '') {
|
||||
$blobs[] = $this->storeBlob($data, $packs, $stats);
|
||||
}
|
||||
|
||||
@@ -61,6 +61,16 @@ final class AlertService {
|
||||
'examples' => mb_strimwidth(implode(', ', array_merge($examples['unreadable'], $examples['changing'])), 0, 300, '…')]);
|
||||
}
|
||||
|
||||
/**
|
||||
* A backup that finished but left out files or folders because of their name.
|
||||
*
|
||||
* @param list<string> $examples
|
||||
*/
|
||||
public function runBadNames(Run $run, string $targetName, int $count, array $examples): void {
|
||||
$this->notifyAdmins('run_bad_names', 'run-' . $run->getId(), ['target' => $targetName, 'count' => (string)$count,
|
||||
'examples' => mb_strimwidth(implode(', ', $examples), 0, 300, '…')]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Remember how much is stored on a location (for the settings page) and warn administrators
|
||||
* once a day when it is at 80% of its limit or more.
|
||||
@@ -91,9 +101,36 @@ final class AlertService {
|
||||
return $v === '' ? null : (int)$v;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether it is time to remind administrators of a location without a schedule: once a week
|
||||
* while another location is still backed up, every day when none is. An hour of margin, so
|
||||
* that the moment of the cron run does not push a reminder to the next day.
|
||||
*/
|
||||
public static function offReminderDue(int $last, int $now, bool $othersActive): bool {
|
||||
return $now - $last >= ($othersActive ? 7 * 86400 : 86400) - 3600;
|
||||
}
|
||||
|
||||
public function checkOff(Target $target, bool $othersActive): void {
|
||||
if ((string)$target->getSchedule() !== '') {
|
||||
return;
|
||||
}
|
||||
$key = 'off_notified_' . $target->getId();
|
||||
$now = time();
|
||||
$last = max((int)$target->getScheduleSince(), $this->appConfig->getValueInt(Application::APP_ID, $key, 0));
|
||||
if (!self::offReminderDue($last, $now, $othersActive)) {
|
||||
return;
|
||||
}
|
||||
$this->appConfig->setValueInt(Application::APP_ID, $key, $now);
|
||||
$this->notifyAdmins('schedule_off', 'off-' . $target->getId() . '-' . date('Y-m-d', $now),
|
||||
['target' => $target->getName(), 'others' => $othersActive ? '1' : '0']);
|
||||
}
|
||||
|
||||
/** At most one notification per location per day. */
|
||||
public function checkStale(Target $target): void {
|
||||
$last = $this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
|
||||
if ((string)$target->getSchedule() === '') {
|
||||
return; // no schedule is a choice, as for a location that is only connected now and then
|
||||
}
|
||||
$last =$this->runs->lastFinished($target->getId(), BackupService::KIND_FULL);
|
||||
$lastTime = $last?->getFinishedAt() ?? $target->getCreatedAt();
|
||||
if (time() - $lastTime < self::STALE_AFTER || $this->runs->findRunning($target->getId(), BackupService::KIND_FULL) !== []) {
|
||||
return;
|
||||
|
||||
@@ -58,6 +58,18 @@ final class BackupService {
|
||||
$this->openedWith = $opened;
|
||||
}
|
||||
|
||||
/**
|
||||
* The running backups in the order they get their turn: first come, first served. The first one
|
||||
* runs, the others are queued; only one at a time reads the disks.
|
||||
*
|
||||
* @return list<Run>
|
||||
*/
|
||||
public function runsInTurn(): array {
|
||||
$running = $this->runs->findRunning(null, self::KIND_FULL);
|
||||
usort($running, static fn (Run $a, Run $b): int => [$a->getStartedAt(), $a->getId()] <=> [$b->getStartedAt(), $b->getId()]);
|
||||
return $running;
|
||||
}
|
||||
|
||||
public function start(Target $target, string $label = ''): Run {
|
||||
$label = trim($label);
|
||||
if (mb_strlen($label) > self::LABEL_MAX) {
|
||||
@@ -210,6 +222,13 @@ final class BackupService {
|
||||
$state = json_decode($run->getState(), true, 512, JSON_THROW_ON_ERROR);
|
||||
|
||||
if ($run->getPhase() === 'db') {
|
||||
if (!isset($state['dump'])) {
|
||||
// It may have been queued for hours: the restore point has the time of its data.
|
||||
$state['files']['started'] = gmdate('c');
|
||||
// Locations from before 1.1.0 get their sealed settings at their next backup.
|
||||
$this->targets->recordSettings($repo);
|
||||
$this->targets->catchUpWrappedKey($target, $repo);
|
||||
}
|
||||
$dumper = new DbDumper($this->db->getInner(), $this->config->getSystemValueString('dbtableprefix', 'oc_'));
|
||||
$dbStats = $state['dbStats'] ?? [];
|
||||
// With system cron the whole dump usually fits in one step (one transaction, fully
|
||||
@@ -253,8 +272,13 @@ final class BackupService {
|
||||
try {
|
||||
$this->alerts->storedBytes($target, $repo->storedBytes());
|
||||
$stats = $state['files']['stats'];
|
||||
$warnings = BackupRun::warnings($state['files']);
|
||||
if (($stats['badNames'] ?? 0) > 0) {
|
||||
$this->logger->warning(sprintf('NG Backup run %d to %s: %d left out because their name is not supported (rename them): %s',
|
||||
$run->getId(), $target->getName(), $stats['badNames'], implode(', ', $warnings['badNames'])));
|
||||
$this->alerts->runBadNames($run, $target->getName(), (int)$stats['badNames'], $warnings['badNames']);
|
||||
}
|
||||
if (($stats['unreadable'] ?? 0) > 0 || ($stats['changing'] ?? 0) > 0) {
|
||||
$warnings = BackupRun::warnings($state['files']);
|
||||
$this->logger->warning(sprintf('NG Backup run %d to %s: %d unreadable (missing from the snapshot): %s; %d changed while being read: %s',
|
||||
$run->getId(), $target->getName(), $stats['unreadable'] ?? 0, implode(', ', $warnings['unreadable']),
|
||||
$stats['changing'] ?? 0, implode(', ', $warnings['changing'])));
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Service;
|
||||
|
||||
use OCA\NgBackup\AppInfo\Application;
|
||||
use OCP\DB\QueryBuilder\IQueryBuilder;
|
||||
use OCP\IAppConfig;
|
||||
use OCP\IDBConnection;
|
||||
use Psr\Log\LoggerInterface;
|
||||
|
||||
/**
|
||||
* Keeps heavy background jobs of other apps from running while a backup reads the same disks, by
|
||||
* reserving them in Nextcloud's job list: no app is turned off, and a job that is running goes on.
|
||||
* Nextcloud itself gives up on a reservation after 12 hours, so a pause that is never ended, because
|
||||
* cron or this app is gone, ends by itself. It lasts at most MAX_SECONDS, then the jobs get REST_SECONDS.
|
||||
*/
|
||||
final class HeavyJobPause {
|
||||
public const ENABLED_KEY = 'pause_heavy_jobs';
|
||||
private const SINCE_KEY = 'heavy_jobs_paused_since';
|
||||
private const REST_KEY = 'heavy_jobs_rest_until';
|
||||
public const MAX_SECONDS = 6 * 3600;
|
||||
public const REST_SECONDS = 3600;
|
||||
/** Nextcloud runs a job again this long after it was reserved. */
|
||||
private const RESERVATION_SECONDS = 12 * 3600;
|
||||
|
||||
/** Class, or the start of a class name ending in a backslash => what to call it */
|
||||
private const JOBS = [
|
||||
'OCA\\Memories\\' => 'Memories',
|
||||
'OCA\\Files_Antivirus\\BackgroundJob\\BackgroundScanner' => 'Antivirus background scan',
|
||||
'OCA\\Recognize\\' => 'Recognize',
|
||||
'OCA\\PreviewGenerator\\' => 'Preview Generator',
|
||||
'OCA\\FullTextSearch\\' => 'Full text search',
|
||||
];
|
||||
|
||||
public const NOTHING = 'nothing';
|
||||
public const PAUSE = 'pause';
|
||||
public const RELEASE = 'release';
|
||||
public const REST = 'rest';
|
||||
|
||||
public function __construct(
|
||||
private IDBConnection $db,
|
||||
private IAppConfig $appConfig,
|
||||
private LoggerInterface $logger,
|
||||
) {
|
||||
}
|
||||
|
||||
/** @return self::NOTHING|self::PAUSE|self::RELEASE|self::REST */
|
||||
public static function decide(bool $enabled, bool $backupRunning, int $since, int $restUntil, int $now): string {
|
||||
if ($since > 0 && $now - $since >= self::MAX_SECONDS) {
|
||||
return self::REST;
|
||||
}
|
||||
if ($enabled && $backupRunning && $now >= $restUntil) {
|
||||
return self::PAUSE;
|
||||
}
|
||||
return $since > 0 ? self::RELEASE : self::NOTHING;
|
||||
}
|
||||
|
||||
/** Call regularly: pauses or ends the pause as needed. A problem here never stops a backup. */
|
||||
public function sync(bool $backupRunning): void {
|
||||
try {
|
||||
$this->apply($backupRunning, time());
|
||||
} catch (\Throwable $e) {
|
||||
$this->logger->warning('NG Backup: pausing heavy background jobs failed: ' . $e->getMessage(), ['exception' => $e]);
|
||||
}
|
||||
}
|
||||
|
||||
private function apply(bool $backupRunning, int $now): void {
|
||||
$since = $this->appConfig->getValueInt(Application::APP_ID, self::SINCE_KEY, 0);
|
||||
$restUntil = $this->appConfig->getValueInt(Application::APP_ID, self::REST_KEY, 0);
|
||||
switch (self::decide($this->enabled(), $backupRunning, $since, $restUntil, $now)) {
|
||||
case self::PAUSE:
|
||||
if ($since === 0) {
|
||||
$since = $now;
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::SINCE_KEY, $since);
|
||||
$this->logger->info('NG Backup: heavy background jobs paused while a backup runs');
|
||||
}
|
||||
$this->reserve($since, $now); // also the jobs that were added or running until now
|
||||
break;
|
||||
case self::REST:
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::REST_KEY, $now + self::REST_SECONDS);
|
||||
$this->release($since);
|
||||
$this->logger->info('NG Backup: heavy background jobs get their turn for an hour during a long backup');
|
||||
break;
|
||||
case self::RELEASE:
|
||||
$this->release($since);
|
||||
$this->logger->info('NG Backup: heavy background jobs run again');
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private function reserve(int $stamp, int $now): void {
|
||||
$qb = $this->db->getQueryBuilder();
|
||||
$qb->update('jobs')
|
||||
->set('reserved_at', $qb->createNamedParameter($stamp, IQueryBuilder::PARAM_INT))
|
||||
->where($qb->expr()->lte('reserved_at', $qb->createNamedParameter($now - self::RESERVATION_SECONDS, IQueryBuilder::PARAM_INT)))
|
||||
->andWhere($this->isHeavy($qb));
|
||||
$qb->executeStatement();
|
||||
}
|
||||
|
||||
/** Only our own reservations: a job that Nextcloud reserved at another moment is running. */
|
||||
private function release(int $stamp): void {
|
||||
$qb = $this->db->getQueryBuilder();
|
||||
$qb->update('jobs')
|
||||
->set('reserved_at', $qb->createNamedParameter(0, IQueryBuilder::PARAM_INT))
|
||||
->where($qb->expr()->eq('reserved_at', $qb->createNamedParameter($stamp, IQueryBuilder::PARAM_INT)))
|
||||
->andWhere($this->isHeavy($qb));
|
||||
$qb->executeStatement();
|
||||
$this->appConfig->deleteKey(Application::APP_ID, self::SINCE_KEY);
|
||||
}
|
||||
|
||||
/** @return \OCP\DB\QueryBuilder\ICompositeExpression */
|
||||
private function isHeavy(IQueryBuilder $qb) {
|
||||
$conditions = [];
|
||||
foreach (array_keys(self::JOBS) as $class) {
|
||||
$conditions[] = str_ends_with($class, '\\')
|
||||
? $qb->expr()->like('class', $qb->createNamedParameter($this->db->escapeLikeParameter($class) . '%'))
|
||||
: $qb->expr()->eq('class', $qb->createNamedParameter($class));
|
||||
}
|
||||
return $qb->expr()->orX(...$conditions);
|
||||
}
|
||||
|
||||
public function enabled(): bool {
|
||||
return $this->appConfig->getValueString(Application::APP_ID, self::ENABLED_KEY, '1') !== '0';
|
||||
}
|
||||
|
||||
public function setEnabled(bool $enabled): void {
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::ENABLED_KEY, $enabled ? '1' : '0');
|
||||
}
|
||||
|
||||
/** @return array{enabled: bool, since: int, jobs: list<string>} what is paused now */
|
||||
public function status(): array {
|
||||
$since = $this->appConfig->getValueInt(Application::APP_ID, self::SINCE_KEY, 0);
|
||||
$jobs = [];
|
||||
if ($since > 0) {
|
||||
$qb = $this->db->getQueryBuilder();
|
||||
$qb->selectDistinct('class')->from('jobs')
|
||||
->where($qb->expr()->eq('reserved_at', $qb->createNamedParameter($since, IQueryBuilder::PARAM_INT)))
|
||||
->andWhere($this->isHeavy($qb));
|
||||
$result = $qb->executeQuery();
|
||||
while (($class = $result->fetchOne()) !== false) {
|
||||
foreach (self::JOBS as $start => $name) {
|
||||
if (str_starts_with((string)$class, $start)) {
|
||||
$jobs[$name] = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
$result->closeCursor();
|
||||
}
|
||||
return ['enabled' => $this->enabled(), 'since' => $since, 'jobs' => array_map('strval', array_keys($jobs))];
|
||||
}
|
||||
}
|
||||
+85
-10
@@ -30,6 +30,9 @@ final class KeyService {
|
||||
private const KEY_DELETE_DELAY = 'delete_delay_days';
|
||||
private const KEY_KIT_VERSION = 'recovery_kit_version';
|
||||
private const KEY_KIT_CODE = 'recovery_kit_code';
|
||||
/** Set once a location was created or adopted with this key: from then on it is never replaced. */
|
||||
private const KEY_USED = 'master_key_used';
|
||||
private const KEY_WRAPPED_PENDING = 'master_key_wrapped_pending';
|
||||
public const DEFAULT_DELETE_DELAY = 7;
|
||||
/**
|
||||
* The location holds the passphrase-wrapped key too, so whoever can read it can try
|
||||
@@ -83,28 +86,100 @@ final class KeyService {
|
||||
* @param array<string, mixed> $kit as produced by recoveryKit()
|
||||
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase, or a damaged/foreign kit
|
||||
*/
|
||||
public function importRecoveryKit(array $kit, #[\SensitiveParameter] string $passphrase): void {
|
||||
if ($this->isInitialized()) {
|
||||
public function importRecoveryKit(array $kit, #[\SensitiveParameter] string $passphrase, bool $replaceUnused = false): void {
|
||||
if ($this->isInitialized() && !$replaceUnused) {
|
||||
throw new \RuntimeException('A backup key already exists');
|
||||
}
|
||||
if (!isset($kit['wrapped_key'])) {
|
||||
throw new \InvalidArgumentException('Not a NG Backup recovery kit file');
|
||||
}
|
||||
$keys = KeyRing::unwrapAny($kit['wrapped_key'], $passphrase);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
|
||||
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
|
||||
$this->cached = $keys;
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode($kit['wrapped_key'], JSON_THROW_ON_ERROR), true, true);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
|
||||
$this->discardUnused($keys, $replaceUnused);
|
||||
// Kits from before this field existed fall back to the default.
|
||||
$delay = (int)($kit['delete_delay_days'] ?? self::DEFAULT_DELETE_DELAY);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY, ($delay >= 1 && $delay <= 365) ? $delay : self::DEFAULT_DELETE_DELAY, true);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, (int)($kit['kit_version'] ?? 1), true);
|
||||
$this->store($keys, $kit['wrapped_key'], (int)($kit['delete_delay_days'] ?? self::DEFAULT_DELETE_DELAY), (int)($kit['kit_version'] ?? 1));
|
||||
// Having and successfully using a real kit file is stronger proof than the normal
|
||||
// download-and-confirm flow it replaces.
|
||||
$this->confirmRecoveryKit('cli-import');
|
||||
}
|
||||
|
||||
/**
|
||||
* Disaster recovery without the recovery kit: adopt the master key that a passphrase opened
|
||||
* from the key slots in a location's config (TargetService::addWithPassphrase). There is no
|
||||
* kit, so none is confirmed: backups wait until a new kit is downloaded and confirmed.
|
||||
*
|
||||
* @param ?int $deleteDelayDays from the location's sealed settings, null when it has none
|
||||
*/
|
||||
public function adoptKey(KeyRing $keys, array $wrappedKey, ?int $deleteDelayDays, bool $replaceUnused = false): void {
|
||||
if ($this->isInitialized() && !$replaceUnused) {
|
||||
throw new \RuntimeException('A backup key already exists');
|
||||
}
|
||||
$this->discardUnused($keys, $replaceUnused);
|
||||
$this->store($keys, $wrappedKey, $deleteDelayDays ?? self::DEFAULT_DELETE_DELAY, 1);
|
||||
}
|
||||
|
||||
/** Whether a location was ever created or adopted with this key (TargetService::keyIsUnused() also looks at the database). */
|
||||
public function wasUsed(): bool {
|
||||
return $this->appConfig->getValueBool(Application::APP_ID, self::KEY_USED, false, true);
|
||||
}
|
||||
|
||||
public function markUsed(): void {
|
||||
if (!$this->wasUsed()) {
|
||||
$this->appConfig->setValueBool(Application::APP_ID, self::KEY_USED, true, true);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Locations that missed the last passphrase change (offline at the time): they still hold
|
||||
* the previous slots until the change reaches them, which is tried again at each backup.
|
||||
*/
|
||||
public function wrappedKeyPending(int $targetId): bool {
|
||||
return $this->appConfig->getValueBool(Application::APP_ID, self::KEY_WRAPPED_PENDING . '_' . $targetId);
|
||||
}
|
||||
|
||||
/** One flag per location, so that two processes never overwrite each other's. */
|
||||
public function setWrappedKeyPending(int $targetId, bool $pending): void {
|
||||
$key = self::KEY_WRAPPED_PENDING . '_' . $targetId;
|
||||
if ($pending) {
|
||||
$this->appConfig->setValueBool(Application::APP_ID, $key, true);
|
||||
} elseif ($this->wrappedKeyPending($targetId)) {
|
||||
$this->appConfig->deleteKey(Application::APP_ID, $key);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A key made by mistake on a server that is being restored: remove it, so the key of the
|
||||
* backups can take its place. Only after the new key was opened with its passphrase, and
|
||||
* only when the caller checked that the old key never protected anything
|
||||
* (TargetService::keyIsUnused()). Its passphrases and recovery kit become worthless.
|
||||
*/
|
||||
private function discardUnused(KeyRing $replacement, bool $replaceUnused): void {
|
||||
if (!$replaceUnused || !$this->isInitialized()) {
|
||||
return;
|
||||
}
|
||||
if ($this->wasUsed()) {
|
||||
throw new \RuntimeException('The backup key of this server has been used for a backup location: it is never replaced');
|
||||
}
|
||||
if ($this->fingerprintOf($replacement) === $this->fingerprint()) {
|
||||
throw new \RuntimeException('This server already has this backup key: nothing to replace');
|
||||
}
|
||||
foreach ([self::KEY_MASTER, self::KEY_WRAPPED, self::KEY_CONFIRMED, self::KEY_FINGERPRINT, self::KEY_DELETE_DELAY,
|
||||
self::KEY_KIT_VERSION, self::KEY_KIT_CODE, self::KEY_USED] as $key) {
|
||||
$this->appConfig->deleteKey(Application::APP_ID, $key);
|
||||
}
|
||||
$this->cached = null;
|
||||
}
|
||||
|
||||
private function store(KeyRing $keys, array $wrappedKey, int $deleteDelayDays, int $kitVersion): void {
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_MASTER,
|
||||
$this->crypto->encrypt(base64_encode($keys->exportMasterKey())), true, true);
|
||||
$this->cached = $keys;
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_WRAPPED, json_encode($wrappedKey, JSON_THROW_ON_ERROR), true, true);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::KEY_FINGERPRINT, $this->fingerprintOf($keys), true);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_DELETE_DELAY,
|
||||
($deleteDelayDays >= 1 && $deleteDelayDays <= 365) ? $deleteDelayDays : self::DEFAULT_DELETE_DELAY, true);
|
||||
$this->appConfig->setValueInt(Application::APP_ID, self::KEY_KIT_VERSION, max(1, $kitVersion), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* The master key and the recovery kit code are stored encrypted with Nextcloud's own
|
||||
* instance secret (ICrypto), not with themselves; the wrapped key is not (it is sealed with
|
||||
|
||||
@@ -43,8 +43,10 @@ final class KeySlotService {
|
||||
foreach ($this->targets->list() as $t) {
|
||||
try {
|
||||
$this->targets->repository($t)->updateWrappedKey($wrapped);
|
||||
$this->keys->setWrappedKeyPending($t->getId(), false);
|
||||
} catch (\Throwable $e) {
|
||||
$failed[] = $t->getName();
|
||||
$this->keys->setWrappedKeyPending($t->getId(), true);
|
||||
}
|
||||
}
|
||||
$label = '';
|
||||
@@ -55,7 +57,7 @@ final class KeySlotService {
|
||||
}
|
||||
$this->alerts->securityEvent($event, ['slot' => (string)$slot, 'label' => $label]);
|
||||
if ($failed !== []) {
|
||||
throw new \RuntimeException('Saved, but these locations could not be updated (they still accept the previous passphrases): ' . implode(', ', $failed));
|
||||
throw new \RuntimeException('Saved, but these locations could not be updated (they still accept the previous passphrases; the change is tried again at their next backup): ' . implode(', ', $failed));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Service;
|
||||
|
||||
/**
|
||||
* When a location is backed up: "" (off), "HH:MM" (daily) or "HH:MM/N" (from HH:MM every N hours,
|
||||
* starting over at HH:MM the next day), in the location's time zone.
|
||||
*/
|
||||
final class LocationSchedule {
|
||||
/** Attempts per moment after a failure, an hour apart, instead of one every few minutes. */
|
||||
public const MAX_ATTEMPTS = 3;
|
||||
public const RETRY_AFTER = 3600;
|
||||
|
||||
/** @throws \InvalidArgumentException */
|
||||
public static function normalize(string $schedule): string {
|
||||
$schedule = trim($schedule);
|
||||
if ($schedule === '' || strtolower($schedule) === 'off') {
|
||||
return '';
|
||||
}
|
||||
if (preg_match('/^(([01]\d|2[0-3]):[0-5]\d)(?:\/(\d{1,2}))?$/', $schedule, $m) !== 1) {
|
||||
throw new \InvalidArgumentException('Use "off", HH:MM for once a day, or HH:MM/N for every N hours from HH:MM');
|
||||
}
|
||||
$hours = (int)($m[3] ?? 24);
|
||||
if ($hours < 1 || $hours > 24) {
|
||||
throw new \InvalidArgumentException('Every 1 to 24 hours');
|
||||
}
|
||||
return $hours === 24 ? $m[1] : $m[1] . '/' . $hours;
|
||||
}
|
||||
|
||||
/**
|
||||
* The most recent scheduled moment at or before $now; null without a schedule.
|
||||
*
|
||||
* @param non-empty-string $zone
|
||||
*/
|
||||
public static function lastMoment(string $schedule, string $zone, int $now): ?int {
|
||||
if ($schedule === '') {
|
||||
return null;
|
||||
}
|
||||
$tz = new \DateTimeZone($zone);
|
||||
$today = (new \DateTimeImmutable('@' . $now))->setTimezone($tz);
|
||||
$moments = array_merge(self::moments($schedule, $today->modify('-1 day'), $tz), self::moments($schedule, $today, $tz));
|
||||
$last = null;
|
||||
foreach ($moments as $moment) {
|
||||
if ($moment <= $now && ($last === null || $moment > $last)) {
|
||||
$last = $moment;
|
||||
}
|
||||
}
|
||||
return $last;
|
||||
}
|
||||
|
||||
/** @return list<int> the moments of one day */
|
||||
private static function moments(string $schedule, \DateTimeImmutable $day, \DateTimeZone $tz): array {
|
||||
[$time, $every] = array_pad(explode('/', $schedule), 2, '24');
|
||||
[$hour, $minute] = array_map('intval', explode(':', $time));
|
||||
$moments = [];
|
||||
for ($h = $hour; $h < 24; $h += (int)$every) {
|
||||
$moments[] = (new \DateTimeImmutable($day->format('Y-m-d') . sprintf(' %02d:%02d', $h, $minute), $tz))->getTimestamp();
|
||||
}
|
||||
return $moments;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a backup has to start for $moment. One that was made since then is enough, also when
|
||||
* several moments went by during a long backup. A moment from before the schedule was set does
|
||||
* not count: choosing a time must not start a backup right away.
|
||||
*/
|
||||
public static function isDue(int $moment, int $since, ?int $lastDoneStarted, int $attempts, ?int $lastAttemptFinished, int $now): bool {
|
||||
if ($moment < $since || ($lastDoneStarted !== null && $lastDoneStarted >= $moment) || $attempts >= self::MAX_ATTEMPTS) {
|
||||
return false;
|
||||
}
|
||||
return $attempts === 0 || $now - (int)$lastAttemptFinished >= self::RETRY_AFTER;
|
||||
}
|
||||
}
|
||||
@@ -11,6 +11,7 @@ use OCA\NgBackup\AppInfo\Application;
|
||||
use OCA\NgBackup\Db\RunMapper;
|
||||
use OCA\NgBackup\Db\SnapshotMapper;
|
||||
use OCA\NgBackup\Db\Target;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCP\IAppConfig;
|
||||
use OCP\Lock\ILockingProvider;
|
||||
use OCP\Lock\LockedException;
|
||||
@@ -33,6 +34,54 @@ final class PruneService {
|
||||
) {
|
||||
}
|
||||
|
||||
/** Characters of a note on a snapshot (the column holds 1000) */
|
||||
public const NOTE_MAX = 500;
|
||||
|
||||
/** Pin or unpin a snapshot: the retention policy leaves a pinned one alone. */
|
||||
public function setPinned(Target $target, string $snapshotId, bool $pinned): void {
|
||||
$changed = $this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $pinned): bool {
|
||||
$was = in_array($snapshotId, $repo->pinnedSnapshots(), true);
|
||||
$repo->setPinned($snapshotId, $pinned);
|
||||
return $was !== $pinned;
|
||||
});
|
||||
$this->targets->syncSnapshot($target, $snapshotId, pinned: $pinned);
|
||||
if ($changed) {
|
||||
// Unpinning opens the way to removing it, so every administrator hears about that.
|
||||
$this->alerts->securityEvent($pinned ? 'snapshot_pinned' : 'snapshot_unpinned', ['target' => $target->getName(), 'snapshot' => $snapshotId], !$pinned);
|
||||
}
|
||||
}
|
||||
|
||||
public function setNote(Target $target, string $snapshotId, string $note): void {
|
||||
$note = trim((string)preg_replace('/[\x00-\x1f\x7f]+/', ' ', $note));
|
||||
if (!mb_check_encoding($note, 'UTF-8')) {
|
||||
throw new \InvalidArgumentException('A note must be valid UTF-8');
|
||||
}
|
||||
if (mb_strlen($note) > self::NOTE_MAX) {
|
||||
throw new \InvalidArgumentException('A note has at most ' . self::NOTE_MAX . ' characters');
|
||||
}
|
||||
$this->changeCatalog($target, function (Repository $repo) use ($snapshotId, $note): bool {
|
||||
$repo->setSnapshotNote($snapshotId, $note);
|
||||
return true;
|
||||
});
|
||||
$this->targets->syncSnapshot($target, $snapshotId, note: $note);
|
||||
}
|
||||
|
||||
/**
|
||||
* Write to the catalog next to a running backup or restore, which only takes it for a moment to
|
||||
* commit. Not during a cleanup, which holds a repository opened earlier and writes from that.
|
||||
*
|
||||
* @param callable(Repository): bool $change
|
||||
*/
|
||||
private function changeCatalog(Target $target, callable $change): bool {
|
||||
try {
|
||||
return $this->leases->with(self::lockKey($target), LeaseService::SHARED, 300,
|
||||
fn (): bool => $this->leases->with(self::catalogLockKey($target), LeaseService::EXCLUSIVE, 300,
|
||||
fn (): bool => $change($this->targets->repository($target))));
|
||||
} catch (LockedException) {
|
||||
throw new \RuntimeException('A cleanup of ' . $target->getName() . ' or another change to its restore points is in progress; try again later');
|
||||
}
|
||||
}
|
||||
|
||||
public function untrash(Target $target, string $snapshotId): void {
|
||||
$lease = $this->lockExclusive($target, 300);
|
||||
try {
|
||||
@@ -87,6 +136,7 @@ final class PruneService {
|
||||
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
||||
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
||||
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
||||
$s->setNote($repo->snapshotNotes()[$snapshotId] ?? null);
|
||||
try {
|
||||
$this->snapshots->insert($s);
|
||||
} catch (\Throwable) {
|
||||
@@ -208,7 +258,7 @@ final class PruneService {
|
||||
$meta = $repo->snapshotMeta($id);
|
||||
$times[$id] = (int)strtotime((string)($meta['time'] ?? '@0'));
|
||||
}
|
||||
$keep = $this->policy()->keep($times);
|
||||
$keep = $this->policy()->keepWithPinned($times, $repo->pinnedSnapshots());
|
||||
$forget = array_values(array_diff(array_keys($times), array_keys($keep)));
|
||||
$actor = $scheduled ? 'retention policy' : $this->alerts->actor();
|
||||
$purged = [];
|
||||
|
||||
@@ -155,6 +155,17 @@ final class RestoreService {
|
||||
$dest = $this->ensureFolder($userFolder, $parentRel);
|
||||
$names = $single ? [basename($rel) => $entries['']] : $entries;
|
||||
|
||||
// The folders with something in them that this restore creates get their time back; one that
|
||||
// was already there keeps its own, as its content changes now. Noted before anything is written.
|
||||
/** @var array<array-key, int> $newFolders a folder called "2026" is an integer key */
|
||||
$newFolders = [];
|
||||
foreach ($single ? [] : $repo->folderTimes($snapshotId) as [$dir, $mtime]) {
|
||||
$relative = $dir === $source ? '' : (str_starts_with($dir, $source . '/') ? substr($dir, strlen($source) + 1) : null);
|
||||
if ($relative !== null && ($relative === '' ? !$destExisted : !$dest->nodeExists($relative))) {
|
||||
$newFolders[$relative] = $mtime;
|
||||
}
|
||||
}
|
||||
|
||||
$this->cleaned = [];
|
||||
$restored = 0;
|
||||
foreach ($names as $relPath => $entry) {
|
||||
@@ -190,6 +201,13 @@ final class RestoreService {
|
||||
if ($mode === self::MODE_REPLACE && !$single) {
|
||||
$trashed = $this->trashExtras($dest, '', array_flip(array_keys($names)) + $emptyDirs);
|
||||
}
|
||||
// Last and deepest first: a change in a folder also touches the folders above it.
|
||||
foreach (array_reverse($newFolders, true) as $dir => $mtime) {
|
||||
$this->ensureFolder($dest, (string)$dir)->touch($mtime);
|
||||
if ($progress !== null) {
|
||||
$progress($restored, count($names), (string)$dir); // keeps the lease alive too
|
||||
}
|
||||
}
|
||||
return ['restored' => $restored, 'trashed' => $trashed, 'target' => $uid . ':' . $userFolder->getRelativePath($dest->getPath())];
|
||||
}
|
||||
|
||||
|
||||
@@ -28,6 +28,19 @@ final class RetentionPolicy {
|
||||
return ['last' => $this->last, 'daily' => $this->daily, 'weekly' => $this->weekly, 'monthly' => $this->monthly];
|
||||
}
|
||||
|
||||
/**
|
||||
* keep() for the snapshots that are not pinned, plus the pinned ones. A pinned snapshot does not
|
||||
* take the place of a daily, weekly or monthly one.
|
||||
*
|
||||
* @param array<string, int> $snapshots id => unix time
|
||||
* @param list<string> $pinned
|
||||
* @return array<string, list<string>>
|
||||
*/
|
||||
public function keepWithPinned(array $snapshots, array $pinned): array {
|
||||
$pins = array_intersect_key($snapshots, array_flip($pinned));
|
||||
return $this->keep(array_diff_key($snapshots, $pins)) + array_fill_keys(array_keys($pins), ['pinned']);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param array<string, int> $snapshots id => unix time
|
||||
* @return array<string, list<string>> id => reasons it is kept (ids not in the result are removed)
|
||||
|
||||
@@ -13,10 +13,11 @@ use OCP\IConfig;
|
||||
use OCP\IGroupManager;
|
||||
|
||||
/**
|
||||
* The daily backup time: "HH:MM" in a time zone. Nextcloud runs PHP in UTC, so a time without a
|
||||
* zone meant UTC (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone
|
||||
* is converted once to the zone of the server or of an administrator, at today's offset: it then
|
||||
* shows as it effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
||||
* The time zone times are shown in, and the daily time from before each location had its own
|
||||
* schedule (see LocationSchedule). Nextcloud runs PHP in UTC, so a time without a zone meant UTC
|
||||
* (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone is converted once
|
||||
* to the zone of the server or of an administrator, at today's offset: it then shows as it
|
||||
* effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
||||
*/
|
||||
final class Schedule {
|
||||
public const TIME_KEY = 'schedule_time';
|
||||
@@ -44,27 +45,9 @@ final class Schedule {
|
||||
return ['time' => $time, 'timezone' => $zone];
|
||||
}
|
||||
|
||||
/** @throws \InvalidArgumentException for a malformed time or an unknown zone */
|
||||
public function set(string $time, string $zone): void {
|
||||
if ($time !== '' && !self::validTime($time)) {
|
||||
throw new \InvalidArgumentException('Use HH:MM or empty');
|
||||
}
|
||||
if (!self::validZone($zone)) {
|
||||
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
||||
}
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
|
||||
}
|
||||
|
||||
/** Today's scheduled moment as a Unix time, in the schedule's zone; null without a schedule. */
|
||||
public function todayAt(?int $now = null): ?int {
|
||||
['time' => $time, 'timezone' => $zone] = $this->get();
|
||||
if ($time === '' || !self::validTime($time)) {
|
||||
return null;
|
||||
}
|
||||
$tz = new \DateTimeZone($zone);
|
||||
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone($tz)->format('Y-m-d');
|
||||
return (new \DateTimeImmutable($day . ' ' . $time, $tz))->getTimestamp();
|
||||
/** @return non-empty-string the zone of a location's schedule, or else this one */
|
||||
public function zoneOr(?string $zone): string {
|
||||
return $zone !== null && self::validZone($zone) ? $zone : $this->get()['timezone'];
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
+219
-11
@@ -27,6 +27,9 @@ use OCP\Server;
|
||||
* classes but its enabled/disabled state is left as the admin set it.
|
||||
*/
|
||||
final class TargetService {
|
||||
/** Deletion delay taken when a location's own cannot be read: nothing leaves the Bin sooner than on the old server. */
|
||||
public const LONGEST_DELETE_DELAY = 365;
|
||||
|
||||
public function __construct(
|
||||
private TargetMapper $mapper,
|
||||
private KeyService $keys,
|
||||
@@ -35,6 +38,8 @@ final class TargetService {
|
||||
private \OCA\NgBackup\Db\DbIndexCache $indexCache,
|
||||
private \OCA\NgBackup\Db\AppConfigCatalogAnchor $anchor,
|
||||
private SnapshotMapper $snapshots,
|
||||
private AlertService $alerts,
|
||||
private \OCA\NgBackup\Db\RunMapper $runs,
|
||||
) {
|
||||
}
|
||||
|
||||
@@ -51,18 +56,31 @@ final class TargetService {
|
||||
* @param ?callable(int, int): void $progress reading an existing repository's index: files done, and in all
|
||||
* @return array{target: Target, created: bool}
|
||||
*/
|
||||
public function add(string $name, string $backend, string $auth, array $options, string $basePath, ?callable $progress = null): array {
|
||||
public function add(string $name, string $backend, string $auth, array $options, string $basePath, ?callable $progress = null, bool $repairConfig = false): array {
|
||||
$this->validateName($name);
|
||||
$this->requireUsableBackend($backend);
|
||||
$keys = $this->keys->keyRing();
|
||||
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath);
|
||||
if ($storageBackend->exists('config')) {
|
||||
$repo = Repository::openWithKey($storageBackend, $keys, $this->indexCache, $this->anchor, $progress);
|
||||
$created = false;
|
||||
} else {
|
||||
$repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey(), $this->anchor);
|
||||
$created = true;
|
||||
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath, !$repairConfig);
|
||||
try {
|
||||
if ($repairConfig) {
|
||||
Repository::repairConfig($storageBackend, $keys, $this->keys->wrappedKey());
|
||||
}
|
||||
if ($storageBackend->exists('config')) {
|
||||
$repo = Repository::openWithKey($storageBackend, $keys, $this->indexCache, $this->anchor, $progress);
|
||||
$created = false;
|
||||
} else {
|
||||
$repo = Repository::initWithKey($storageBackend, $keys, $this->keys->wrappedKey(), $this->anchor);
|
||||
$created = true;
|
||||
}
|
||||
} catch (\OCA\NgBackup\Repository\RepositoryException $e) {
|
||||
$otherKey = str_contains($e->getMessage(), 'different key') || str_contains($e->getMessage(), 'do not open with this key');
|
||||
if ($otherKey && $this->keyIsUnused()) {
|
||||
throw new \OCA\NgBackup\Repository\RepositoryException(rtrim($e->getMessage(), '.') . '. ' . self::UNUSED_KEY_HINT, 0, $e);
|
||||
}
|
||||
throw $e;
|
||||
}
|
||||
$this->keys->markUsed();
|
||||
$this->recordSettings($repo);
|
||||
|
||||
foreach ($this->mapper->findAll() as $existing) {
|
||||
if ($existing->getRepositoryId() === $repo->id()) {
|
||||
@@ -78,6 +96,11 @@ final class TargetService {
|
||||
$target->setBasePath(trim($basePath, '/'));
|
||||
$target->setRepositoryId($repo->id());
|
||||
$target->setCreatedAt(time());
|
||||
// Like the location before it: a backup scheme is usually one scheme.
|
||||
$first = $this->mapper->findAll()[0] ?? null;
|
||||
$target->setSchedule($first?->getSchedule() ?? '');
|
||||
$target->setScheduleZone($first?->getScheduleZone());
|
||||
$target->setScheduleSince(time());
|
||||
$target = $this->mapper->insert($target);
|
||||
if (!$created) {
|
||||
// Disaster recovery: occ backup:list reads this local cache, not the repository
|
||||
@@ -87,6 +110,128 @@ final class TargetService {
|
||||
return ['target' => $target, 'created' => $created];
|
||||
}
|
||||
|
||||
public const UNUSED_KEY_HINT = 'The backup key of this server was never used (no location, no backup), so it can make way for the key of these backups: '
|
||||
. 'with the old server\'s recovery kit, occ backup:key:import <kit file> --replace-unused-key; without the kit, '
|
||||
. 'occ backup:target:add with the same details and --replace-unused-key (asks for a passphrase of the old server)';
|
||||
|
||||
/**
|
||||
* A key that never protected anything: no location was created or adopted with it, and the
|
||||
* database has no location and no run (installations from before the flag existed). Only such
|
||||
* a key may make way for the key of existing backups, e.g. after a new key was made by
|
||||
* mistake on a server that is being restored.
|
||||
*/
|
||||
public function keyIsUnused(): bool {
|
||||
return $this->keys->isInitialized() && !$this->keys->wasUsed()
|
||||
&& $this->mapper->findAll() === [] && $this->runs->countAll() === 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* occ backup:key:import on a server whose own key was never used: the kit's key takes its place.
|
||||
*
|
||||
* @param array<string, mixed> $kit
|
||||
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase: nothing changed
|
||||
*/
|
||||
public function importKitReplacingUnusedKey(array $kit, #[\SensitiveParameter] string $passphrase): void {
|
||||
$old = $this->requireUnusedKey();
|
||||
$this->keys->importRecoveryKit($kit, $passphrase, true);
|
||||
$this->alerts->securityEvent('key_replaced', ['old' => $old, 'new' => $this->keys->fingerprint()]);
|
||||
}
|
||||
|
||||
/** @return string the fingerprint of the unused key */
|
||||
private function requireUnusedKey(): string {
|
||||
if (!$this->keys->isInitialized()) {
|
||||
throw new \RuntimeException('There is no backup key here to replace');
|
||||
}
|
||||
if (!$this->keyIsUnused()) {
|
||||
throw new \RuntimeException('The backup key of this server (fingerprint ' . $this->keys->fingerprint() . ') has been used for a backup location: it is never replaced');
|
||||
}
|
||||
return $this->keys->fingerprint();
|
||||
}
|
||||
|
||||
public const LOCATION_REPOSITORY = 'repository';
|
||||
public const LOCATION_DATA_WITHOUT_CONFIG = 'data-without-config';
|
||||
public const LOCATION_EMPTY = 'empty';
|
||||
|
||||
/**
|
||||
* What these settings lead to: a repository, backup data whose config is gone, or nothing
|
||||
* of NG Backup. Creates nothing.
|
||||
*
|
||||
* @return self::LOCATION_*
|
||||
*/
|
||||
public function locationState(string $backend, string $auth, array $options, string $basePath): string {
|
||||
$this->requireUsableBackend($backend);
|
||||
$storage = $this->buildBackend($backend, $auth, $options, $basePath, false);
|
||||
if ($storage->exists('config')) {
|
||||
return self::LOCATION_REPOSITORY;
|
||||
}
|
||||
return Repository::holdsData($storage) ? self::LOCATION_DATA_WITHOUT_CONFIG : self::LOCATION_EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* Disaster recovery without the recovery kit, on an installation without a backup key: take
|
||||
* the key from the location's config with one of its passphrases, then add the location as
|
||||
* usual. The key is only kept when it really is the one the repository was made with.
|
||||
* With $replaceUnusedKey, a key this server has but never used makes way (keyIsUnused()).
|
||||
*
|
||||
* @param array<string, mixed> $options
|
||||
* @param ?callable(int, int): void $progress
|
||||
* @return array{target: Target, created: bool, deleteDelayDays: ?int, settingsUnreadable: bool} the deletion delay the location had stored, if any, and whether its settings could not be read
|
||||
* @throws \OCA\NgBackup\Crypto\CryptoException wrong passphrase
|
||||
*/
|
||||
public function addWithPassphrase(string $name, string $backend, string $auth, array $options, string $basePath,
|
||||
#[\SensitiveParameter] string $passphrase, ?callable $progress = null, bool $replaceUnusedKey = false): array {
|
||||
$old = null;
|
||||
if ($replaceUnusedKey) {
|
||||
$old = $this->requireUnusedKey();
|
||||
} elseif ($this->keys->isInitialized()) {
|
||||
throw new \RuntimeException('A backup key already exists here');
|
||||
}
|
||||
$this->validateName($name);
|
||||
$this->requireUsableBackend($backend);
|
||||
$storageBackend = $this->buildBackend($backend, $auth, $options, $basePath, false);
|
||||
if (!$storageBackend->exists('config')) {
|
||||
throw new \InvalidArgumentException("No NG Backup repository found in folder '" . trim($basePath, '/') . "'");
|
||||
}
|
||||
$wrapped = Repository::wrappedKeyAt($storageBackend);
|
||||
$keys = \OCA\NgBackup\Crypto\KeyRing::unwrapAny($wrapped, $passphrase);
|
||||
if (!Repository::madeWithKey($storageBackend, $keys)) {
|
||||
throw new \RuntimeException('The key in the config at this location does not open its backups: the config is damaged or was replaced. Only the recovery kit can help now: occ backup:key:import <kit file>, then this command again.');
|
||||
}
|
||||
$settings = Repository::settingsAt($storageBackend, $keys);
|
||||
$stored = $settings['delete_delay_days'] ?? null;
|
||||
$delay = is_int($stored) && $stored >= 1 && $stored <= 365 ? $stored : null;
|
||||
// Settings that are there but cannot be read, or hold no valid delay (also none at all): the old server's delay
|
||||
// is unknown, so take the longest one; a shorter guess could empty the Bin too soon.
|
||||
$unknown = ($settings['unreadable'] ?? false) === true || ($settings !== [] && $delay === null);
|
||||
$this->keys->adoptKey($keys, $wrapped, $delay ?? ($unknown ? self::LONGEST_DELETE_DELAY : null), $replaceUnusedKey);
|
||||
if ($old !== null) {
|
||||
$this->alerts->securityEvent('key_replaced', ['old' => $old, 'new' => $this->keys->fingerprint()]);
|
||||
}
|
||||
return $this->add($name, $backend, $auth, $options, $basePath, $progress) + ['deleteDelayDays' => $delay, 'settingsUnreadable' => $unknown];
|
||||
}
|
||||
|
||||
/** A passphrase change that missed this location (offline at the time) reaches it now. A failure is tried again at the next backup. */
|
||||
public function catchUpWrappedKey(Target $target, Repository $repo): void {
|
||||
if (!$this->keys->wrappedKeyPending($target->getId())) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
$repo->updateWrappedKey($this->keys->wrappedKey());
|
||||
$this->keys->setWrappedKeyPending($target->getId(), false);
|
||||
} catch (\Throwable $e) {
|
||||
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: the last passphrase change still does not reach ' . $target->getName()
|
||||
. ' (it keeps the previous passphrases); tried again at its next backup: ' . $e->getMessage(), ['exception' => $e]);
|
||||
}
|
||||
}
|
||||
|
||||
/** Seal the settings fixed at installation on the location, once. A failure is tried again at the next backup. */
|
||||
public function recordSettings(Repository $repo): void {
|
||||
try {
|
||||
$repo->recordSettings(['delete_delay_days' => $this->keys->deleteDelayDays()]);
|
||||
} catch (\Throwable) {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Change how an existing location is reached (credentials, host, folder, even the connection
|
||||
* type) or its name. The new settings must lead to the same repository: the history stays
|
||||
@@ -208,7 +353,25 @@ final class TargetService {
|
||||
return $result;
|
||||
}
|
||||
|
||||
/** Update the local copy of a snapshot's note or pin, if the snapshot is listed here at all. */
|
||||
public function syncSnapshot(Target $target, string $snapshotId, ?string $note = null, ?bool $pinned = null): void {
|
||||
try {
|
||||
$s = $this->snapshots->findOne($target->getId(), $snapshotId);
|
||||
} catch (DoesNotExistException) {
|
||||
return;
|
||||
}
|
||||
if ($note !== null) {
|
||||
$s->setNote($note !== '' ? $note : null);
|
||||
}
|
||||
if ($pinned !== null) {
|
||||
$s->setPinned($pinned);
|
||||
}
|
||||
$this->snapshots->update($s);
|
||||
}
|
||||
|
||||
private function adoptSnapshots(Target $target, Repository $repo): void {
|
||||
$notes = $repo->snapshotNotes();
|
||||
$pinned = array_flip($repo->pinnedSnapshots());
|
||||
foreach ($repo->listSnapshots() as $snapshotId) {
|
||||
try {
|
||||
$this->snapshots->findOne($target->getId(), $snapshotId);
|
||||
@@ -227,6 +390,8 @@ final class TargetService {
|
||||
$s->setCreatedAt((int)strtotime((string)($meta['time'] ?? 'now')));
|
||||
$s->setFiles((int)($meta['stats']['files'] ?? 0));
|
||||
$s->setBytes((int)($meta['stats']['bytes'] ?? 0));
|
||||
$s->setNote($notes[$snapshotId] ?? null);
|
||||
$s->setPinned(isset($pinned[$snapshotId]));
|
||||
$this->snapshots->insert($s);
|
||||
} catch (\Throwable $e) {
|
||||
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: snapshot ' . $snapshotId . ' on ' . $target->getName() . ' not listed: ' . $e->getMessage(), ['exception' => $e]);
|
||||
@@ -248,6 +413,30 @@ final class TargetService {
|
||||
}
|
||||
|
||||
/** Size limit for the data stored on this location (null = none). */
|
||||
/**
|
||||
* @param string $schedule see LocationSchedule
|
||||
* @throws \InvalidArgumentException
|
||||
*/
|
||||
public function setSchedule(Target $target, string $schedule, string $zone): Target {
|
||||
if (!Schedule::validZone($zone)) {
|
||||
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
||||
}
|
||||
$new = LocationSchedule::normalize($schedule);
|
||||
$turnedOff = $new === '' && (string)$target->getSchedule() !== '';
|
||||
if ($new === (string)$target->getSchedule() && $zone === $target->getScheduleZone()) {
|
||||
// Saved again unchanged: a moment that already passed still gets its backup
|
||||
return $target;
|
||||
}
|
||||
$target->setSchedule($new);
|
||||
$target->setScheduleZone($zone);
|
||||
$target->setScheduleSince(time());
|
||||
$target = $this->mapper->update($target);
|
||||
if ($turnedOff) {
|
||||
$this->alerts->securityEvent('schedule_off', ['target' => $target->getName()]);
|
||||
}
|
||||
return $target;
|
||||
}
|
||||
|
||||
public function setMaxBytes(Target $target, ?int $maxBytes): Target {
|
||||
$target->setMaxBytes($maxBytes);
|
||||
return $this->mapper->update($target);
|
||||
@@ -261,10 +450,29 @@ final class TargetService {
|
||||
/** Remove the target from NG Backup. The backups on the target itself are not touched. */
|
||||
public function remove(Target $target): void {
|
||||
$this->mapper->delete($target);
|
||||
// The location is gone now: tidying up must not fail the removal, or the caller would skip
|
||||
// the removal event and a retry would no longer find the location
|
||||
$this->tidyUp($target, fn () => $this->keys->setWrappedKeyPending($target->getId(), false));
|
||||
// The cached index of its repository is only of use while a location points at it
|
||||
$repositoryId = $target->getRepositoryId();
|
||||
if ($repositoryId !== null && !array_filter($this->list(), fn (Target $t) => $t->getRepositoryId() === $repositoryId)) {
|
||||
$this->indexCache->forget($repositoryId);
|
||||
$this->tidyUp($target, function () use ($target): void {
|
||||
$repositoryId = $target->getRepositoryId();
|
||||
if ($repositoryId !== null && !array_filter($this->list(), fn (Target $t) => $t->getRepositoryId() === $repositoryId)) {
|
||||
$this->indexCache->forget($repositoryId);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** One tidy-up step after a removal: a failure is logged (if even that works) and never thrown. */
|
||||
private function tidyUp(Target $target, callable $step): void {
|
||||
try {
|
||||
$step();
|
||||
} catch (\Throwable $e) {
|
||||
try {
|
||||
Server::get(\Psr\Log\LoggerInterface::class)->warning('NG Backup: location ' . $target->getName()
|
||||
. ' removed, tidying up after it failed: ' . $e->getMessage(), ['exception' => $e]);
|
||||
} catch (\Throwable) {
|
||||
// nothing left to tell it to
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+16
-1
@@ -300,6 +300,22 @@
|
||||
<code><![CDATA[$t]]></code>
|
||||
</UndefinedDocblockClass>
|
||||
</file>
|
||||
<file src="lib/Migration/Version000008Date20261107010000.php">
|
||||
<UndefinedDocblockClass>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
</UndefinedDocblockClass>
|
||||
</file>
|
||||
<file src="lib/Migration/Version000007Date20261107000000.php">
|
||||
<UndefinedDocblockClass>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
<code><![CDATA[$t]]></code>
|
||||
</UndefinedDocblockClass>
|
||||
</file>
|
||||
<file src="lib/Migration/Version000005Date20261002040000.php">
|
||||
<UndefinedDocblockClass>
|
||||
<code><![CDATA[$t]]></code>
|
||||
@@ -351,7 +367,6 @@
|
||||
<code><![CDATA[$encrypted]]></code>
|
||||
<code><![CDATA[$encrypted]]></code>
|
||||
<code><![CDATA[$encrypted]]></code>
|
||||
<code><![CDATA[$fh]]></code>
|
||||
<code><![CDATA[$plain]]></code>
|
||||
<code><![CDATA[$plain]]></code>
|
||||
<code><![CDATA[$plain]]></code>
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 259 KiB After Width: | Height: | Size: 244 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 268 KiB After Width: | Height: | Size: 228 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 252 KiB After Width: | Height: | Size: 222 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 189 KiB After Width: | Height: | Size: 172 KiB |
+4
-1
@@ -34,8 +34,11 @@ export default {
|
||||
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
|
||||
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
|
||||
tick: () => call('post', '/tick'),
|
||||
setSchedule: (time, timezone) => call('put', '/schedule', { time, timezone }),
|
||||
setPauseHeavyJobs: (enabled) => call('put', '/settings/pause-heavy-jobs', { enabled }),
|
||||
setTargetSchedule: (id, schedule, timezone) => call('put', `/targets/${id}/schedule`, { schedule, timezone }),
|
||||
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
|
||||
setSnapshotNote: (targetId, snapshotId, note) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/note`, { note }),
|
||||
setSnapshotPinned: (targetId, snapshotId, pinned) => call('put', `/targets/${targetId}/snapshots/${snapshotId}/pin`, { pinned }),
|
||||
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
|
||||
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
|
||||
}
|
||||
|
||||
+31
-29
@@ -5,10 +5,11 @@
|
||||
<template>
|
||||
<NcSettingsSection :name="t('ng_backup', 'Backups')"
|
||||
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
|
||||
<div class="ngb-row">
|
||||
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, {zone}, empty = off)', { zone: browserZone })" />
|
||||
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||
</div>
|
||||
<LocationScheduleRow v-for="target in status.targets" :key="target.id" :target="target" @changed="emit('changed')" @error="error = $event" />
|
||||
<NcCheckboxRadioSwitch :model-value="status.heavyJobs.enabled" @update:model-value="savePause">
|
||||
{{ t('ng_backup', 'Pause heavy background jobs (Memories, Recognize, Preview Generator, full text search, antivirus scan) while a backup runs, at most 6 hours in a row') }}
|
||||
</NcCheckboxRadioSwitch>
|
||||
<p v-if="status.heavyJobs.jobs.length" class="ngb-muted">{{ t('ng_backup', 'Paused now: {jobs}', { jobs: status.heavyJobs.jobs.join(', ') }) }}</p>
|
||||
<div class="ngb-row">
|
||||
<NcTextField v-model="label" class="ngb-label" :maxlength="100" :label="t('ng_backup', 'Label (optional)')"
|
||||
:placeholder="t('ng_backup', 'e.g. before the upgrade to Nextcloud 35')" />
|
||||
@@ -37,11 +38,11 @@
|
||||
<td>{{ formatDateTime(run.startedAt) }}</td>
|
||||
<td>{{ targetName(run.targetId) }}</td>
|
||||
<td>{{ run.kind }}</td>
|
||||
<td>{{ run.snapshot ? formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') : '' }}</td>
|
||||
<td class="ngb-point">{{ run.snapshot ? (run.snapshot.pinned ? '📌 ' : '') + formatDateTime(run.snapshot.createdAt) + (run.snapshot.label ? ' – ' + run.snapshot.label : '') + (run.snapshot.note ? ' – ' + run.snapshot.note : '') : '' }}</td>
|
||||
<td :class="'ngb-' + run.status">{{ runStatus(run) }}{{ run.error ? ': ' + run.error : '' }}</td>
|
||||
<td>{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
|
||||
<td>{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td>
|
||||
<td>{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}</td>
|
||||
<td class="ngb-num">{{ run.stats?.files?.files ?? run.stats?.restored ?? '' }}</td>
|
||||
<td class="ngb-num">{{ run.stats?.files ? formatSize(run.stats.files.bytes) : '' }}</td>
|
||||
<td class="ngb-num">{{ run.stats?.files ? formatSize(run.stats.files.uploaded) : '' }}</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
@@ -60,24 +61,20 @@
|
||||
</template>
|
||||
|
||||
<script setup>
|
||||
import { ref, watch } from 'vue'
|
||||
import { ref } from 'vue'
|
||||
import { n, t } from '@nextcloud/l10n'
|
||||
import NcButton from '@nextcloud/vue/components/NcButton'
|
||||
import NcCheckboxRadioSwitch from '@nextcloud/vue/components/NcCheckboxRadioSwitch'
|
||||
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||
import LocationScheduleRow from './LocationScheduleRow.vue'
|
||||
import api from '../api.js'
|
||||
import { formatDateTime, formatSize, toZone } from '../format.js'
|
||||
import { formatDateTime, formatSize } from '../format.js'
|
||||
|
||||
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
|
||||
const emit = defineEmits(['changed', 'show-all-runs'])
|
||||
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
|
||||
// zone is shown in this one, so it reads as the moment it runs here.
|
||||
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
||||
const shown = (s) => (s && props.status.scheduleTimezone && props.status.scheduleTimezone !== browserZone) ? toZone(s, props.status.scheduleTimezone, browserZone) : s
|
||||
const schedule = ref(shown(props.status.schedule))
|
||||
const error = ref('')
|
||||
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
|
||||
|
||||
const label = ref('')
|
||||
|
||||
@@ -97,6 +94,9 @@ const runStatus = (run) => {
|
||||
if (run.status !== 'running') {
|
||||
return run.status
|
||||
}
|
||||
if (run.queued) {
|
||||
return t('ng_backup', 'queued, after the backup that is running now')
|
||||
}
|
||||
const detail = run.phase + percent(run)
|
||||
if (run.waitingFor) {
|
||||
return t('ng_backup', 'waiting for background jobs ({detail}, last step {minutes} min ago)', { detail, minutes: Math.floor(run.waitingFor / 60) })
|
||||
@@ -106,6 +106,16 @@ const runStatus = (run) => {
|
||||
const isRunning = (targetId) => props.status.runs.some(r => r.targetId === targetId && r.kind === 'full' && r.status === 'running')
|
||||
const targetName = (id) => props.status.targets.find(tg => tg.id === id)?.name ?? '#' + id
|
||||
|
||||
async function savePause(enabled) {
|
||||
error.value = ''
|
||||
try {
|
||||
await api.setPauseHeavyJobs(enabled)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
error.value = e.message
|
||||
}
|
||||
}
|
||||
|
||||
async function start(target) {
|
||||
error.value = ''
|
||||
try {
|
||||
@@ -116,25 +126,17 @@ async function start(target) {
|
||||
error.value = e.message
|
||||
}
|
||||
}
|
||||
|
||||
async function saveSchedule() {
|
||||
error.value = ''
|
||||
try {
|
||||
await api.setSchedule(schedule.value.trim(), browserZone)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
error.value = e.message
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
||||
.ngb-time { max-width: 260px; }
|
||||
.ngb-label { max-width: 360px; }
|
||||
.ngb-table-wrap { overflow-x: auto; }
|
||||
.ngb-table-wrap { max-width: 90%; overflow-x: auto; } /* scrolls only on a really narrow screen */
|
||||
.ngb-table { width: 100%; border-collapse: collapse; }
|
||||
.ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); }
|
||||
.ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); white-space: normal; overflow-wrap: break-word; }
|
||||
/* Text wraps; numbers stay on one line, the restore point keeps room for its label and note */
|
||||
.ngb-table td.ngb-num { white-space: nowrap; }
|
||||
.ngb-table td.ngb-point { min-width: 9em; }
|
||||
.ngb-failed { color: var(--color-error-text); }
|
||||
.ngb-muted { color: var(--color-text-maxcontrast); }
|
||||
.ngb-done { color: var(--color-success-text); }
|
||||
|
||||
@@ -4,8 +4,11 @@
|
||||
-->
|
||||
<template>
|
||||
<NcSettingsSection :name="t('ng_backup', 'Backup key and recovery kit')"
|
||||
:description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups. To restore on a new server you need the recovery kit and one of the passphrases.')">
|
||||
:description="t('ng_backup', 'All backups are encrypted with one key. The server keeps it for scheduled backups. To restore on a new server you need one of the passphrases; the recovery kit also helps when the key on the backup location is damaged.')">
|
||||
<!-- 1. Create the key -->
|
||||
<NcNoteCard v-if="!status.key.initialized" type="warning">
|
||||
{{ t('ng_backup', 'Restoring a server after a crash? Do not create a key here: a new key cannot read your backups. Follow the disaster recovery guide instead (occ backup:target:add asks for one of your passphrases, or import the recovery kit with occ backup:key:import).') }}
|
||||
</NcNoteCard>
|
||||
<form v-if="!status.key.initialized" class="ngb-form" @submit.prevent="createKey">
|
||||
<NcTextField v-model="label" :label="t('ng_backup', 'Who keeps this passphrase (e.g. Safe, CTO)')" />
|
||||
<NcPasswordField v-model="pass1" :label="t('ng_backup', 'Passphrase (at least 16 characters)')" autocomplete="new-password" />
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
<!--
|
||||
- SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
- SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
-->
|
||||
<template>
|
||||
<div class="ngb-row">
|
||||
<strong class="ngb-name">{{ target.name }}</strong>
|
||||
<NcSelect v-model="kind" :options="kinds" label="label" :input-label="t('ng_backup', 'Schedule')" :clearable="false" />
|
||||
<NcTextField v-if="kind.id !== 'off'" v-model="time" class="ngb-time" :label="kind.id === 'daily' ? t('ng_backup', 'At (HH:MM, {zone})', { zone }) : t('ng_backup', 'From (HH:MM, {zone})', { zone })" />
|
||||
<NcTextField v-if="kind.id === 'hours'" v-model="hours" class="ngb-hours" type="number" min="1" max="23" :label="t('ng_backup', 'Every … hours')" />
|
||||
<NcButton :disabled="busy || wanted === current" @click="save">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script setup>
|
||||
import { computed, ref, watch } from 'vue'
|
||||
import { t } from '@nextcloud/l10n'
|
||||
import NcButton from '@nextcloud/vue/components/NcButton'
|
||||
import NcSelect from '@nextcloud/vue/components/NcSelect'
|
||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||
import api from '../api.js'
|
||||
|
||||
const props = defineProps({ target: { type: Object, required: true } })
|
||||
const emit = defineEmits(['changed', 'error'])
|
||||
|
||||
const kinds = [
|
||||
{ id: 'off', label: t('ng_backup', 'Off') },
|
||||
{ id: 'daily', label: t('ng_backup', 'Once a day') },
|
||||
{ id: 'hours', label: t('ng_backup', 'Every few hours') },
|
||||
]
|
||||
// A schedule is shown and edited in the zone it was saved in, named next to the time: "every 6
|
||||
// hours from 02:00" runs until midnight of that zone, so moving it to another zone would change
|
||||
// how often it runs. A location without a schedule gets the administrator's own zone (the browser's).
|
||||
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
||||
const zone = computed(() => props.target.schedule !== '' && props.target.scheduleZone ? props.target.scheduleZone : browserZone)
|
||||
const kind = ref(kinds[0])
|
||||
const time = ref('02:00')
|
||||
const hours = ref('6')
|
||||
const busy = ref(false)
|
||||
|
||||
const current = computed(() => {
|
||||
const [at, every] = props.target.schedule.split('/')
|
||||
return at === '' ? '' : at + (every ? '/' + every : '')
|
||||
})
|
||||
const wanted = computed(() => {
|
||||
if (kind.value.id === 'off') {
|
||||
return ''
|
||||
}
|
||||
return time.value.trim() + (kind.value.id === 'hours' ? '/' + String(hours.value).trim() : '')
|
||||
})
|
||||
|
||||
watch(current, (schedule) => {
|
||||
const [at, every] = schedule.split('/')
|
||||
kind.value = kinds[at === '' ? 0 : (every ? 2 : 1)]
|
||||
time.value = at === '' ? time.value : at
|
||||
hours.value = every ?? hours.value
|
||||
}, { immediate: true })
|
||||
|
||||
async function save() {
|
||||
busy.value = true
|
||||
emit('error', '')
|
||||
try {
|
||||
await api.setTargetSchedule(props.target.id, wanted.value, zone.value)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
emit('error', e.message)
|
||||
} finally {
|
||||
busy.value = false
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
||||
.ngb-name { min-width: 120px; align-self: center; }
|
||||
.ngb-time { max-width: 220px; }
|
||||
.ngb-hours { max-width: 120px; }
|
||||
</style>
|
||||
@@ -11,6 +11,13 @@
|
||||
</div>
|
||||
|
||||
<template v-if="snapshot">
|
||||
<div class="ngb-row ngb-marks">
|
||||
<NcTextField v-model="note" class="ngb-note" :maxlength="500" :label="t('ng_backup', 'Note (can be changed later)')" />
|
||||
<NcButton :disabled="marking || note.trim() === (snapshot.note ?? '')" @click="saveNote">{{ t('ng_backup', 'Save note') }}</NcButton>
|
||||
<NcCheckboxRadioSwitch :model-value="snapshot.pinned" :disabled="marking" @update:model-value="savePinned">
|
||||
{{ t('ng_backup', 'Pin: the retention policy does not remove this restore point') }}
|
||||
</NcCheckboxRadioSwitch>
|
||||
</div>
|
||||
<div class="ngb-crumbs">
|
||||
<NcButton variant="tertiary" @click="open('data')">data</NcButton>
|
||||
<template v-for="(crumb, i) in crumbs" :key="crumb.path">
|
||||
@@ -49,6 +56,7 @@ import NcLoadingIcon from '@nextcloud/vue/components/NcLoadingIcon'
|
||||
import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||
import NcSelect from '@nextcloud/vue/components/NcSelect'
|
||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||
import { confirmPassword } from '@nextcloud/password-confirmation'
|
||||
import '@nextcloud/password-confirmation/style.css'
|
||||
import api from '../api.js'
|
||||
@@ -74,7 +82,10 @@ const modes = [
|
||||
{ id: 'replace', label: t('ng_backup', 'Into the original place, overwrite and move files that are not in the backup to the trash') },
|
||||
]
|
||||
|
||||
const snapshotLabel = (s) => formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '')
|
||||
const note = ref('')
|
||||
const marking = ref(false)
|
||||
|
||||
const snapshotLabel = (s) => (s.pinned ? '📌 ' : '') + formatDateTime(s.createdAt) + (s.label ? ' – ' + s.label : '') + (s.note ? ' – ' + s.note : '')
|
||||
const join = (name) => path.value + '/' + name
|
||||
const restorable = (p) => /^data\/[^/]+\/files(\/.*)?$/.test(p)
|
||||
const crumbs = computed(() => {
|
||||
@@ -88,7 +99,29 @@ watch(target, async (tg) => {
|
||||
snapshots.value = list.map(s => ({ ...s, displayLabel: snapshotLabel(s) }))
|
||||
snapshot.value = snapshots.value[0] ?? null
|
||||
}, { immediate: true })
|
||||
watch(snapshot, () => { selected.value = ''; open('data') })
|
||||
watch(snapshot, (s) => { selected.value = ''; note.value = s?.note ?? ''; open('data') })
|
||||
|
||||
// Changes the snapshot in place: choosing another one, which opens its folders again, is not needed
|
||||
async function mark(call, change) {
|
||||
const s = snapshot.value
|
||||
marking.value = true
|
||||
error.value = message.value = ''
|
||||
try {
|
||||
await call(target.value.id, s.id)
|
||||
change(s)
|
||||
s.displayLabel = snapshotLabel(s)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
error.value = e.message
|
||||
} finally {
|
||||
marking.value = false
|
||||
}
|
||||
}
|
||||
const saveNote = () => {
|
||||
const text = note.value.trim()
|
||||
return mark((targetId, id) => api.setSnapshotNote(targetId, id, text), (s) => { s.note = text })
|
||||
}
|
||||
const savePinned = (pinned) => mark((targetId, id) => api.setSnapshotPinned(targetId, id, pinned), (s) => { s.pinned = pinned })
|
||||
|
||||
async function open(p) {
|
||||
if (!snapshot.value) {
|
||||
@@ -125,6 +158,8 @@ async function restore() {
|
||||
<style scoped>
|
||||
.ngb-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 12px; }
|
||||
.ngb-row :deep(.ngb-snapshot) { min-width: min(360px, 100%); }
|
||||
.ngb-marks { align-items: flex-end; }
|
||||
.ngb-note { max-width: 360px; }
|
||||
.ngb-crumbs { display: flex; align-items: center; flex-wrap: wrap; }
|
||||
.ngb-entries li { display: flex; align-items: center; gap: 12px; min-height: 36px; }
|
||||
.ngb-entries li.selected { background: var(--color-primary-element-light); border-radius: var(--border-radius); }
|
||||
|
||||
@@ -26,30 +26,3 @@ export function formatSize(bytes) {
|
||||
}
|
||||
return i === 0 ? `${b} B` : `${b.toFixed(1)} ${units[i]}`
|
||||
}
|
||||
|
||||
/**
|
||||
* A time of day "HH:MM" in zone `from`, as "HH:MM" in zone `to`, at today's offsets.
|
||||
*
|
||||
* @param {string} hhmm the time
|
||||
* @param {string} from IANA zone it is in
|
||||
* @param {string} to IANA zone to show it in
|
||||
* @param {Date} now the day to convert on (today)
|
||||
* @return {string}
|
||||
*/
|
||||
export function toZone(hhmm, from, to, now = new Date()) {
|
||||
const [h, m] = hhmm.split(':').map(Number)
|
||||
const parts = (zone, date) => Object.fromEntries(new Intl.DateTimeFormat('en-GB', {
|
||||
timeZone: zone, hourCycle: 'h23', year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit',
|
||||
}).formatToParts(date).map((p) => [p.type, p.value]))
|
||||
const offsetAt = (at) => {
|
||||
const seen = parts(from, new Date(at))
|
||||
return Date.UTC(+seen.year, +seen.month - 1, +seen.day, +seen.hour, +seen.minute) - at
|
||||
}
|
||||
const today = parts(from, now)
|
||||
const wall = Date.UTC(+today.year, +today.month - 1, +today.day, h, m)
|
||||
// the offset at the wall time itself: on a day the clocks change it differs from the one at
|
||||
// midnight, so it is looked up a second time at the moment found first
|
||||
const at = wall - offsetAt(wall - offsetAt(wall))
|
||||
const there = parts(to, new Date(at))
|
||||
return `${there.hour}:${there.minute}`
|
||||
}
|
||||
|
||||
@@ -22,6 +22,27 @@ class KeyRingTest extends TestCase {
|
||||
KeyRing::unwrap($wrapped, 'wrong');
|
||||
}
|
||||
|
||||
public function testInvalidBase64IsADamagedKey(): void {
|
||||
$wrapped = KeyRing::generate()->wrap('right');
|
||||
$wrapped['salt'] = '***not base64***';
|
||||
$this->expectException(CryptoException::class);
|
||||
$this->expectExceptionMessage('Damaged key file');
|
||||
KeyRing::unwrapAny($wrapped, 'right');
|
||||
}
|
||||
|
||||
public function testAShortSaltOrNonceIsADamagedKey(): void {
|
||||
foreach (['salt', 'nonce'] as $part) {
|
||||
$wrapped = KeyRing::generate()->wrap('right');
|
||||
$wrapped[$part] = base64_encode('short');
|
||||
try {
|
||||
KeyRing::unwrapAny($wrapped, 'right');
|
||||
$this->fail("a short $part was accepted");
|
||||
} catch (CryptoException $e) {
|
||||
$this->assertStringContainsString('Damaged key file', $e->getMessage(), $part);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public function testBlobIdsAreKeyed(): void {
|
||||
$this->assertNotSame(KeyRing::generate()->blobId('same'), KeyRing::generate()->blobId('same'));
|
||||
$k = KeyRing::generate();
|
||||
@@ -34,6 +55,20 @@ class KeyRingTest extends TestCase {
|
||||
new KeyRing('short');
|
||||
}
|
||||
|
||||
public function testADamagedSlotDoesNotStopTheNextOne(): void {
|
||||
$keys = KeyRing::generate();
|
||||
$broken = $keys->wrap('passphrase in the safe');
|
||||
$broken['salt'] = '*** not base64 ***';
|
||||
$field = ['slots' => [
|
||||
['slot' => 1, 'label' => 'Safe', 'key' => $broken],
|
||||
['slot' => 2, 'label' => 'Missing parts', 'key' => ['v' => 1]],
|
||||
['slot' => 3, 'label' => 'CTO', 'key' => $keys->wrap('passphrase of the cto')],
|
||||
]];
|
||||
$this->assertSame($keys->blobId('x'), KeyRing::unwrapAny($field, 'passphrase of the cto')->blobId('x'));
|
||||
$this->expectException(CryptoException::class);
|
||||
KeyRing::unwrapAny($field, 'passphrase in the safe');
|
||||
}
|
||||
|
||||
public function testAnySlotOpensTheSameKey(): void {
|
||||
$keys = KeyRing::generate();
|
||||
$field = ['slots' => [
|
||||
|
||||
@@ -82,7 +82,11 @@ class EmptyDirsTest extends TestCase {
|
||||
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
|
||||
}
|
||||
$this->assertSame('commit', $state['phase']);
|
||||
$this->assertSame(4, $state['stats']['unreadable'], 'each reported once, over several steps');
|
||||
$this->assertSame(4, $state['stats']['badNames'], 'each reported once, over several steps');
|
||||
$this->assertArrayNotHasKey('unreadable', $state['stats'], 'not a problem with permissions');
|
||||
$shown = BackupRun::warnings($state)['badNames'];
|
||||
$this->assertContains('data/alice/files/line?break', $shown, 'no control character in a message');
|
||||
$this->assertSame($shown, array_filter($shown, fn (string $name): bool => mb_check_encoding($name, 'UTF-8')));
|
||||
$this->repo()->recordSnapshot($state['snapshot']);
|
||||
|
||||
$this->assertSame(['data/alice/files/Hollow/Nested', 'data/skipped/inner'], array_column($this->repo()->emptyDirs($state['snapshot']), 0));
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||
|
||||
use OCA\NgBackup\Backend\LocalBackend;
|
||||
use OCA\NgBackup\Crypto\KeyRing;
|
||||
use OCA\NgBackup\Job\BackupRun;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** Folders with something in them get their original modification time back on a restore. */
|
||||
class FolderTimesTest extends TestCase {
|
||||
use TempDirTrait;
|
||||
|
||||
private const OLD = 1500000000;
|
||||
private const OLDER = 1400000000;
|
||||
|
||||
private LocalBackend $backend;
|
||||
private KeyRing $keys;
|
||||
private string $src;
|
||||
|
||||
protected function setUp(): void {
|
||||
$this->backend = new LocalBackend($this->tempDir());
|
||||
$this->keys = KeyRing::generate();
|
||||
Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'));
|
||||
$this->src = $this->tempDir();
|
||||
mkdir("$this->src/data/alice/files/Old/Deep", 0700, true);
|
||||
mkdir("$this->src/data/alice/files/2026", 0700, true);
|
||||
file_put_contents("$this->src/data/alice/files/Old/Deep/a.txt", 'a');
|
||||
file_put_contents("$this->src/data/alice/files/Old/b.txt", 'b');
|
||||
file_put_contents("$this->src/data/alice/files/2026/c.txt", 'c');
|
||||
touch("$this->src/data/alice/files/Old/Deep", self::OLDER);
|
||||
touch("$this->src/data/alice/files/Old", self::OLD);
|
||||
touch("$this->src/data/alice/files/2026", self::OLD);
|
||||
}
|
||||
|
||||
private function repo(): Repository {
|
||||
return Repository::openWithKey($this->backend, $this->keys);
|
||||
}
|
||||
|
||||
private function backup(float $seconds = 30.0): string {
|
||||
$state = BackupRun::start(['data' => "$this->src/data"]);
|
||||
for ($i = 0; $i < 500 && $state['phase'] !== 'commit'; $i++) {
|
||||
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, microtime(true) + $seconds)), true);
|
||||
}
|
||||
$this->assertSame('commit', $state['phase']);
|
||||
$this->repo()->recordSnapshot($state['snapshot']);
|
||||
return $state['snapshot'];
|
||||
}
|
||||
|
||||
private function time(string $path): int {
|
||||
clearstatcache();
|
||||
return (int)filemtime($path);
|
||||
}
|
||||
|
||||
public function testEachFolderIsRecordedOnceAlsoInSmallSteps(): void {
|
||||
$expected = ['data', 'data/alice', 'data/alice/files', 'data/alice/files/2026', 'data/alice/files/Old', 'data/alice/files/Old/Deep'];
|
||||
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup()), 0));
|
||||
$this->assertSame($expected, array_column($this->repo()->folderTimes($this->backup(0.0)), 0), 'a step per entry');
|
||||
}
|
||||
|
||||
public function testNoEmptyFolderIsAmongThem(): void {
|
||||
mkdir("$this->src/data/alice/files/Hollow");
|
||||
$snapshot = $this->backup();
|
||||
$this->assertNotContains('data/alice/files/Hollow', array_column($this->repo()->folderTimes($snapshot), 0));
|
||||
$this->assertContains('data/alice/files/Hollow', array_column($this->repo()->emptyDirs($snapshot), 0));
|
||||
}
|
||||
|
||||
public function testARestoreBringsThemBack(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$this->repo()->restore($snapshot, $target);
|
||||
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
|
||||
$this->assertSame(self::OLDER, $this->time("$target/data/alice/files/Old/Deep"));
|
||||
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/2026"));
|
||||
}
|
||||
|
||||
public function testARestoreThatGoesOnAfterAStopToo(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$this->repo()->restore($snapshot, $target);
|
||||
touch("$target/data/alice/files/Old", 1800000000);
|
||||
$this->repo()->restore($snapshot, $target, keepUnchanged: true);
|
||||
$this->assertSame(self::OLD, $this->time("$target/data/alice/files/Old"));
|
||||
}
|
||||
|
||||
public function testAPrefixLeavesTheTargetAndOthersAlone(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$this->repo()->restore($snapshot, $target, 'data/alice/files/Old/', true);
|
||||
$this->assertSame(self::OLDER, $this->time("$target/Deep"));
|
||||
$this->assertNotSame(self::OLD, $this->time($target), 'the directory that was chosen is not ours to change');
|
||||
$this->assertDirectoryDoesNotExist("$target/2026");
|
||||
}
|
||||
|
||||
public function testALinkOutOfTheTargetIsRefused(): void {
|
||||
$snapshot = $this->backup();
|
||||
$target = $this->tempDir();
|
||||
$outside = $this->tempDir();
|
||||
touch($outside, 1800000000);
|
||||
mkdir("$target/data/alice/files", 0700, true);
|
||||
symlink($outside, "$target/data/alice/files/Old");
|
||||
$this->expectException(\OCA\NgBackup\Repository\RestoreTargetException::class);
|
||||
$this->repo()->restore($snapshot, $target);
|
||||
}
|
||||
|
||||
public function testAFolderWithABadNameIsLeftOutAndWhatIsInItReported(): void {
|
||||
mkdir("$this->src/data/odd\\folder");
|
||||
file_put_contents("$this->src/data/odd\\folder/x", 'x');
|
||||
$state = BackupRun::start(['data' => "$this->src/data"]);
|
||||
for ($i = 0; $i < 50 && $state['phase'] !== 'commit'; $i++) {
|
||||
$state = json_decode(json_encode(BackupRun::step($this->repo(), $state, 0.0), JSON_THROW_ON_ERROR), true, 512, JSON_THROW_ON_ERROR);
|
||||
}
|
||||
$this->assertSame('commit', $state['phase']);
|
||||
$this->assertSame(1, $state['stats']['badNames']);
|
||||
$this->repo()->recordSnapshot($state['snapshot']);
|
||||
$this->assertNotContains("data/odd\\folder", array_column($this->repo()->folderTimes($state['snapshot']), 0));
|
||||
}
|
||||
|
||||
public function testASnapshotWithoutThemHasNone(): void {
|
||||
$snapshot = $this->repo()->backupDirectory($this->src)['snapshot'];
|
||||
$this->assertSame([], $this->repo()->folderTimes($snapshot));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||
|
||||
use OCA\NgBackup\Backend\LocalBackend;
|
||||
use OCA\NgBackup\Crypto\CryptoException;
|
||||
use OCA\NgBackup\Crypto\KeyRing;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** Disaster recovery without the recovery kit: the location gives its key with a passphrase. */
|
||||
class KeyFromLocationTest extends TestCase {
|
||||
use TempDirTrait;
|
||||
|
||||
private string $dir;
|
||||
private LocalBackend $backend;
|
||||
private KeyRing $keys;
|
||||
|
||||
protected function setUp(): void {
|
||||
$this->dir = $this->tempDir();
|
||||
$this->backend = new LocalBackend($this->dir);
|
||||
$this->keys = KeyRing::generate();
|
||||
$slots = ['slots' => [
|
||||
['slot' => 1, 'label' => 'A', 'created' => 0, 'key' => $this->keys->wrap('first passphrase')],
|
||||
['slot' => 2, 'label' => 'B', 'created' => 0, 'key' => $this->keys->wrap('second passphrase')],
|
||||
]];
|
||||
Repository::initWithKey($this->backend, $this->keys, $slots)->recordSettings(['delete_delay_days' => 30]);
|
||||
}
|
||||
|
||||
public function testEachPassphraseGivesTheKeyThatOpensTheBackups(): void {
|
||||
foreach (['first passphrase', 'second passphrase'] as $passphrase) {
|
||||
$keys = KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), $passphrase);
|
||||
$this->assertTrue(Repository::madeWithKey($this->backend, $keys));
|
||||
$this->assertSame($this->keys->exportMasterKey(), $keys->exportMasterKey());
|
||||
}
|
||||
}
|
||||
|
||||
public function testAWrongPassphraseGivesNothing(): void {
|
||||
$this->expectException(CryptoException::class);
|
||||
KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), 'not the passphrase');
|
||||
}
|
||||
|
||||
public function testAnotherKeyIsNotTakenForTheRightOne(): void {
|
||||
$this->assertFalse(Repository::madeWithKey($this->backend, KeyRing::generate()));
|
||||
}
|
||||
|
||||
public function testTheSettingsAreSealedAndWrittenOnce(): void {
|
||||
$this->assertSame(30, Repository::settingsAt($this->backend, $this->keys)['delete_delay_days']);
|
||||
Repository::openWithKey($this->backend, $this->keys)->recordSettings(['delete_delay_days' => 1]);
|
||||
$this->assertSame(30, Repository::settingsAt($this->backend, $this->keys)['delete_delay_days']);
|
||||
$this->assertSame(['unreadable' => true], Repository::settingsAt($this->backend, KeyRing::generate()), 'another key: there, but not readable');
|
||||
unlink($this->dir . '/settings');
|
||||
$this->assertSame([], Repository::settingsAt($this->backend, $this->keys), 'none at all: a location from before 1.1.0');
|
||||
}
|
||||
|
||||
public function testAlteredSettingsAreNotBelieved(): void {
|
||||
$file = $this->dir . '/settings';
|
||||
$raw = file_get_contents($file);
|
||||
$raw[strlen($raw) - 1] = $raw[strlen($raw) - 1] ^ "\x01";
|
||||
file_put_contents($file, $raw);
|
||||
$this->assertSame(['unreadable' => true], Repository::settingsAt($this->backend, $this->keys));
|
||||
}
|
||||
|
||||
public function testTheKitWritesALostConfigAgainAndTheBackupsOpen(): void {
|
||||
$src = $this->tempDir();
|
||||
file_put_contents("$src/a", 'still here');
|
||||
$repo = Repository::openWithKey($this->backend, $this->keys);
|
||||
$id = $repo->id();
|
||||
$snapshot = $repo->backupDirectory($src)['snapshot'];
|
||||
unlink($this->dir . '/config');
|
||||
$this->assertTrue(Repository::holdsData($this->backend));
|
||||
|
||||
Repository::repairConfig($this->backend, $this->keys, $this->keys->wrap('kit passphrase'));
|
||||
$repo = Repository::openWithKey($this->backend, $this->keys);
|
||||
$this->assertSame($id, $repo->id());
|
||||
$this->assertSame([$snapshot], $repo->listSnapshots());
|
||||
$out = $this->tempDir();
|
||||
$repo->restore($snapshot, $out);
|
||||
$this->assertSame('still here', file_get_contents("$out/a"));
|
||||
$this->assertSame($this->keys->exportMasterKey(), KeyRing::unwrapAny(Repository::wrappedKeyAt($this->backend), 'kit passphrase')->exportMasterKey());
|
||||
}
|
||||
|
||||
public function testAnotherKeyWritesNoConfig(): void {
|
||||
unlink($this->dir . '/config');
|
||||
$other = KeyRing::generate();
|
||||
try {
|
||||
Repository::repairConfig($this->backend, $other, $other->wrap('x'));
|
||||
$this->fail('a config was written with another key');
|
||||
} catch (\OCA\NgBackup\Repository\RepositoryException $e) {
|
||||
$this->assertStringContainsString('another server', $e->getMessage());
|
||||
}
|
||||
$this->assertFileDoesNotExist($this->dir . '/config');
|
||||
}
|
||||
|
||||
public function testAConfigThatIsThereIsNeverRepaired(): void {
|
||||
$before = file_get_contents($this->dir . '/config');
|
||||
try {
|
||||
Repository::repairConfig($this->backend, $this->keys, $this->keys->wrap('x'));
|
||||
$this->fail('an existing config was replaced');
|
||||
} catch (\OCA\NgBackup\Repository\RepositoryException) {
|
||||
}
|
||||
$this->assertSame($before, file_get_contents($this->dir . '/config'));
|
||||
}
|
||||
|
||||
public function testALocationWithoutSettingsHasNone(): void {
|
||||
$other = new LocalBackend($this->tempDir());
|
||||
Repository::initWithKey($other, $this->keys, $this->keys->wrap('x'));
|
||||
$this->assertSame([], Repository::settingsAt($other, $this->keys));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||
|
||||
use OCA\NgBackup\Backend\LocalBackend;
|
||||
use OCA\NgBackup\Crypto\KeyRing;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** A pin and a note on a snapshot live in the catalog on the location, so a new server sees them too. */
|
||||
class SnapshotMarksTest extends TestCase {
|
||||
use TempDirTrait;
|
||||
|
||||
private LocalBackend $backend;
|
||||
private KeyRing $keys;
|
||||
private string $snapshot;
|
||||
|
||||
protected function setUp(): void {
|
||||
$this->backend = new LocalBackend($this->tempDir());
|
||||
$this->keys = KeyRing::generate();
|
||||
$src = $this->tempDir();
|
||||
file_put_contents("$src/a", 'x');
|
||||
$this->snapshot = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'))->backupDirectory($src)['snapshot'];
|
||||
}
|
||||
|
||||
private function repo(): Repository {
|
||||
return Repository::openWithKey($this->backend, $this->keys);
|
||||
}
|
||||
|
||||
public function testTheyAreThereAfterOpeningAgain(): void {
|
||||
$this->repo()->setPinned($this->snapshot, true);
|
||||
$this->repo()->setSnapshotNote($this->snapshot, 'before the upgrade');
|
||||
$this->assertSame([$this->snapshot], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([$this->snapshot => 'before the upgrade'], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testTheyCanBeTakenOffAgain(): void {
|
||||
$this->repo()->setPinned($this->snapshot, true);
|
||||
$this->repo()->setSnapshotNote($this->snapshot, 'x');
|
||||
$this->repo()->setPinned($this->snapshot, false);
|
||||
$this->repo()->setSnapshotNote($this->snapshot, '');
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testNoChangeWritesNoCatalog(): void {
|
||||
$repo = $this->repo();
|
||||
$repo->setPinned($this->snapshot, true);
|
||||
$generation = $repo->catalogGeneration();
|
||||
$repo->setPinned($this->snapshot, true);
|
||||
$repo->setSnapshotNote($this->snapshot, '');
|
||||
$this->assertSame($generation, $repo->catalogGeneration());
|
||||
}
|
||||
|
||||
public function testASnapshotThatIsNotThereIsRefused(): void {
|
||||
$this->expectException(\InvalidArgumentException::class);
|
||||
$this->repo()->setPinned('0123456789abcdef0123456789abcdef', true);
|
||||
}
|
||||
|
||||
public function testATrashedSnapshotLosesItsPinAndKeepsItsNote(): void {
|
||||
$repo = $this->repo();
|
||||
$repo->setPinned($this->snapshot, true);
|
||||
$repo->setSnapshotNote($this->snapshot, 'kept');
|
||||
$repo->forget($this->snapshot);
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
|
||||
|
||||
$repo = $this->repo();
|
||||
$repo->untrash($this->snapshot);
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots(), 'back in the list, as a normal one');
|
||||
$this->assertSame([$this->snapshot => 'kept'], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testThePurgeTakesTheNoteAlong(): void {
|
||||
$repo = $this->repo();
|
||||
$repo->setSnapshotNote($this->snapshot, 'kept');
|
||||
$repo->forget($this->snapshot);
|
||||
$repo->purgeTrash(0);
|
||||
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||
}
|
||||
|
||||
public function testACatalogFromBeforeThisVersionReadsAsNone(): void {
|
||||
$this->assertSame([], $this->repo()->pinnedSnapshots());
|
||||
$this->assertSame([], $this->repo()->snapshotNotes());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||
|
||||
use OCA\NgBackup\Service\HeavyJobPause as P;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
class HeavyJobPauseTest extends TestCase {
|
||||
private const NOW = 1000000;
|
||||
|
||||
public function testDuringABackupTheyArePaused(): void {
|
||||
$this->assertSame(P::PAUSE, P::decide(true, true, 0, 0, self::NOW));
|
||||
$this->assertSame(P::PAUSE, P::decide(true, true, self::NOW - 60, 0, self::NOW), 'jobs added since then too');
|
||||
}
|
||||
|
||||
public function testWithoutABackupNothingHappensOrThePauseEnds(): void {
|
||||
$this->assertSame(P::NOTHING, P::decide(true, false, 0, 0, self::NOW));
|
||||
$this->assertSame(P::RELEASE, P::decide(true, false, self::NOW - 60, 0, self::NOW));
|
||||
}
|
||||
|
||||
public function testTurningItOffEndsThePause(): void {
|
||||
$this->assertSame(P::NOTHING, P::decide(false, true, 0, 0, self::NOW));
|
||||
$this->assertSame(P::RELEASE, P::decide(false, true, self::NOW - 60, 0, self::NOW));
|
||||
}
|
||||
|
||||
public function testAfterSixHoursTheyGetTheirTurnAndThenPauseAgain(): void {
|
||||
$this->assertSame(P::PAUSE, P::decide(true, true, self::NOW - P::MAX_SECONDS + 1, 0, self::NOW));
|
||||
$this->assertSame(P::REST, P::decide(true, true, self::NOW - P::MAX_SECONDS, 0, self::NOW));
|
||||
$rest = self::NOW + P::REST_SECONDS;
|
||||
$this->assertSame(P::NOTHING, P::decide(true, true, 0, $rest, $rest - 1));
|
||||
$this->assertSame(P::PAUSE, P::decide(true, true, 0, $rest, $rest));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||
|
||||
use OCA\NgBackup\Service\LocationSchedule as S;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
class LocationScheduleTest extends TestCase {
|
||||
private const BERLIN = 'Europe/Berlin';
|
||||
|
||||
private static function at(string $time): int {
|
||||
return (int)strtotime($time);
|
||||
}
|
||||
|
||||
public function testWhatCanBeSaved(): void {
|
||||
$this->assertSame('', S::normalize('off'));
|
||||
$this->assertSame('', S::normalize(' '));
|
||||
$this->assertSame('02:30', S::normalize('02:30'));
|
||||
$this->assertSame('02:30', S::normalize('02:30/24'));
|
||||
$this->assertSame('00:00/6', S::normalize('00:00/6'));
|
||||
foreach (['2:30', '24:00', '02:30/0', '02:30/25', 'daily', '02:30/x'] as $bad) {
|
||||
try {
|
||||
S::normalize($bad);
|
||||
$this->fail("$bad accepted");
|
||||
} catch (\InvalidArgumentException) {
|
||||
$this->addToAssertionCount(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public function testOffHasNoMoment(): void {
|
||||
$this->assertNull(S::lastMoment('', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
|
||||
}
|
||||
|
||||
public function testDailyIsInTheZoneOfTheLocation(): void {
|
||||
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 12:00 UTC')));
|
||||
$this->assertSame(self::at('2026-11-09 02:00 UTC'), S::lastMoment('03:00', self::BERLIN, self::at('2026-11-10 01:00 UTC')), 'not yet today');
|
||||
}
|
||||
|
||||
public function testEveryNHoursStartsOverTheNextDay(): void {
|
||||
$zone = 'UTC';
|
||||
$this->assertSame(self::at('2026-11-10 12:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 13:00 UTC')));
|
||||
$this->assertSame(self::at('2026-11-09 22:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 01:59 UTC')), '02, 07, 12, 17, 22, then 02 again');
|
||||
$this->assertSame(self::at('2026-11-10 02:00 UTC'), S::lastMoment('02:00/5', $zone, self::at('2026-11-10 02:00 UTC')), 'at the moment itself');
|
||||
}
|
||||
|
||||
public function testAClockChangeInTheNightStillGivesOneMoment(): void {
|
||||
$this->assertSame(self::at('2026-03-29 01:30 UTC'), S::lastMoment('02:30', self::BERLIN, self::at('2026-03-29 12:00 UTC')), '02:30 does not exist that night');
|
||||
$autumn = S::lastMoment('02:30', self::BERLIN, self::at('2026-10-25 12:00 UTC'));
|
||||
$this->assertNotNull($autumn);
|
||||
$local = (new \DateTimeImmutable('@' . $autumn))->setTimezone(new \DateTimeZone(self::BERLIN));
|
||||
$this->assertSame('2026-10-25 02:30', $local->format('Y-m-d H:i'), '02:30 happens twice that night: one of them, on that day');
|
||||
}
|
||||
|
||||
public function testChoosingATimeDoesNotStartABackup(): void {
|
||||
$moment = self::at('2026-11-10 02:00 UTC');
|
||||
$this->assertFalse(S::isDue($moment, $moment + 1, null, 0, null, $moment + 3600));
|
||||
$this->assertTrue(S::isDue($moment, $moment - 1, null, 0, null, $moment + 3600));
|
||||
$this->assertTrue(S::isDue($moment, 0, null, 0, null, $moment), 'a location from before schedules per location');
|
||||
}
|
||||
|
||||
public function testABackupSinceTheMomentIsEnough(): void {
|
||||
$moment = 1000000;
|
||||
$this->assertFalse(S::isDue($moment, 0, $moment, 1, $moment + 10, $moment + 20));
|
||||
$this->assertTrue(S::isDue($moment, 0, $moment - 1, 0, null, $moment + 20), 'it started before this moment: one more');
|
||||
}
|
||||
|
||||
public function testAfterAFailureOnlyAFewAttemptsAnHourApart(): void {
|
||||
$moment = 1000000;
|
||||
$finished = $moment + 60;
|
||||
$this->assertFalse(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER - 1));
|
||||
$this->assertTrue(S::isDue($moment, 0, null, 1, $finished, $finished + S::RETRY_AFTER));
|
||||
$this->assertFalse(S::isDue($moment, 0, null, S::MAX_ATTEMPTS, $finished, $finished + 86400));
|
||||
}
|
||||
}
|
||||
@@ -25,6 +25,17 @@ class RetentionPolicyTest extends TestCase {
|
||||
$this->assertCount(3, $months);
|
||||
}
|
||||
|
||||
public function testAPinnedSnapshotStaysAndDoesNotCount(): void {
|
||||
$policy = new RetentionPolicy(1, 0, 0, 0);
|
||||
$snaps = ['old' => 100, 'pinned' => 300, 'new' => 200];
|
||||
$this->assertSame(['pinned'], array_keys($policy->keep($snaps)), 'without the pin it would take the place of "last"');
|
||||
$keep = $policy->keepWithPinned($snaps, ['pinned', 'gone-already']);
|
||||
$this->assertSame(['pinned'], $keep['pinned']);
|
||||
$this->assertSame(['last'], $keep['new'], 'the newest one that is not pinned');
|
||||
$this->assertArrayNotHasKey('old', $keep);
|
||||
$this->assertArrayNotHasKey('gone-already', $keep);
|
||||
}
|
||||
|
||||
public function testKeepsEverythingWhenFewSnapshots(): void {
|
||||
$keep = (new RetentionPolicy())->keep(['a' => 100, 'b' => 200]);
|
||||
$this->assertSame(['a', 'b'], array_values(array_intersect(['a', 'b'], array_keys($keep))));
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||
|
||||
use OCA\NgBackup\Service\AlertService;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** A location without a schedule: a weekly reminder while another is backed up, else a daily one. */
|
||||
class ScheduleOffReminderTest extends TestCase {
|
||||
public function testOnceAWeekWhileAnotherLocationIsActive(): void {
|
||||
$this->assertFalse(AlertService::offReminderDue(0, 6 * 86400, true));
|
||||
$this->assertTrue(AlertService::offReminderDue(0, 7 * 86400, true));
|
||||
}
|
||||
|
||||
public function testEveryDayWhenNoLocationIsActive(): void {
|
||||
$this->assertFalse(AlertService::offReminderDue(0, 3600, false));
|
||||
$this->assertTrue(AlertService::offReminderDue(0, 86400, false));
|
||||
$this->assertTrue(AlertService::offReminderDue(0, 86400 - 60, false), 'a cron run a minute earlier than yesterday');
|
||||
}
|
||||
}
|
||||
@@ -57,15 +57,4 @@ class ScheduleTest extends TestCase {
|
||||
$this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get());
|
||||
}
|
||||
|
||||
public function testTodayAtIsTheMomentInTheSchedulesZone(): void {
|
||||
$this->values = [Schedule::TIME_KEY => '03:00', Schedule::ZONE_KEY => 'Europe/Berlin'];
|
||||
$at = $this->schedule()->todayAt(strtotime('2026-10-22 12:00 UTC'));
|
||||
$this->assertSame(strtotime('2026-10-22 01:00 UTC'), $at, '03:00 in Berlin (summer time) is 01:00 UTC');
|
||||
}
|
||||
|
||||
public function testAnUnknownZoneOrABadTimeIsRefused(): void {
|
||||
$s = $this->schedule();
|
||||
$this->expectException(\InvalidArgumentException::class);
|
||||
$s->set('03:00', 'Mars/Olympus');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,8 +2,10 @@
|
||||
# Disaster recovery: occ backup:key:import + backup:target:add + backup:restore:full on a
|
||||
# SECOND, fresh Nextcloud instance that never saw the first one, sharing only the backup
|
||||
# location (a local directory, bind-mounted into both containers) -- see PLAN.md 3.7.
|
||||
# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|salvage|salvage-files|to-pgsql|to-mysql|to-sqlite]
|
||||
# Usage: full-restore-roundtrip.sh <nextcloud-image-tag> [sqlite|pgsql|mysql] [legacy|sftp|salvage|salvage-files|to-pgsql|to-mysql|to-sqlite|no-kit]
|
||||
# e.g. full-restore-roundtrip.sh 35-apache pgsql
|
||||
# With "no-kit" the recovery kit is gone: backup:target:add takes the key from the location with
|
||||
# the passphrase, and backups wait until a new kit is confirmed.
|
||||
# With "salvage-files" one pack is damaged and another is gone from the location. The restore must
|
||||
# stop and point at --salvage; run again as the guide says, it restores the database and every file
|
||||
# that can be read, and lists the rest (exit 2). NGB_FRT_DATA_MB sets the amount of data (default 200).
|
||||
@@ -93,7 +95,10 @@ run_on ngb-frt-origin app:enable ng_backup >/dev/null
|
||||
ok "origin instance up ($DB${LEGACY:+, NG Backup 0.9.7})"
|
||||
|
||||
docker exec ngb-frt-origin sh -c "printf 'full restore roundtrip passphrase 2026\n' > /tmp/pass"
|
||||
run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI >/dev/null || fail "key init"
|
||||
DELAY_OPT=()
|
||||
[ "$MODE" = no-kit ] && DELAY_OPT=(--delete-delay-days=21)
|
||||
run_on ngb-frt-origin backup:key:init --passphrase-file=/tmp/pass --label=CI "${DELAY_OPT[@]}" >/dev/null || fail "key init"
|
||||
ORIGIN_FP=$(run_on ngb-frt-origin backup:status | grep -o 'Key: [0-9a-f-]*' | cut -d' ' -f2)
|
||||
run_on ngb-frt-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||
CODE=$(run_on ngb-frt-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-frt-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||
@@ -187,8 +192,29 @@ docker exec ngb-frt-restore chmod 644 /tmp/kit.json
|
||||
docker cp /tmp/ngb-frt-pass ngb-frt-restore:/tmp/pass
|
||||
ok "fresh restore instance up"
|
||||
|
||||
run_on ngb-frt-restore backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "key import"
|
||||
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo 2>&1) || { printf '%s\n' "$ADD_OUT"; fail "target add on the restore instance"; }
|
||||
if [ "$MODE" = no-kit ]; then
|
||||
# No recovery kit: backup:target:add takes the key from the location with a passphrase
|
||||
docker exec ngb-frt-restore rm -f /tmp/kit.json
|
||||
docker exec ngb-frt-restore sh -c "printf 'not the passphrase of the old server\n' > /tmp/wrongpass && chmod 644 /tmp/wrongpass"
|
||||
OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo --passphrase-file=/tmp/wrongpass 2>&1) \
|
||||
&& { printf '%s\n' "$OUT"; fail "a wrong passphrase was accepted"; }
|
||||
printf '%s\n' "$OUT" | grep -q "Nothing was changed" || { printf '%s\n' "$OUT"; fail "a wrong passphrase did not say that nothing was changed"; }
|
||||
STATUS_OUT=$(run_on ngb-frt-restore backup:status 2>&1 || true)
|
||||
printf '%s\n' "$STATUS_OUT" | grep -q "Key: not created" || { printf '%s\n' "$STATUS_OUT"; fail "a wrong passphrase left a key behind"; }
|
||||
[ -z "$(run_on ngb-frt-restore backup:target:list 2>&1 | grep offsite)" ] || fail "a wrong passphrase left a location behind"
|
||||
OUT=$(run_on ngb-frt-restore backup:target:add other "${LOCATION[@]}" --path=nothing-here --passphrase-file=/tmp/pass 2>&1) \
|
||||
&& { printf '%s\n' "$OUT"; fail "a folder without backups was added without a key"; }
|
||||
printf '%s\n' "$OUT" | grep -q "no backups at this location" || { printf '%s\n' "$OUT"; fail "a folder without backups did not say so"; }
|
||||
ok "without a kit: a wrong passphrase and a folder without backups change nothing"
|
||||
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo --passphrase-file=/tmp/pass 2>&1) \
|
||||
|| { printf '%s\n' "$ADD_OUT"; fail "target add with the passphrase on the restore instance"; }
|
||||
printf '%s\n' "$ADD_OUT" | grep -q "fingerprint $ORIGIN_FP" || { printf '%s\n' "$ADD_OUT"; fail "the key from the location is not the origin's ($ORIGIN_FP)"; }
|
||||
printf '%s\n' "$ADD_OUT" | grep -q "21 days (from the location)" || { printf '%s\n' "$ADD_OUT"; fail "the deletion delay did not come from the location"; }
|
||||
ok "without a kit: the key and the deletion delay came from the location"
|
||||
else
|
||||
run_on ngb-frt-restore backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "key import"
|
||||
ADD_OUT=$(run_on ngb-frt-restore backup:target:add offsite "${LOCATION[@]}" --path=repo 2>&1) || { printf '%s\n' "$ADD_OUT"; fail "target add on the restore instance"; }
|
||||
fi
|
||||
LIST_OUT=$(run_on ngb-frt-restore backup:list offsite 2>&1)
|
||||
printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
|
||||
printf 'target:add said:\n%s\nbackup:list said:\n%s\n' "$ADD_OUT" "$LIST_OUT"
|
||||
@@ -196,7 +222,8 @@ printf '%s\n' "$LIST_OUT" | grep -q "$SNAP" || {
|
||||
docker exec ngb-frt-restore sh -c 'ls -la /mnt/shared/repo /mnt/shared/repo/snapshots' || true
|
||||
fail "the origin's snapshot is not listed after connecting the location"
|
||||
}
|
||||
ok "recovery kit imported, existing repository adopted, snapshot visible"
|
||||
if [ "$MODE" = no-kit ]; then KEY_FROM="key from the location"; else KEY_FROM="recovery kit imported"; fi
|
||||
ok "$KEY_FROM, existing repository adopted, snapshot visible"
|
||||
|
||||
if [ "$MODE" = salvage ]; then
|
||||
set +e
|
||||
@@ -371,6 +398,13 @@ if [ "$MODE" = sftp ]; then
|
||||
ok "over SFTP: read ahead on connections of its own ($LOGINS SFTP logins during the restore), large file identical, no spool left"
|
||||
fi
|
||||
|
||||
if [ "$MODE" = no-kit ]; then
|
||||
OUT=$(run_on ngb-frt-restore backup:run offsite -l post-restore-check 2>&1) && { printf '%s\n' "$OUT"; fail "a backup ran without a confirmed recovery kit"; }
|
||||
printf '%s\n' "$OUT" | grep -q "Confirm the recovery kit" || { printf '%s\n' "$OUT"; fail "the refused backup does not point at the recovery kit"; }
|
||||
CODE=$(run_on ngb-frt-restore backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-frt-restore backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm a new kit after the restore"
|
||||
ok "without a kit: backups wait for a new recovery kit to be confirmed"
|
||||
fi
|
||||
run_on ngb-frt-restore backup:run offsite -l post-restore-check >/dev/null || fail "a new backup after the restore fails"
|
||||
ok "the restored instance can run a new backup of its own"
|
||||
|
||||
|
||||
Executable
+311
@@ -0,0 +1,311 @@
|
||||
#!/bin/bash
|
||||
# Getting the backup key onto a new server, with and without the recovery kit, and what the
|
||||
# administrator is told in each case. Every scenario starts on a fresh Nextcloud (SQLite) with a
|
||||
# copy of the origin's location; nothing is restored (full-restore-roundtrip.sh does that).
|
||||
# Usage: recovery-scenarios.sh [nextcloud-image-tag] e.g. recovery-scenarios.sh 35-apache
|
||||
#
|
||||
# kit kit + passphrase: backup:key:import, then backup:target:add asks nothing
|
||||
# kit-wrong kit with a wrong passphrase: refused, no key left behind
|
||||
# no-kit passphrase only: target:add asks for it (or --passphrase-file), a wrong one
|
||||
# changes nothing; backups wait for a new kit
|
||||
# no-kit-old as no-kit, on a location backed up only before 1.1.0 (no settings file):
|
||||
# the deletion delay is the default, and the admin is told so
|
||||
# no-kit-damaged as no-kit, with a damaged settings file: the longest deletion delay, and why
|
||||
# no-config-no-kit the location's config is gone and there is no kit: told that only the kit helps
|
||||
# no-config-kit config gone, kit there: --repair-config writes it again, the backups open
|
||||
# other-kit the kit of another server: refused, told to use the old server's kit
|
||||
# empty nothing at the location: told to check the details, not to make a new key
|
||||
# unused-key-kit a new key was made by mistake: key:import refuses and names
|
||||
# --replace-unused-key; with it the kit's key takes its place
|
||||
# unused-key-no-kit the same without a kit: target:add with --replace-unused-key and a passphrase
|
||||
# of the old server; a wrong one keeps the unused key
|
||||
# used-key a key that protected a location is never replaced, also after the location was removed
|
||||
# slot-catch-up a passphrase added while a location was unreachable reaches it at its next
|
||||
# backup; a new server then gets the key from it with that passphrase
|
||||
set -euo pipefail
|
||||
IMAGE="nextcloud:${1:-35-apache}"
|
||||
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
||||
SHARED=$(mktemp -d)
|
||||
chmod 777 "$SHARED"
|
||||
PASS='recovery scenarios passphrase 2026'
|
||||
fail() { echo "FAIL: $*"; exit 1; }
|
||||
ok() { echo "ok $*"; }
|
||||
|
||||
cleanup() {
|
||||
docker rm -fv ngb-rs-origin ngb-rs-other ngb-rs-new >/dev/null 2>&1 || true
|
||||
docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
|
||||
rmdir "$SHARED" 2>/dev/null || true
|
||||
return 0
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
|
||||
# Output of a command that must fail; the scenario fails when it succeeds.
|
||||
must_fail() { # container what args...
|
||||
local out
|
||||
if out=$(run_on "$1" "${@:3}" 2>&1); then printf '%s\n' "$out"; fail "$2: the command succeeded"; fi
|
||||
printf '%s\n' "$out"
|
||||
}
|
||||
expect() { # output text what
|
||||
printf '%s\n' "$1" | grep -qF -- "$2" || { printf '%s\n' "$1"; fail "$3: the output does not say \"$2\""; }
|
||||
}
|
||||
not_expect() {
|
||||
printf '%s\n' "$1" | grep -qF -- "$2" && { printf '%s\n' "$1"; fail "$3: the output says \"$2\""; }
|
||||
return 0
|
||||
}
|
||||
|
||||
start() { # name
|
||||
docker rm -fv "$1" >/dev/null 2>&1 || true
|
||||
docker run -d --name "$1" -e SQLITE_DATABASE=nextcloud -e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin \
|
||||
-v "$APP:/var/www/html/custom_apps/ng_backup:ro" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
|
||||
for i in $(seq 1 90); do run_on "$1" status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
|
||||
docker exec "$1" chown www-data:www-data /var/www/html/custom_apps
|
||||
run_on "$1" app:enable ng_backup >/dev/null
|
||||
docker exec "$1" sh -c "printf '%s\n' '$PASS' > /tmp/pass; printf 'not the passphrase at all\n' > /tmp/wrongpass; chmod 644 /tmp/pass /tmp/wrongpass"
|
||||
}
|
||||
key_of() { run_on "$1" backup:status 2>&1 | sed -n 's/^Key: \([0-9a-f-]*\),.*/\1/p'; }
|
||||
no_key() { # container what
|
||||
local s
|
||||
s=$(run_on "$1" backup:status 2>&1 || true)
|
||||
expect "$s" "Key: not created" "$2: a key was left behind"
|
||||
[ -z "$(run_on "$1" backup:target:list 2>&1 | grep offsite || true)" ] || fail "$2: a location was left behind"
|
||||
}
|
||||
location() { echo "local -a null::null -o datadir=/mnt/shared/$1 --path=repo"; }
|
||||
# A fresh server with its own copy of the origin's location
|
||||
scenario() { # name
|
||||
docker exec ngb-rs-origin sh -c "rm -rf /mnt/shared/$1 && mkdir -p /mnt/shared/$1 && cp -a /mnt/shared/origin/repo /mnt/shared/$1/ && chown -R www-data /mnt/shared/$1"
|
||||
start ngb-rs-new
|
||||
docker exec -i ngb-rs-new sh -c 'cat > /tmp/kit.json && chmod 644 /tmp/kit.json' < "$SHARED/kit.json"
|
||||
}
|
||||
|
||||
# The origin: a key with a deletion delay of its own, a kit, one backup
|
||||
start ngb-rs-origin
|
||||
run_on ngb-rs-origin backup:key:init --passphrase-file=/tmp/pass --label=Safe --delete-delay-days=21 >/dev/null || fail "key init"
|
||||
run_on ngb-rs-origin backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||
CODE=$(run_on ngb-rs-origin backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-rs-origin backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||
docker exec ngb-rs-origin sh -c 'mkdir -p /mnt/shared/origin && chown www-data /mnt/shared/origin'
|
||||
# shellcheck disable=SC2046
|
||||
run_on ngb-rs-origin backup:target:add offsite $(location origin) >/dev/null || fail "target add on the origin"
|
||||
run_on ngb-rs-origin backup:run offsite >/dev/null || fail "backup on the origin"
|
||||
SNAP=$(run_on ngb-rs-origin backup:list offsite | grep -o '[0-9a-f]\{32\}' | head -1)
|
||||
FP=$(key_of ngb-rs-origin)
|
||||
docker exec ngb-rs-origin cat /tmp/kit.json > "$SHARED/kit.json"
|
||||
docker exec ngb-rs-origin test -f /mnt/shared/origin/repo/settings || fail "the origin's location has no settings file"
|
||||
ok "origin: key $FP, kit, snapshot $SNAP, settings on the location"
|
||||
|
||||
# kit: the passphrase once, for the kit; target:add asks nothing more
|
||||
scenario kit
|
||||
OUT=$(run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "kit: key import"; }
|
||||
expect "$OUT" "fingerprint $FP" "kit"
|
||||
expect "$OUT" "21 days (from the kit)" "kit"
|
||||
expect "$OUT" "backup:target:add" "kit: the next step"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location kit) --no-interaction 2>&1) || { printf '%s\n' "$OUT"; fail "kit: target add"; }
|
||||
not_expect "$OUT" "Passphrase" "kit: target:add asked for a passphrase"
|
||||
expect "$OUT" "found and adopted" "kit"
|
||||
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "kit: the snapshot is not listed"
|
||||
run_on ngb-rs-new backup:run offsite >/dev/null || fail "kit: a backup after the kit import"
|
||||
ok "kit: imported with the passphrase, location added without asking again, backups run"
|
||||
|
||||
# kit-wrong
|
||||
scenario kit-wrong
|
||||
OUT=$(must_fail ngb-rs-new "kit-wrong" backup:key:import /tmp/kit.json --passphrase-file=/tmp/wrongpass)
|
||||
expect "$OUT" "Wrong passphrase" "kit-wrong"
|
||||
no_key ngb-rs-new "kit-wrong"
|
||||
ok "kit-wrong: refused, nothing left behind"
|
||||
|
||||
# no-kit
|
||||
scenario no-kit
|
||||
docker exec ngb-rs-new rm /tmp/kit.json
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-kit without a passphrase" backup:target:add offsite $(location no-kit) --no-interaction)
|
||||
expect "$OUT" "this location holds backups" "no-kit: explains where the key comes from"
|
||||
expect "$OUT" "--passphrase-file" "no-kit: says how to give the passphrase"
|
||||
no_key ngb-rs-new "no-kit without a passphrase"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-kit wrong passphrase" backup:target:add offsite $(location no-kit) --passphrase-file=/tmp/wrongpass)
|
||||
expect "$OUT" "Wrong passphrase" "no-kit wrong passphrase"
|
||||
expect "$OUT" "Nothing was changed" "no-kit wrong passphrase"
|
||||
no_key ngb-rs-new "no-kit wrong passphrase"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit: target add"; }
|
||||
expect "$OUT" "fingerprint $FP" "no-kit"
|
||||
expect "$OUT" "21 days (from the location)" "no-kit"
|
||||
expect "$OUT" "download a new recovery kit" "no-kit: asks for a new kit"
|
||||
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "no-kit: the snapshot is not listed"
|
||||
OUT=$(must_fail ngb-rs-new "no-kit backup before a new kit" backup:run offsite)
|
||||
expect "$OUT" "Confirm the recovery kit" "no-kit: a backup before a new kit"
|
||||
CODE=$(run_on ngb-rs-new backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-rs-new backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "no-kit: confirm the new kit"
|
||||
run_on ngb-rs-new backup:run offsite >/dev/null || fail "no-kit: a backup after the new kit"
|
||||
ok "no-kit: passphrase asked, wrong one changes nothing, key and delay from the location, backups wait for a new kit"
|
||||
|
||||
# no-kit-old: a location from before 1.1.0 has no settings file
|
||||
scenario no-kit-old
|
||||
docker exec ngb-rs-new rm /tmp/kit.json /mnt/shared/no-kit-old/repo/settings
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit-old) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit-old: target add"; }
|
||||
expect "$OUT" "fingerprint $FP" "no-kit-old"
|
||||
expect "$OUT" "7 days (the default" "no-kit-old: the default delay and why"
|
||||
docker exec ngb-rs-new test -f /mnt/shared/no-kit-old/repo/settings || fail "no-kit-old: the settings were not written when the location was added"
|
||||
ok "no-kit-old: key from the location, default delay with the reason, settings written now"
|
||||
|
||||
# no-kit-damaged: the settings file is there but damaged: the longest delay, and the admin is told
|
||||
scenario no-kit-damaged
|
||||
docker exec ngb-rs-new sh -c 'rm /tmp/kit.json && printf garbage >> /mnt/shared/no-kit-damaged/repo/settings'
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-kit-damaged) --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "no-kit-damaged: target add"; }
|
||||
expect "$OUT" "fingerprint $FP" "no-kit-damaged"
|
||||
expect "$OUT" "365 days (the longest" "no-kit-damaged: the longest delay and why"
|
||||
ok "no-kit-damaged: key from the location, the longest delay with the reason"
|
||||
|
||||
# no-config-no-kit
|
||||
scenario no-config-no-kit
|
||||
docker exec ngb-rs-new rm /tmp/kit.json /mnt/shared/no-config-no-kit/repo/config
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-config-no-kit" backup:target:add offsite $(location no-config-no-kit) --passphrase-file=/tmp/pass)
|
||||
expect "$OUT" "its config file is gone" "no-config-no-kit"
|
||||
expect "$OUT" "only the recovery kit has the key" "no-config-no-kit"
|
||||
expect "$OUT" "--repair-config" "no-config-no-kit"
|
||||
no_key ngb-rs-new "no-config-no-kit"
|
||||
docker exec ngb-rs-new test ! -f /mnt/shared/no-config-no-kit/repo/config || fail "no-config-no-kit: a config was written"
|
||||
ok "no-config-no-kit: told the kit is needed, nothing written"
|
||||
|
||||
# no-config-kit
|
||||
scenario no-config-kit
|
||||
docker exec ngb-rs-new rm /mnt/shared/no-config-kit/repo/config
|
||||
run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass >/dev/null || fail "no-config-kit: key import"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "no-config-kit without --repair-config" backup:target:add offsite $(location no-config-kit))
|
||||
expect "$OUT" "--repair-config" "no-config-kit: points at --repair-config"
|
||||
docker exec ngb-rs-new test ! -f /mnt/shared/no-config-kit/repo/config || fail "no-config-kit: a config was written without --repair-config"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location no-config-kit) --repair-config 2>&1) || { printf '%s\n' "$OUT"; fail "no-config-kit: --repair-config"; }
|
||||
expect "$OUT" "written again" "no-config-kit"
|
||||
expect "$OUT" "found and adopted" "no-config-kit"
|
||||
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "no-config-kit: the snapshot is not listed after the repair"
|
||||
run_on ngb-rs-new backup:browse offsite "$SNAP" >/dev/null 2>&1 || fail "no-config-kit: the snapshot cannot be read after the repair"
|
||||
ok "no-config-kit: refused without --repair-config, config written again with it, the backups open"
|
||||
|
||||
# other-kit: the kit of another server
|
||||
start ngb-rs-other
|
||||
run_on ngb-rs-other backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "other: key init"
|
||||
run_on ngb-rs-other backup:key:kit --output=/tmp/other.json >/dev/null
|
||||
docker exec ngb-rs-other cat /tmp/other.json > "$SHARED/other.json"
|
||||
docker rm -fv ngb-rs-other >/dev/null
|
||||
scenario other-kit
|
||||
docker exec ngb-rs-new rm /tmp/kit.json
|
||||
docker exec -i ngb-rs-new sh -c 'cat > /tmp/other.json && chmod 644 /tmp/other.json' < "$SHARED/other.json"
|
||||
run_on ngb-rs-new backup:key:import /tmp/other.json --passphrase-file=/tmp/pass >/dev/null || fail "other-kit: key import"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "other-kit" backup:target:add offsite $(location other-kit))
|
||||
expect "$OUT" "different key" "other-kit"
|
||||
expect "$OUT" "must be the old server's" "other-kit: points at the right kit"
|
||||
[ -z "$(run_on ngb-rs-new backup:target:list 2>&1 | grep offsite || true)" ] || fail "other-kit: a location was left behind"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "other-kit repair" backup:target:add offsite $(location other-kit) --repair-config)
|
||||
expect "$OUT" "leave out --repair-config" "other-kit: a config that is there is not repaired"
|
||||
ok "other-kit: refused, told to use the old server's kit, the config untouched"
|
||||
|
||||
# empty
|
||||
scenario empty
|
||||
docker exec ngb-rs-new sh -c 'mkdir -p /mnt/shared/nothing && chown www-data /mnt/shared/nothing'
|
||||
OUT=$(must_fail ngb-rs-new "empty" backup:target:add offsite local -a null::null -o datadir=/mnt/shared/nothing --path=repo --passphrase-file=/tmp/pass)
|
||||
expect "$OUT" "no backups at this location" "empty"
|
||||
expect "$OUT" "Check the connection details and --path" "empty"
|
||||
no_key ngb-rs-new "empty"
|
||||
ok "empty: told to check the details, no key made"
|
||||
|
||||
# unused-key-kit: a key made by mistake, then the kit
|
||||
scenario unused-key-kit
|
||||
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "unused-key-kit: key init"
|
||||
MISTAKE=$(key_of ngb-rs-new)
|
||||
OUT=$(must_fail ngb-rs-new "unused-key-kit without the option" backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass)
|
||||
expect "$OUT" "never used" "unused-key-kit"
|
||||
expect "$OUT" "--replace-unused-key" "unused-key-kit: names the option"
|
||||
[ "$(key_of ngb-rs-new)" = "$MISTAKE" ] || fail "unused-key-kit: the key changed without the option"
|
||||
OUT=$(must_fail ngb-rs-new "unused-key-kit wrong passphrase" backup:key:import /tmp/kit.json --passphrase-file=/tmp/wrongpass --replace-unused-key)
|
||||
expect "$OUT" "Wrong passphrase" "unused-key-kit wrong passphrase"
|
||||
[ "$(key_of ngb-rs-new)" = "$MISTAKE" ] || fail "unused-key-kit: a wrong passphrase changed the key"
|
||||
OUT=$(run_on ngb-rs-new backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass --replace-unused-key 2>&1) || { printf '%s\n' "$OUT"; fail "unused-key-kit: replace"; }
|
||||
expect "$OUT" "The unused key was replaced" "unused-key-kit"
|
||||
expect "$OUT" "fingerprint $FP" "unused-key-kit"
|
||||
expect "$OUT" "21 days (from the kit)" "unused-key-kit"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location unused-key-kit) --no-interaction 2>&1) || { printf '%s\n' "$OUT"; fail "unused-key-kit: target add"; }
|
||||
expect "$OUT" "found and adopted" "unused-key-kit"
|
||||
run_on ngb-rs-new backup:run offsite >/dev/null || fail "unused-key-kit: a backup after the replacement"
|
||||
ok "unused-key-kit: refused without the option, a wrong passphrase changes nothing, replaced with it, backups run"
|
||||
|
||||
# unused-key-no-kit: a key made by mistake, no kit
|
||||
scenario unused-key-no-kit
|
||||
docker exec ngb-rs-new rm /tmp/kit.json
|
||||
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "unused-key-no-kit: key init"
|
||||
MISTAKE=$(key_of ngb-rs-new)
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "unused-key-no-kit without the option" backup:target:add offsite $(location unused-key-no-kit) --no-interaction)
|
||||
expect "$OUT" "different key" "unused-key-no-kit"
|
||||
expect "$OUT" "never used" "unused-key-no-kit: says the key can make way"
|
||||
expect "$OUT" "--replace-unused-key" "unused-key-no-kit: names the option"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "unused-key-no-kit wrong passphrase" backup:target:add offsite $(location unused-key-no-kit) --replace-unused-key --passphrase-file=/tmp/wrongpass)
|
||||
expect "$OUT" "Nothing was changed" "unused-key-no-kit wrong passphrase"
|
||||
[ "$(key_of ngb-rs-new)" = "$MISTAKE" ] || fail "unused-key-no-kit: a wrong passphrase changed the key"
|
||||
[ -z "$(run_on ngb-rs-new backup:target:list 2>&1 | grep offsite || true)" ] || fail "unused-key-no-kit: a location was left behind"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-new backup:target:add offsite $(location unused-key-no-kit) --replace-unused-key --passphrase-file=/tmp/pass 2>&1) || { printf '%s\n' "$OUT"; fail "unused-key-no-kit: replace"; }
|
||||
expect "$OUT" "The unused key was replaced" "unused-key-no-kit"
|
||||
expect "$OUT" "fingerprint $FP" "unused-key-no-kit"
|
||||
expect "$OUT" "download a new recovery kit" "unused-key-no-kit: asks for a new kit"
|
||||
run_on ngb-rs-new backup:list offsite | grep -q "$SNAP" || fail "unused-key-no-kit: the snapshot is not listed"
|
||||
OUT=$(must_fail ngb-rs-new "unused-key-no-kit backup before a new kit" backup:run offsite)
|
||||
expect "$OUT" "Confirm the recovery kit" "unused-key-no-kit: the old key's kit confirmation does not count"
|
||||
ok "unused-key-no-kit: hint on the refusal, a wrong passphrase changes nothing, replaced with a passphrase, a new kit is needed"
|
||||
|
||||
# used-key: a key that protected a location stays, also after that location is removed
|
||||
scenario used-key
|
||||
docker exec ngb-rs-new sh -c 'mkdir -p /mnt/shared/own && chown www-data /mnt/shared/own'
|
||||
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "used-key: key init"
|
||||
OWN=$(key_of ngb-rs-new)
|
||||
run_on ngb-rs-new backup:target:add own local -a null::null -o datadir=/mnt/shared/own --path=repo >/dev/null || fail "used-key: own location"
|
||||
run_on ngb-rs-new backup:target:remove own >/dev/null || fail "used-key: remove the own location"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "used-key target:add" backup:target:add offsite $(location used-key) --replace-unused-key --passphrase-file=/tmp/pass)
|
||||
expect "$OUT" "never replaced" "used-key target:add"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(must_fail ngb-rs-new "used-key target:add without the option" backup:target:add offsite $(location used-key) --no-interaction)
|
||||
not_expect "$OUT" "--replace-unused-key" "used-key: offers a replacement for a used key"
|
||||
OUT=$(must_fail ngb-rs-new "used-key key:import" backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass --replace-unused-key)
|
||||
expect "$OUT" "never replaced" "used-key key:import"
|
||||
[ "$(key_of ngb-rs-new)" = "$OWN" ] || fail "used-key: the key changed"
|
||||
ok "used-key: never replaced, by kit or passphrase, also after its location was removed"
|
||||
|
||||
# slot-catch-up: a passphrase change that missed a location reaches it at the next backup
|
||||
start ngb-rs-new
|
||||
docker exec ngb-rs-new sh -c "printf '%s\n' 'catch up passphrase of the cto' > /tmp/pass2 && chmod 644 /tmp/pass2 && rm -rf /mnt/shared/catchup && mkdir -p /mnt/shared/catchup && chown www-data /mnt/shared/catchup"
|
||||
run_on ngb-rs-new backup:key:init --passphrase-file=/tmp/pass --label=Safe >/dev/null || fail "slot-catch-up: key init"
|
||||
CODE=$(run_on ngb-rs-new backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-rs-new backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "slot-catch-up: confirm"
|
||||
# shellcheck disable=SC2046
|
||||
run_on ngb-rs-new backup:target:add offsite $(location catchup) >/dev/null || fail "slot-catch-up: target add"
|
||||
run_on ngb-rs-new backup:run offsite >/dev/null || fail "slot-catch-up: first backup"
|
||||
BEFORE=$(docker exec ngb-rs-new md5sum /mnt/shared/catchup/repo/config | cut -d' ' -f1)
|
||||
docker exec ngb-rs-new chmod 000 /mnt/shared/catchup/repo
|
||||
OUT=$(must_fail ngb-rs-new "slot-catch-up: slot add with the location unreachable" backup:key:slot:add --label=CTO --passphrase-file=/tmp/pass2)
|
||||
expect "$OUT" "tried again at their next backup" "slot-catch-up: says the change follows"
|
||||
docker exec ngb-rs-new chmod 755 /mnt/shared/catchup/repo
|
||||
[ "$(docker exec ngb-rs-new md5sum /mnt/shared/catchup/repo/config | cut -d' ' -f1)" = "$BEFORE" ] || fail "slot-catch-up: the location changed while unreachable"
|
||||
CODE=$(run_on ngb-rs-new backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
run_on ngb-rs-new backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "slot-catch-up: confirm the new kit"
|
||||
run_on ngb-rs-new backup:run offsite >/dev/null || fail "slot-catch-up: backup after the location came back"
|
||||
[ "$(docker exec ngb-rs-new md5sum /mnt/shared/catchup/repo/config | cut -d' ' -f1)" != "$BEFORE" ] || fail "slot-catch-up: the backup did not bring the new passphrase to the location"
|
||||
CATCHFP=$(key_of ngb-rs-new)
|
||||
start ngb-rs-other
|
||||
docker exec ngb-rs-other sh -c "printf '%s\n' 'catch up passphrase of the cto' > /tmp/pass2 && chmod 644 /tmp/pass2"
|
||||
# shellcheck disable=SC2046
|
||||
OUT=$(run_on ngb-rs-other backup:target:add offsite $(location catchup) --passphrase-file=/tmp/pass2 2>&1) || { printf '%s\n' "$OUT"; fail "slot-catch-up: a new server with the new passphrase"; }
|
||||
expect "$OUT" "fingerprint $CATCHFP" "slot-catch-up: the new server"
|
||||
ok "slot-catch-up: a passphrase that missed a location reaches it at the next backup and opens it on a new server"
|
||||
|
||||
echo "RECOVERY SCENARIOS PASS"
|
||||
@@ -23,12 +23,15 @@ ok "key, kit code and confirmation"
|
||||
occ backup:target:add ci $TARGET_ARGS --path=repo >/dev/null || fail "target add"
|
||||
ok "location added"
|
||||
|
||||
# Test data: a user with files.
|
||||
# Test data: a user with files. No skeleton files on the first login, below, in the middle of the checks.
|
||||
occ config:system:set skeletondirectory --value='' >/dev/null
|
||||
occ config:system:set templatedirectory --value='' >/dev/null
|
||||
$X sh -c "OC_PASS='Roundtrip-pw-2026' php occ user:add --password-from-env $U >/dev/null"
|
||||
# large.bin (40 MB) fills more than one pack: a restore to a directory reads ahead over several ranges
|
||||
$X sh -c "mkdir -p $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs/Empty/Nested $DATA/$U/files/Hollow && touch -d @1700000000 $DATA/$U/files/Hollow && for i in 1 2 3 4 5; do head -c 300000 /dev/urandom > $DATA/$U/files/Docs/f\$i.bin; done; echo hello > $DATA/$U/files/Docs/Sub/note.txt; head -c 6000000 /dev/urandom > $DATA/$U/files/big.bin; head -c 40000000 /dev/urandom > $DATA/$U/files/large.bin"
|
||||
$X sh -c "touch -d @1600000000 $DATA/$U/files/Docs/Sub $DATA/$U/files/Docs"
|
||||
occ files:scan "$U" >/dev/null
|
||||
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||
SUM_BEFORE=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||
|
||||
occ backup:run ci -l first >/dev/null || fail "first backup"
|
||||
SECOND=$(occ backup:run ci -l second)
|
||||
@@ -44,6 +47,14 @@ occ files:scan "$U" >/dev/null
|
||||
occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m new-folder >/dev/null || fail "restore new-folder"
|
||||
$X sh -c "ls $DATA/$U/files | grep -q '^Restored '" || fail "no Restored folder"
|
||||
$X sh -c "test -d $DATA/$U/files/Restored\ */Docs/Empty/Nested" || fail "the empty folder is not in the Restored folder"
|
||||
# Folders with files: on disk and in what Nextcloud shows, the parent as well as the child.
|
||||
RDIR=$($X sh -c "cd $DATA/$U/files && ls -d Restored\ *")
|
||||
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs/Sub")" = 1600000000 ] || fail "a folder with files did not get its time back"
|
||||
[ "$($X sh -c "stat -c %Y $DATA/$U/files/Restored\ */Docs")" = 1600000000 ] || fail "the parent folder did not get its time back"
|
||||
DAV=$($X curl -sf -u "$U:Roundtrip-pw-2026" -X PROPFIND -H 'Depth: 1' "http://localhost/remote.php/dav/files/$U/$(php -r 'echo rawurlencode($argv[1]);' "$RDIR")/Docs/") || fail "PROPFIND"
|
||||
[ "$(echo "$DAV" | grep -o 'Sun, 13 Sep 2020 12:26:40 GMT' | wc -l)" = 2 ] || fail "Nextcloud does not show the time of the folders: $(echo "$DAV" | grep -o '<d:getlastmodified>[^<]*' | sort | uniq -c)"
|
||||
occ backup:restore ci "$SNAP" "data/$U/files/Docs/Sub" -m merge >/dev/null || fail "restore into a folder that is there"
|
||||
[ "$($X stat -c %Y "$DATA/$U/files/Docs/Sub")" != 1600000000 ] || fail "a folder that was already there got the old time"
|
||||
occ backup:browse ci "$SNAP" "data/$U/files" | grep Hollow >/dev/null || fail "an empty folder is not shown when browsing the snapshot"
|
||||
occ backup:restore ci "$SNAP" "data/$U/files/Hollow" -m merge >/dev/null || fail "restore of a folder that was empty"
|
||||
$X sh -c "test -d $DATA/$U/files/Hollow" || fail "the empty folder did not come back by itself"
|
||||
@@ -52,8 +63,8 @@ occ backup:restore ci "$SNAP" "data/$U/files/Docs" -m replace >/dev/null || fail
|
||||
$X sh -c "test -d $DATA/$U/files/Docs/Empty/Nested" || fail "replace did not bring back the empty folder, or moved it to the trash"
|
||||
ok "empty folders: restored in each mode, by themselves and shown when browsing"
|
||||
$X sh -c "rm -rf '$DATA/$U/files/Restored '*" ; occ files:scan "$U" >/dev/null
|
||||
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore"
|
||||
SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after restore: $(diff <(echo "$SUM_BEFORE") <(echo "$SUM_AFTER") | cut -c1-120)"
|
||||
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
||||
ok "restore new-folder and replace: files identical, extra file in the trash"
|
||||
|
||||
@@ -67,7 +78,7 @@ ok "restore of one file to a directory; the temporary of a stopped restore remov
|
||||
|
||||
# All of the user's files to a directory: over S3 and SFTP the packs are read ahead
|
||||
occ backup:restore ci "$SNAP" "data/$U/files" -d /tmp/ngb-rt-dir >/dev/null || fail "restore of a folder to a directory"
|
||||
SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||
SUM_DIR=$($X sh -c "cd /tmp/ngb-rt-dir/data/$U/files && find . -type f -print0 | sort -z | xargs -0 md5sum")
|
||||
[ "$SUM_BEFORE" = "$SUM_DIR" ] || fail "the files restored to a directory differ"
|
||||
$X sh -c "ls -A /tmp/ngb-rt-dir | grep -q ngb-restore-spool" && fail "the read-ahead spool was left in the directory"
|
||||
$X sh -c "test -d /tmp/ngb-rt-dir/data/$U/files/Docs/Empty/Nested" || fail "the empty folder is not in the directory"
|
||||
@@ -101,13 +112,98 @@ ok "without system cron: warning in backup:status and the API, backup finished b
|
||||
# The Bin reads in parts, as the page does (what is in the parts: CatalogTest)
|
||||
api "$API/targets/$TID/trash" | php -r '$p = json_decode(stream_get_contents(STDIN), true); exit(is_int($p["total"] ?? null) && is_array($p["items"] ?? null) && count($p["items"]) <= $p["total"] ? 0 : 1);' || fail "the Bin API does not give a total and items"
|
||||
ok "the Bin API reads in parts, with a total"
|
||||
# Locations are backed up one after the other: the second one started is queued until the first is done.
|
||||
for L in 2 3; do
|
||||
$X sh -c "mkdir -p /tmp/ngb-rt-target$L"
|
||||
occ backup:target:add ci$L local -a null::null -o datadir=/tmp/ngb-rt-target$L --path=repo >/dev/null || fail "target add ci$L"
|
||||
done
|
||||
IDS=$(api "$API/status" | php -r 'foreach (json_decode(stream_get_contents(STDIN), true)["targets"] as $t) { if ($t["name"] === "ci2" || $t["name"] === "ci3") { echo $t["id"], " "; } }')
|
||||
for ID in $IDS; do api -X POST -d "{\"targetId\": $ID, \"label\": \"turn\"}" "$API/backups" >/dev/null || fail "start a backup to location $ID"; done
|
||||
# Never two runs to one location: a second start gets the run that is there.
|
||||
ID=${IDS%% *}
|
||||
RUNS=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||
api -X POST -d "{\"targetId\": $ID, \"label\": \"again\"}" "$API/backups" >/dev/null || fail "second start"
|
||||
AGAIN=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||
[ "$RUNS" = 2 ] && [ "$AGAIN" = 2 ] || fail "a second start to one location: $RUNS, then $AGAIN running"
|
||||
occ backup:status | grep "queued" >/dev/null || fail "the second backup is not queued"
|
||||
QUEUED=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["queued"]));')
|
||||
[ "$QUEUED" = 1 ] || fail "$QUEUED backups queued instead of 1"
|
||||
for i in $(seq 1 60); do
|
||||
api -X POST "$API/tick" >/dev/null || fail "tick"
|
||||
RUNNING=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||
[ "$RUNNING" != 0 ] || break
|
||||
done
|
||||
[ "$RUNNING" = 0 ] || fail "the queued backups did not finish"
|
||||
occ backup:list ci3 | grep " turn " >/dev/null || fail "the queued backup made no snapshot"
|
||||
ok "locations in turn: the second backup queued, and done after the first"
|
||||
# Heavy background jobs of other apps wait during a backup, and only then. Ticks keep this one moving.
|
||||
HEAVY='OCA\Recognize\BackgroundJobs\NgbTestJob'
|
||||
reserved() { $X php -r 'require "lib/base.php"; $q = \OC::$server->get(\OCP\IDBConnection::class)->getQueryBuilder(); echo (int)$q->select("reserved_at")->from("jobs")->where($q->expr()->eq("class", $q->createNamedParameter($argv[1])))->executeQuery()->fetchOne();' "$HEAVY"; }
|
||||
tick_until_done() {
|
||||
for i in $(seq 1 60); do
|
||||
api -X POST "$API/tick" >/dev/null || fail "tick"
|
||||
RUNNING=$(api "$API/status" | php -r 'echo count(array_filter(json_decode(stream_get_contents(STDIN), true)["runs"], fn ($r) => $r["status"] === "running"));')
|
||||
[ "$RUNNING" != 0 ] || return 0
|
||||
done
|
||||
fail "the backup did not finish"
|
||||
}
|
||||
$X php -r 'require "lib/base.php"; \OC::$server->get(\OCP\BackgroundJob\IJobList::class)->add($argv[1]);' "$HEAVY" || fail "add a job"
|
||||
[ "$(reserved)" = 0 ] || fail "a new job is reserved"
|
||||
L3=$(echo "$IDS" | awk '{print $2}')
|
||||
api -X POST -d "{\"targetId\": $L3, \"label\": \"heavy\"}" "$API/backups" >/dev/null || fail "start a backup"
|
||||
tick_until_done
|
||||
[ "$(reserved)" != 0 ] || fail "the job was not paused during the backup"
|
||||
api -X POST "$API/tick" >/dev/null
|
||||
[ "$(reserved)" = 0 ] || fail "the job stays paused after the backup"
|
||||
occ backup:pause-jobs off >/dev/null || fail "turn off"
|
||||
sleep 8 # APCu app config cache of the web server
|
||||
api -X POST -d "{\"targetId\": $L3, \"label\": \"heavy\"}" "$API/backups" >/dev/null || fail "start a backup"
|
||||
tick_until_done
|
||||
[ "$(reserved)" = 0 ] || fail "the job was paused with the setting off"
|
||||
occ backup:pause-jobs on >/dev/null
|
||||
$X php -r 'require "lib/base.php"; \OC::$server->get(\OCP\BackgroundJob\IJobList::class)->remove($argv[1]);' "$HEAVY"
|
||||
ok "heavy background jobs: paused during a backup, back after it, left alone with the setting off"
|
||||
occ background:cron >/dev/null
|
||||
$X php -f /var/www/html/cron.php >/dev/null 2>&1 || true
|
||||
occ backup:status | grep "system cron" >/dev/null && fail "backup:status still warns after cron ran"
|
||||
sleep 8 # APCu app config cache of the web server
|
||||
api "$API/status" | php -r 'exit(json_decode(stream_get_contents(STDIN), true)["cron"]["ok"] ? 0 : 1);' || fail "the status API still reports a missing cron after cron ran"
|
||||
ok "with a system cron that ran: no warning in backup:status or the API"
|
||||
|
||||
# A schedule per location. Choosing a time starts nothing; the moment itself does, and only there.
|
||||
JOB=$(occ background-job:list --class='OCA\NgBackup\BackgroundJob\BackupJob' --output=json | php -r 'echo json_decode(stream_get_contents(STDIN), true)[0]["id"];')
|
||||
job() { occ background-job:execute "$JOB" --force-execute >/dev/null; }
|
||||
occ backup:target:schedule ci off >/dev/null
|
||||
occ backup:target:schedule ci3 off >/dev/null
|
||||
occ backup:target:schedule ci2 00:00 --timezone=UTC >/dev/null || fail "set a schedule"
|
||||
occ backup:target:schedule ci2 | grep "00:00 (UTC)" >/dev/null || fail "schedule not shown"
|
||||
job
|
||||
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 0 ] || fail "choosing a time started a backup"
|
||||
occ backup:target:schedule ci2 "$($X date -u -d '+1 minute' +%H:%M)/6" --timezone=UTC >/dev/null || fail "set every 6 hours"
|
||||
sleep 65
|
||||
job
|
||||
occ backup:list ci2 | grep " scheduled " >/dev/null || fail "the scheduled moment started no backup"
|
||||
[ "$(occ backup:list ci3 | grep -c ' scheduled ')" = 0 ] || fail "a location that is off was backed up"
|
||||
job
|
||||
[ "$(occ backup:list ci2 | grep -c ' scheduled ')" = 1 ] || fail "one moment started more than one backup"
|
||||
occ backup:target:schedule ci2 nonsense >/dev/null 2>&1 && fail "a bad schedule was accepted"
|
||||
ok "schedule per location: nothing on choosing a time, one backup at the moment, none when off"
|
||||
|
||||
# What administrators are told, as Nextcloud shows it.
|
||||
notes() { api "http://localhost/ocs/v2.php/apps/notifications/api/v2/notifications?format=json"; }
|
||||
occ backup:target:schedule ci2 off >/dev/null
|
||||
$X sh -c "touch '$DATA/ngb-rt-bad\\name.txt'"
|
||||
occ backup:run ci3 -l names | grep "their name is not supported" >/dev/null || fail "occ does not report the name"
|
||||
$X sh -c "rm '$DATA/ngb-rt-bad\\name.txt'"
|
||||
NOTES=$(notes) || fail "read the notifications"
|
||||
echo "$NOTES" | grep "turned off the schedule of backup location ci2" >/dev/null || fail "no notification about the schedule that was turned off"
|
||||
echo "$NOTES" | grep "is missing because of its name" >/dev/null || fail "no notification about the name"
|
||||
echo "$NOTES" | grep "Rename them" >/dev/null || fail "the notification about the name does not ask to rename"
|
||||
job
|
||||
notes | grep "has no schedule" >/dev/null && fail "a reminder right after turning it off"
|
||||
for L in 2 3; do occ backup:target:remove ci$L >/dev/null || fail "target remove ci$L"; done
|
||||
occ user:delete "$ADM" >/dev/null
|
||||
ok "notifications: schedule turned off, name not supported, no early reminder"
|
||||
|
||||
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
|
||||
occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify"
|
||||
@@ -120,6 +216,17 @@ ok "database dump and restore"
|
||||
|
||||
# Retention, trash and catalog.
|
||||
occ backup:retention --last=1 --daily=0 --weekly=0 --monthly=0 >/dev/null
|
||||
# A pinned restore point stays, and a note can change afterwards.
|
||||
FIRST=$(occ backup:list ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
|
||||
occ backup:pin ci "$FIRST" >/dev/null || fail "pin"
|
||||
occ backup:note ci "$FIRST" "a note made later" >/dev/null || fail "note"
|
||||
occ backup:list ci | grep " first " | grep " yes " | grep "a note made later" >/dev/null || fail "pin and note not listed"
|
||||
occ backup:prune ci --dry-run | grep "would forget 1 " >/dev/null || fail "a pinned snapshot would be removed"
|
||||
occ backup:pin ci "$FIRST" --unpin >/dev/null || fail "unpin"
|
||||
occ backup:prune ci --dry-run | grep "would forget 2 " >/dev/null || fail "an unpinned snapshot would stay"
|
||||
occ backup:note ci "$FIRST" >/dev/null || fail "remove note"
|
||||
occ backup:list ci | grep -q "a note made later" && fail "note not removed"
|
||||
ok "pin and note on a restore point"
|
||||
occ backup:prune ci | grep "forgot 2" >/dev/null || fail "prune" # first and second; "page" (above) is the last
|
||||
occ backup:trash ci | grep " first " >/dev/null || fail "trash"
|
||||
TID=$(occ backup:trash ci | awk -F'|' '!f && / first /{gsub(/ /,"",$3); print $3; f=1}')
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
#!/bin/bash
|
||||
# The update from a released version to this working tree, on a server that was backing up.
|
||||
# Usage: upgrade-from-release.sh <release package .tar.gz> [nextcloud-image-tag] [sqlite|pgsql] [new-version]
|
||||
# e.g. upgrade-from-release.sh ~/dev-local/release/ng_backup-1.0.3/ng_backup-1.0.3.tar.gz 34-apache pgsql 1.1.0
|
||||
#
|
||||
# The old version gets a key, a confirmed kit, a location with a daily time and one backup. Then the
|
||||
# app files are replaced by this working tree (version raised to new-version when given) and
|
||||
# occ upgrade runs. After it: the location keeps its time as its own schedule, the old restore point
|
||||
# is listed, verifies and restores a file byte for byte, a new backup runs, the location gets its
|
||||
# settings file, and the key counts as used.
|
||||
set -euo pipefail
|
||||
PKG=${1:?release package}
|
||||
IMAGE="nextcloud:${2:-34-apache}"
|
||||
DB=${3:-sqlite}
|
||||
NEWVER=${4:-}
|
||||
APP=$(cd "$(dirname "$0")/../.." && pwd)
|
||||
N=ngb-up; NET=ngb-up-net
|
||||
SHARED=$(mktemp -d)
|
||||
NEWAPP=$(mktemp -d)
|
||||
chmod 777 "$SHARED"
|
||||
PASS='upgrade test passphrase 2026'
|
||||
fail() { echo "FAIL: $*"; exit 1; }
|
||||
ok() { echo "ok $*"; }
|
||||
cleanup() {
|
||||
docker rm -fv $N $N-db >/dev/null 2>&1 || true
|
||||
docker network rm $NET >/dev/null 2>&1 || true
|
||||
docker run --rm -v "$SHARED:/target" alpine sh -c 'rm -rf /target/*' >/dev/null 2>&1 || true
|
||||
rm -rf "$NEWAPP"
|
||||
rmdir "$SHARED" 2>/dev/null || true
|
||||
return 0
|
||||
}
|
||||
trap cleanup EXIT
|
||||
occ() { docker exec -u www-data $N php occ "$@"; }
|
||||
|
||||
docker network create $NET >/dev/null
|
||||
env=(-e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=admin-upgrade-pw)
|
||||
case $DB in
|
||||
sqlite) env+=(-e SQLITE_DATABASE=nextcloud) ;;
|
||||
pgsql) docker run -d --name $N-db --network $NET -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nc -e POSTGRES_PASSWORD=nc postgres:16-alpine >/dev/null
|
||||
env+=(-e POSTGRES_HOST=$N-db -e POSTGRES_DB=nextcloud -e POSTGRES_USER=nc -e POSTGRES_PASSWORD=nc)
|
||||
sleep 8 ;;
|
||||
*) fail "unknown database $DB" ;;
|
||||
esac
|
||||
docker run -d --name $N --network $NET "${env[@]}" -v "$SHARED:/mnt/shared" "$IMAGE" >/dev/null
|
||||
for i in $(seq 1 90); do occ status 2>/dev/null | grep -q "installed: true" && break; sleep 3; done
|
||||
docker exec $N chown www-data:www-data /var/www/html/custom_apps
|
||||
docker exec -i -u www-data $N tar xz -C /var/www/html/custom_apps < "$PKG"
|
||||
OLDVER=$(docker exec $N sed -n 's#.*<version>\(.*\)</version>.*#\1#p' /var/www/html/custom_apps/ng_backup/appinfo/info.xml)
|
||||
occ app:enable ng_backup >/dev/null || fail "enable $OLDVER"
|
||||
ok "NG Backup $OLDVER on $(occ -V) / $DB"
|
||||
|
||||
# The old version at work: key, kit, a location with a daily time, a user file, one backup
|
||||
docker exec $N sh -c "printf '%s\n' '$PASS' > /tmp/pass; chmod 644 /tmp/pass; mkdir -p /mnt/shared/loc; chown www-data /mnt/shared/loc"
|
||||
occ backup:key:init --passphrase-file=/tmp/pass >/dev/null || fail "key init"
|
||||
occ backup:key:kit --output=/tmp/kit.json >/dev/null
|
||||
CODE=$(occ backup:key:kit | php -r 'echo json_decode(stream_get_contents(STDIN), true)["confirmation_code"];')
|
||||
occ backup:key:confirm -u admin --code="$CODE" --confirm="Yes, I confirm" >/dev/null || fail "confirm"
|
||||
occ config:app:set ng_backup schedule_time --value=03:15 >/dev/null
|
||||
occ config:app:set ng_backup schedule_timezone --value=America/Toronto >/dev/null
|
||||
occ backup:target:add offsite local -a null::null -o datadir=/mnt/shared/loc --path=repo >/dev/null || fail "target add"
|
||||
docker exec -u www-data $N sh -c 'mkdir -p data/admin/files/Docs && head -c 3000000 /dev/urandom > data/admin/files/Docs/before.bin'
|
||||
occ files:scan admin >/dev/null
|
||||
SUM=$(docker exec $N sha256sum /var/www/html/data/admin/files/Docs/before.bin | cut -d' ' -f1)
|
||||
occ backup:run offsite >/dev/null || fail "backup with $OLDVER"
|
||||
OLDSNAP=$(occ backup:list offsite | grep -o '[0-9a-f]\{32\}' | head -1)
|
||||
[ -n "$OLDSNAP" ] || fail "no restore point with $OLDVER"
|
||||
ok "$OLDVER: key, kit, location at 03:15, restore point $OLDSNAP"
|
||||
|
||||
# The update: this working tree's runtime files in place of the old ones
|
||||
mkdir -p "$NEWAPP/ng_backup/js"
|
||||
cp -r "$APP/appinfo" "$APP/lib" "$APP/templates" "$APP/img" "$APP/css" "$NEWAPP/ng_backup/"
|
||||
cp "$APP"/js/*.mjs "$NEWAPP/ng_backup/js/"
|
||||
[ -n "$NEWVER" ] && sed -i "s#<version>.*</version>#<version>$NEWVER</version>#" "$NEWAPP/ng_backup/appinfo/info.xml"
|
||||
docker exec -u www-data $N rm -rf /var/www/html/custom_apps/ng_backup
|
||||
tar c -C "$NEWAPP" ng_backup | docker exec -i -u www-data $N tar x -C /var/www/html/custom_apps
|
||||
OUT=$(occ upgrade 2>&1) || { printf '%s\n' "$OUT"; fail "occ upgrade"; }
|
||||
VER=$(occ app:list --output=json | php -r '$a = json_decode(stream_get_contents(STDIN), true); echo $a["enabled"]["ng_backup"] ?? "";')
|
||||
[ -z "$NEWVER" ] || [ "$VER" = "$NEWVER" ] || fail "after the upgrade the app has version '$VER'"
|
||||
ok "occ upgrade to $VER"
|
||||
# Before anything else touches the key: only the migration can have marked it as used
|
||||
[ "$(occ config:app:get ng_backup master_key_used 2>/dev/null)" = 1 ] || [ "$(occ config:app:get ng_backup master_key_used 2>/dev/null)" = true ] || fail "the migration did not mark the key from before the upgrade as used"
|
||||
ok "the key from before the upgrade counts as used"
|
||||
|
||||
SCHED=$(occ backup:target:schedule offsite)
|
||||
printf '%s\n' "$SCHED" | grep -qx "offsite: 03:15 (America/Toronto)" || { printf '%s\n' "$SCHED"; fail "the location did not keep 03:15 as its schedule"; }
|
||||
occ backup:list offsite | grep -q "$OLDSNAP" || fail "the old restore point is not listed"
|
||||
occ backup:verify offsite "$OLDSNAP" >/dev/null || fail "the old restore point does not verify"
|
||||
occ backup:status >/dev/null || fail "backup:status"
|
||||
docker exec $N sh -c 'rm -rf /tmp/rst && mkdir /tmp/rst && chown www-data /tmp/rst'
|
||||
occ backup:restore offsite "$OLDSNAP" data/admin/files/Docs/before.bin --to-directory=/tmp/rst >/dev/null || fail "restore from the old restore point"
|
||||
GOT=$(docker exec $N sh -c 'find /tmp/rst -name before.bin -exec sha256sum {} \;' | cut -d' ' -f1)
|
||||
[ "$GOT" = "$SUM" ] || fail "the restored file differs"
|
||||
ok "after the upgrade: schedule kept, old restore point listed, verified, a file restored byte for byte"
|
||||
|
||||
docker exec -u www-data $N sh -c 'head -c 2000000 /dev/urandom > data/admin/files/Docs/after.bin'
|
||||
occ files:scan admin >/dev/null
|
||||
occ backup:run offsite >/dev/null || fail "backup after the upgrade"
|
||||
NEWSNAP=$(occ backup:list offsite | grep -o '[0-9a-f]\{32\}' | grep -v "$OLDSNAP" | head -1)
|
||||
[ -n "$NEWSNAP" ] || fail "no new restore point after the upgrade"
|
||||
SUM2=$(docker exec $N sha256sum /var/www/html/data/admin/files/Docs/after.bin | cut -d' ' -f1)
|
||||
docker exec $N sh -c 'rm -rf /tmp/rst && mkdir /tmp/rst && chown www-data /tmp/rst'
|
||||
occ backup:restore offsite "$NEWSNAP" data/admin/files/Docs/after.bin --to-directory=/tmp/rst >/dev/null || fail "restore from the new restore point"
|
||||
GOT=$(docker exec $N sh -c 'find /tmp/rst -name after.bin -exec sha256sum {} \;' | cut -d' ' -f1)
|
||||
[ "$GOT" = "$SUM2" ] || fail "the file restored from the new restore point differs"
|
||||
docker exec $N test -f /mnt/shared/loc/repo/settings || fail "the location got no settings file"
|
||||
OUT=$(occ backup:key:import /tmp/kit.json --passphrase-file=/tmp/pass --replace-unused-key 2>&1) && fail "a used key from before the upgrade was offered for replacement"
|
||||
printf '%s\n' "$OUT" | grep -q "never replaced" || { printf '%s\n' "$OUT"; fail "the refusal does not say the key is never replaced"; }
|
||||
ok "a new backup runs and restores, the location got its settings, the key counts as used"
|
||||
echo "UPGRADE $OLDVER -> ${VER:-working tree} PASS"
|
||||
Reference in New Issue
Block a user