Compare commits
15 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| be3746e1c3 | |||
| f5473495ce | |||
| 3694184674 | |||
| a0fc91a700 | |||
| a11e6c7254 | |||
| 6f1a37f4f4 | |||
| 5cde57bc1b | |||
| 3eec2e9554 | |||
| cdf0b70e25 | |||
| 66615c2a88 | |||
| 4847ce293d | |||
| 90dd97977a | |||
| 08534975d7 | |||
| 95e3dfcf6d | |||
| 7f4cae0e42 |
@@ -41,6 +41,20 @@ jobs:
|
|||||||
- run: composer install --no-interaction --no-progress
|
- run: composer install --no-interaction --no-progress
|
||||||
- run: composer psalm
|
- run: composer psalm
|
||||||
|
|
||||||
|
frontend:
|
||||||
|
# Builds the admin page, so a broken Vue component fails CI instead of showing up after a release.
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: '22'
|
||||||
|
cache: npm
|
||||||
|
- run: npm ci --no-audit --no-fund
|
||||||
|
- run: npm run build
|
||||||
|
|
||||||
integration:
|
integration:
|
||||||
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
|
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
|
||||||
# restore, retention, trash) on a fresh Nextcloud per version and database.
|
# restore, retention, trash) on a fresh Nextcloud per version and database.
|
||||||
|
|||||||
+72
-11
@@ -7,17 +7,72 @@ the app store accepts a `-beta` suffix, but Nextcloud servers then only offer th
|
|||||||
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
||||||
instead carried by the app description, the git tag, the release, and this file.
|
instead carried by the app description, the git tag, the release, and this file.
|
||||||
|
|
||||||
|
Each heading starts with that `<version>`, once: the app store shows a release the section(s)
|
||||||
|
whose heading starts with its `x.y.z` and ignores anything after it, so `## [1.0.0-rc.6]` would be
|
||||||
|
shown under 1.0.0. The git tag of a release candidate goes after the date.
|
||||||
|
|
||||||
|
## [1.0.1] - 2026-11-01
|
||||||
|
|
||||||
|
Fixes for what was reported after 1.0.0 (#22, #24) and small improvements. Nothing changes in
|
||||||
|
your backups or settings.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- `occ backup:user:list [location] [user]` shows the user exports on a location, with the manifest
|
||||||
|
path `backup:user:restore` expects. Before, that path was only printed once by
|
||||||
|
`backup:user:backup` (#22).
|
||||||
|
- A "Docs" tab on the admin page, with links to the how-to, the disaster recovery guide and the
|
||||||
|
roadmap. It also shows before the backup key is set up.
|
||||||
|
- A label for a backup started with "Back up now" (optional; `occ backup:run -l` already had one).
|
||||||
|
It shows with the restore point.
|
||||||
|
- An explanation in the trash of restore points of what "removed by retention policy" means: the
|
||||||
|
policy keeps the newest restore points and the last one of each recent day, week and month, and
|
||||||
|
moves the others to the trash, where they can be brought back until they are deleted
|
||||||
|
permanently. `occ backup:trash` says the same below its list.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- `occ backup:status`, `backup:list`, `backup:trash` and `backup:browse` show times in the time
|
||||||
|
zone of the daily schedule and name it, as the admin page shows them in the administrator's.
|
||||||
|
Before, they showed the server's UTC, which read as a wrong start time.
|
||||||
|
- The admin page no longer says NG Backup is in beta.
|
||||||
|
- `occ backup:user:restore` also takes a user id instead of the manifest path, when that user has
|
||||||
|
exactly one export on the location. With more, and for a manifest path that is not on the
|
||||||
|
location, the error lists that user's exports; for a user without exports it names the
|
||||||
|
expected form (#22). Before, `backup:user:restore <location> <uid>` only said "is not a
|
||||||
|
recorded user export for this location".
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- An SFTP location with an RSA key in OpenSSH format ("BEGIN OPENSSH PRIVATE KEY") stopped
|
||||||
|
working after an upgrade to Nextcloud 35 with "Unable to read key", and such a key could not be
|
||||||
|
added there at all (#24). The "Private key" field on the admin page is a single line, so a
|
||||||
|
pasted key lost its line breaks; Nextcloud 34 still read it, Nextcloud 35 reads keys with a
|
||||||
|
stricter library. The line breaks are now restored before the key is used, so a stored location
|
||||||
|
works again without re-entering it.
|
||||||
|
- A full restore onto SQLite could stop at once with "database is locked" when another process
|
||||||
|
wrote to the database just as the restore began (a background job that was already running).
|
||||||
|
The restore now takes the write lock first and waits for the other process instead. The
|
||||||
|
full-restore test on SQLite runs with a second process writing alongside.
|
||||||
|
- `occ backup:restore ... --to-directory` of a single file restored nothing ("Restored 0
|
||||||
|
files"); only a folder worked. A path to a file now restores that file.
|
||||||
|
- A restore that was stopped (a crash, a timeout, a restart) left its half-written temporary file
|
||||||
|
(`<name>.ngb-restore-<id>`) next to the file it was restoring, and a `files:scan` then showed it
|
||||||
|
in Files. A restore now removes such leftovers in each folder it writes to.
|
||||||
|
- The app store showed the notes of every release candidate under 1.0.0, because their headings
|
||||||
|
started with "1.0.0-rc". Each heading now starts with the app version of that release.
|
||||||
|
|
||||||
## [1.0.0] - 2026-10-29
|
## [1.0.0] - 2026-10-29
|
||||||
|
|
||||||
First stable release: the same code as 1.0.0-rc.6, which had no blocking problems reported in its
|
First stable release. NG Backup makes encrypted, incremental backups of your whole Nextcloud
|
||||||
week out (about 220 downloads). Only the version number and the README changed. The fixes for
|
(files, database and configuration) to S3, SFTP, WebDAV, SMB and other storage, and restores them
|
||||||
what was reported (#22, #24) follow in 1.0.1.
|
from the web interface or with occ. Disaster recovery has been tested on real installations; see
|
||||||
|
"Tested disaster recoveries" in the README, and read "Known limitations" before you rely on it.
|
||||||
|
|
||||||
## [1.0.0-rc.6] - 2026-10-24
|
The code is the same as 1.0.0-rc.6 (app version 0.9.12): updating from it changes nothing in your
|
||||||
|
backups or settings.
|
||||||
|
|
||||||
|
## [0.9.12] - 2026-10-24 (1.0.0-rc.6)
|
||||||
|
|
||||||
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
||||||
simulations and a user report. If no problems are reported for about a week, 1.0.0 will be this
|
simulations and a user report.
|
||||||
same code.
|
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
||||||
@@ -42,7 +97,7 @@ same code.
|
|||||||
- The full-restore integration test checks a 4-byte character in a table the restore creates, and
|
- The full-restore integration test checks a 4-byte character in a table the restore creates, and
|
||||||
retries app installs from the app store.
|
retries app installs from the app store.
|
||||||
|
|
||||||
## [1.0.0-rc.5] - 2026-10-22
|
## [0.9.11] - 2026-10-22 (1.0.0-rc.5)
|
||||||
|
|
||||||
Fifth release candidate for 1.0.0; the app version is 0.9.11. A full restore of 1 TB over SFTP
|
Fifth release candidate for 1.0.0; the app version is 0.9.11. A full restore of 1 TB over SFTP
|
||||||
stopped after 13 hours with "Blob not in index"; the backup itself was fine.
|
stopped after 13 hours with "Blob not in index"; the backup itself was fine.
|
||||||
@@ -75,7 +130,7 @@ stopped after 13 hours with "Blob not in index"; the backup itself was fine.
|
|||||||
- The extra connection a full restore reads ahead on is replaced when it breaks (up to 20 times),
|
- The extra connection a full restore reads ahead on is replaced when it breaks (up to 20 times),
|
||||||
instead of reading the rest of the restore the slow way.
|
instead of reading the rest of the restore the slow way.
|
||||||
|
|
||||||
## [1.0.0-rc.4] - 2026-10-19
|
## [0.9.10] - 2026-10-19 (1.0.0-rc.4)
|
||||||
|
|
||||||
Fourth release candidate for 1.0.0; the app version is 0.9.10. 1.0.0-rc.3 (0.9.9) was not
|
Fourth release candidate for 1.0.0; the app version is 0.9.10. 1.0.0-rc.3 (0.9.9) was not
|
||||||
released: its fixes are in here.
|
released: its fixes are in here.
|
||||||
@@ -99,7 +154,7 @@ released: its fixes are in here.
|
|||||||
- The app store showed old screenshots: it keeps a copy per URL, and the URLs always pointed at
|
- The app store showed old screenshots: it keeps a copy per URL, and the URLs always pointed at
|
||||||
`main`. They now point at the release's tag.
|
`main`. They now point at the release's tag.
|
||||||
|
|
||||||
## [1.0.0-rc.2] - 2026-10-18
|
## [0.9.8] - 2026-10-18 (1.0.0-rc.2)
|
||||||
|
|
||||||
Second release candidate for 1.0.0; the app version is 0.9.8. Fixes from a full disaster recovery
|
Second release candidate for 1.0.0; the app version is 0.9.8. Fixes from a full disaster recovery
|
||||||
test of a production instance (about 1 TB, Nextcloud All-in-One onto a plain Nextcloud with
|
test of a production instance (about 1 TB, Nextcloud All-in-One onto a plain Nextcloud with
|
||||||
@@ -140,7 +195,7 @@ PostgreSQL).
|
|||||||
- Removing a location left its cached index in the database (tens of thousands of rows for a large
|
- Removing a location left its cached index in the database (tens of thousands of rows for a large
|
||||||
backup). It is removed with the location now, and once for locations that are already gone.
|
backup). It is removed with the location now, and once for locations that are already gone.
|
||||||
|
|
||||||
## [1.0.0-rc.1] - 2026-10-17
|
## [0.9.7] - 2026-10-17 (1.0.0-rc.1)
|
||||||
|
|
||||||
Release candidate for 1.0.0; the app version is 0.9.7. Only bug fixes until 1.0.0. Tested on a
|
Release candidate for 1.0.0; the app version is 0.9.7. Only bug fixes until 1.0.0. Tested on a
|
||||||
production instance (about 1 TB, 150,000 files): full and incremental backups and a verify on
|
production instance (about 1 TB, 150,000 files): full and incremental backups and a verify on
|
||||||
@@ -419,7 +474,13 @@ and not implemented yet.
|
|||||||
- See the "Storage backend experiences" section of the README for provider-specific quirks found
|
- See the "Storage backend experiences" section of the README for provider-specific quirks found
|
||||||
so far (e.g. Hetzner Storage Box sub-accounts).
|
so far (e.g. Hetzner Storage Box sub-accounts).
|
||||||
|
|
||||||
[1.0.0-rc.1]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.1
|
[1.0.1]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.1
|
||||||
|
[1.0.0]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0
|
||||||
|
[0.9.12]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.6
|
||||||
|
[0.9.11]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.5
|
||||||
|
[0.9.10]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.4
|
||||||
|
[0.9.8]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.2
|
||||||
|
[0.9.7]: https://github.com/dillardblom/ng_backup/releases/tag/v1.0.0-rc.1
|
||||||
[0.9.6-beta.7]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.6-beta.7
|
[0.9.6-beta.7]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.6-beta.7
|
||||||
[0.9.5-beta.6]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.5-beta.6
|
[0.9.5-beta.6]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.5-beta.6
|
||||||
[0.9.4-beta.5]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.4-beta.5
|
[0.9.4-beta.5]: https://github.com/dillardblom/ng_backup/releases/tag/v0.9.4-beta.5
|
||||||
|
|||||||
@@ -392,10 +392,12 @@ in README "Known limitations":
|
|||||||
gids (SFTP, Storage Box) → rc4 uitbrengen en om tests vragen → zelf een volledige DR van een
|
gids (SFTP, Storage Box) → rc4 uitbrengen en om tests vragen → zelf een volledige DR van een
|
||||||
productie-instantie van ~250 GB via S3 en via SMB. Drie volledige DR-restores vóór 1.0.0.
|
productie-instantie van ~250 GB via S3 en via SMB. Drie volledige DR-restores vóór 1.0.0.
|
||||||
Problemen of meldingen van gebruikers: rc5.
|
Problemen of meldingen van gebruikers: rc5.
|
||||||
- **1.0.1:**
|
- **1.0.1** (klaar voor release 2026-11-01): alleen de kleine punten (besluit 2026-10-30): 4, 6–13. Punten 1, 2 en 5 gaan naar
|
||||||
1. (Naar rc4: herstelsnelheid over SFTP.) S3 leest al 16 tegelijk bij index en tree, maar een
|
1.0.2 (README-roadmap aangepast); punt 3 is een test, ingepland voor de week van 2 november,
|
||||||
|
niet aangekondigd en los van de release.
|
||||||
|
1. **Naar 1.0.2.** (Naar rc4: herstelsnelheid over SFTP.) S3 leest al 16 tegelijk bij index en tree, maar een
|
||||||
restore leest de packs nog één voor één: ook vooruitlezen via de S3-client.
|
restore leest de packs nog één voor één: ook vooruitlezen via de S3-client.
|
||||||
2. Achtergrondjobs zonder systeem-cron (issue #13; besluit 2026-10-19). De PR die dit
|
2. **Naar 1.0.2.** Achtergrondjobs zonder systeem-cron (issue #13; besluit 2026-10-19). De PR die dit
|
||||||
oplost zegt "Fixes #13" (bevestigd: #13 was de cron; besluit 2026-10-21):
|
oplost zegt "Fixes #13" (bevestigd: #13 was de cron; besluit 2026-10-21):
|
||||||
a. **Waarschuwen** in de beheerpagina en `backup:status`: modus uit `core backgroundjobs_mode`
|
a. **Waarschuwen** in de beheerpagina en `backup:status`: modus uit `core backgroundjobs_mode`
|
||||||
(`ajax`/`webcron`/`cron`) en de laatste run uit `core lastcron`. Niet op `cron`, of de
|
(`ajax`/`webcron`/`cron`) en de laatste run uit `core lastcron`. Niet op `cron`, of de
|
||||||
@@ -424,16 +426,54 @@ in README "Known limitations":
|
|||||||
4. **Label bij "Back up now"** (besluit 2026-10-19): een optioneel tekstveld naast de knop; de
|
4. **Label bij "Back up now"** (besluit 2026-10-19): een optioneel tekstveld naast de knop; de
|
||||||
tekst gaat als label van de snapshot mee (nu kan dat alleen met `occ backup:run -l`; de knop
|
tekst gaat als label van de snapshot mee (nu kan dat alleen met `occ backup:run -l`; de knop
|
||||||
stuurt een leeg label, geplande runs krijgen "scheduled"). Raakt alleen de UI en een
|
stuurt een leeg label, geplande runs krijgen "scheduled"). Raakt alleen de UI en een
|
||||||
bestaande parameter.
|
bestaande parameter. Ten hoogste 100 tekens — **klaar** (2026-10-30).
|
||||||
5. **`--salvage`** voor `backup:restore` en `backup:restore:full` (besluit 2026-10-22): terughalen
|
5. **`--salvage`** voor `backup:restore` en `backup:restore:full` (besluit 2026-10-22): terughalen
|
||||||
wat kan, een bestand dat niet te lezen of te ontsleutelen is overslaan en aan het eind
|
wat kan, een bestand dat niet te lezen of te ontsleutelen is overslaan en aan het eind
|
||||||
opsommen, in plaats van stoppen.
|
opsommen, in plaats van stoppen. **Naar 1.0.2** (besluit 2026-10-30).
|
||||||
6. **Tijdelijke bestanden van een afgebroken restore opruimen** (gevonden in de SMB-DR van
|
6. **Tijdelijke bestanden van een afgebroken restore opruimen** (gevonden in de SMB-DR van
|
||||||
rc6, 2026-10-24): de rc5-poging die stopte liet `VID_0009.mp4.ngb-restore-<id>` staan; de
|
rc6, 2026-10-24): de rc5-poging die stopte liet `VID_0009.mp4.ngb-restore-<id>` staan; de
|
||||||
geslaagde restore daarna haalde het niet weg, en `files:scan` maakte het zichtbaar in Files.
|
geslaagde restore daarna haalde het niet weg, en `files:scan` maakte het zichtbaar in Files.
|
||||||
Een restore ruimt `*.ngb-restore-*` in de mappen die hij terugzet op.
|
Een restore ruimt `*.ngb-restore-*` in de mappen die hij terugzet op (alleen bestanden met
|
||||||
|
precies dat achtervoegsel en 6 of 12 hextekens) — **klaar** (2026-10-30).
|
||||||
7. **Eén bestand terugzetten met `backup:restore`**: een pad naar een bestand zet 0 bestanden
|
7. **Eén bestand terugzetten met `backup:restore`**: een pad naar een bestand zet 0 bestanden
|
||||||
terug (er komt een `/` achter); nu werkt alleen een map.
|
terug (er komt een `/` achter); nu werkt alleen een map. Gold alleen voor `--to-directory`;
|
||||||
|
naar de bestanden van een gebruiker werkte een los bestand al — **klaar** (2026-10-30).
|
||||||
|
8. **Uitleg (i) bij de prullenbak van herstelpunten** (besluit 2026-10-25): bij "removed by
|
||||||
|
retention policy" een infoknop met wat het betekent: het bewaarbeleid houdt voor oudere dagen
|
||||||
|
één herstelpunt per dag (het laatste), weken en maanden; de rest gaat naar de prullenbak en
|
||||||
|
wordt pas na de ingestelde wachttijd (`delete_delay_days`, standaard 30) echt verwijderd, en is
|
||||||
|
tot die datum terug te zetten. Aanleiding: de beheerder zag "19-10 13:12 – after rc4 upgrade
|
||||||
|
removed by retention policy" en dacht dat er iets was weggegooid. `occ backup:trash` zegt
|
||||||
|
hetzelfde onder de lijst — **klaar** (2026-10-30).
|
||||||
|
9. **Tijdzone in `occ backup:status`**: de tabel toont de servertijd (UTC), de webinterface de
|
||||||
|
tijd van de beheerder; dat leest als een verkeerde starttijd. Zet de zone in de kolomkop
|
||||||
|
("started (UTC)") of toon de tijden in de zone van het schema. Gedaan: beide, en ook in
|
||||||
|
`backup:list`, `backup:trash` en `backup:browse` — **klaar** (2026-10-30).
|
||||||
|
10. **User-restore op gebruikersnaam** (issue #22, 2026-10-27): `backup:user:restore <locatie>
|
||||||
|
New_User` gaf alleen "is not a recorded user export"; het argument moet het manifestpad
|
||||||
|
zijn, dat alleen `backup:user:backup` eenmalig toonde. Nu: `backup:user:list` toont de
|
||||||
|
exports, een kale uid werkt bij precies één export, anders een lijst om uit te kiezen —
|
||||||
|
**klaar** (2026-10-27).
|
||||||
|
11. **Tabblad "Docs" in de beheerpagina** (besluit 2026-10-27): een derde knop naast "Backup and
|
||||||
|
restore" en "Settings", met links naar de HOWTO (`docs/HOWTO.md`), de DR-gids
|
||||||
|
(`docs/DISASTER-RECOVERY.md`) en de roadmap (README, `#roadmap`) op GitHub, elk met een
|
||||||
|
regel wat erin staat. Nu zijn ze alleen via de README te vinden. Zichtbaar ook zonder
|
||||||
|
sleutel, zodat de gids vindbaar is vóór de eerste installatie-stap. "NG Backup is in beta"
|
||||||
|
gaat uit de feedbackregel — **klaar** (2026-10-27).
|
||||||
|
12. **SFTP-sleutel zonder regeleinden** (issue #24, 2026-10-28): het veld "Private key" op de
|
||||||
|
beheerpagina is één regel, dus een geplakte sleutel verliest zijn regeleinden. NC34
|
||||||
|
(phpseclib 2) las dat nog, NC35 (phpseclib 3) weigert een OpenSSH-sleutel dan met "Unable
|
||||||
|
to read key". De regeleinden worden nu teruggezet vlak voor gebruik — **klaar** (2026-10-28).
|
||||||
|
13. **"database is locked" bij het begin van `backup:restore:full` op SQLite** (CI, 2026-10-26
|
||||||
|
bij #22 en 2026-10-29 op `main` na de merge van 1.0.0): de restore stopt meteen na het
|
||||||
|
aanzetten van de onderhoudsmodus. Niet altijd, dus een tweede proces dat op hetzelfde moment
|
||||||
|
de database gebruikt (cron, een webverzoek of de opstart van de container). Kan ook bij een
|
||||||
|
echte restore naar SQLite gebeuren. Oorzaak (2026-10-29): SQLite (WAL) begint een transactie
|
||||||
|
als lezer; schrijft een ander proces tussen ons eerste lezen en ons eerste schrijven, dan
|
||||||
|
faalt dat schrijven direct (geen wachttijd helpt). In CI was dat de installer, die
|
||||||
|
"installed" zet en daarna nog seconden doorschrijft. Nu neemt de restore eerst het
|
||||||
|
schrijfslot en wacht hij op de ander; de SQLite-test draait met een tweede schrijver
|
||||||
|
ernaast — **klaar** (2026-10-29).
|
||||||
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
|
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
|
||||||
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
|
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
|
||||||
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
|
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
|
||||||
@@ -470,6 +510,15 @@ in README "Known limitations":
|
|||||||
6. **Notitie achteraf bij een restorepunt**, te wijzigen in de lijst. Het label in de snapshot
|
6. **Notitie achteraf bij een restorepunt**, te wijzigen in de lijst. Het label in de snapshot
|
||||||
is versleuteld en ligt vast; de notitie apart op de locatie bewaren (bijv. in de catalogus),
|
is versleuteld en ligt vast; de notitie apart op de locatie bewaren (bijv. in de catalogus),
|
||||||
zodat een nieuwe server na een DR hem ook ziet.
|
zodat een nieuwe server na een DR hem ook ziet.
|
||||||
|
8. **Herstelpunt vastpinnen** (besluit 2026-10-25): een herstelpunt dat het bewaarbeleid
|
||||||
|
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
||||||
|
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
||||||
|
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
||||||
|
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
|
||||||
|
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
||||||
|
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
|
||||||
|
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
|
||||||
|
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
|
||||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||||
|
|||||||
@@ -7,11 +7,14 @@ installations and hosted Nextcloud. (Choosing SMB as a storage backend is the on
|
|||||||
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
||||||
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
||||||
|
|
||||||
**Status: 1.0.0, stable.** Disaster recovery has been tested on real installations (see "Tested
|
**Status: 1.0.1, stable.** Disaster recovery has been tested on real installations (see "Tested
|
||||||
disaster recoveries" below). See "Known limitations" below, `PLAN.md` for the design and
|
disaster recoveries" below). See "Known limitations" below, `PLAN.md` for the design and
|
||||||
`docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
|
`docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
|
||||||
`docs/DISASTER-RECOVERY.md`.
|
`docs/DISASTER-RECOVERY.md`.
|
||||||
|
|
||||||
|
If you like NG Backup, don't forget to give the app a star in the
|
||||||
|
[app store](https://apps.nextcloud.com/apps/ng_backup) and this repository a star on GitHub!
|
||||||
|
|
||||||
License: AGPL-3.0-or-later. Developed and sponsored by [Ensembia](https://www.ensembia.com).
|
License: AGPL-3.0-or-later. Developed and sponsored by [Ensembia](https://www.ensembia.com).
|
||||||
Need help restoring a backup after a crash? Ensembia offers paid restore assistance:
|
Need help restoring a backup after a crash? Ensembia offers paid restore assistance:
|
||||||
[www.ensembia.com](https://www.ensembia.com).
|
[www.ensembia.com](https://www.ensembia.com).
|
||||||
@@ -260,12 +263,14 @@ What is planned, without dates. Ideas and votes are welcome in the
|
|||||||
[issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
|
[issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
|
||||||
a 👍 on its issue helps decide what comes first.
|
a 👍 on its issue helps decide what comes first.
|
||||||
|
|
||||||
**1.0.1**
|
**1.0.2**
|
||||||
- A warning on the admin page and in `occ backup:status` when background jobs do not run on a
|
- A warning on the admin page and in `occ backup:status` when background jobs do not run on a
|
||||||
system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
|
system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
|
||||||
while the NG Backup page is open ([#13](https://github.com/dillardblom/ng_backup/issues/13)).
|
while the NG Backup page is open, and the database phase of a backup shows its progress
|
||||||
- An optional label for a backup started with "Back up now".
|
([#13](https://github.com/dillardblom/ng_backup/issues/13)).
|
||||||
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
|
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
|
||||||
|
- `--salvage` for a restore: restore what can be read, skip a file that cannot, and list those at
|
||||||
|
the end instead of stopping.
|
||||||
|
|
||||||
**1.1.0**
|
**1.1.0**
|
||||||
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
||||||
@@ -273,6 +278,8 @@ a 👍 on its issue helps decide what comes first.
|
|||||||
- A schedule per location, also "every N hours".
|
- A schedule per location, also "every N hours".
|
||||||
- Locations backed up one after the other instead of at the same time.
|
- Locations backed up one after the other instead of at the same time.
|
||||||
- A note on a restore point that you can change afterwards.
|
- A note on a restore point that you can change afterwards.
|
||||||
|
- Pinning a restore point, so the retention policy never removes it (for example the one from
|
||||||
|
before an upgrade).
|
||||||
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
||||||
Generator) while a backup runs.
|
Generator) while a backup runs.
|
||||||
- Faster backups: reading and uploading at the same time.
|
- Faster backups: reading and uploading at the same time.
|
||||||
@@ -330,4 +337,4 @@ On the backlog; know them before you rely on NG Backup:
|
|||||||
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
||||||
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
||||||
| `backup:key:import` | import a recovery kit on a fresh installation |
|
| `backup:key:import` | import a recovery kit on a fresh installation |
|
||||||
| `backup:user:backup`, `backup:user:restore` | one user via user_migration (asks before installing it) |
|
| `backup:user:backup`, `backup:user:list`, `backup:user:restore` | one user via user_migration (asks before installing it); list the exports |
|
||||||
|
|||||||
+8
-4
@@ -40,9 +40,12 @@ NG Backup needs Nextcloud's background jobs on system cron (a real cron line, no
|
|||||||
Read "Known limitations" in the README before you rely on it, and test a restore. Feedback is very
|
Read "Known limitations" in the README before you rely on it, and test a restore. Feedback is very
|
||||||
welcome: use the "Share your experience" link on the admin page.
|
welcome: use the "Share your experience" link on the admin page.
|
||||||
|
|
||||||
|
If you like NG Backup, don't forget to give the app a star in the app store and the
|
||||||
|
[repository](https://github.com/dillardblom/ng_backup) a star on GitHub!
|
||||||
|
|
||||||
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
||||||
you need help with restoring a backup after a crash.]]></description>
|
you need help with restoring a backup after a crash.]]></description>
|
||||||
<version>1.0.0</version>
|
<version>1.0.1</version>
|
||||||
<licence>agpl</licence>
|
<licence>agpl</licence>
|
||||||
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
||||||
<namespace>NgBackup</namespace>
|
<namespace>NgBackup</namespace>
|
||||||
@@ -53,9 +56,9 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<website>https://github.com/dillardblom/ng_backup</website>
|
<website>https://github.com/dillardblom/ng_backup</website>
|
||||||
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
||||||
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0/screenshots/1-backups.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.1/screenshots/1-backups.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0/screenshots/2-restore.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.1/screenshots/2-restore.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0/screenshots/3-settings.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.1/screenshots/3-settings.png</screenshot>
|
||||||
<dependencies>
|
<dependencies>
|
||||||
<php min-version="8.1"/>
|
<php min-version="8.1"/>
|
||||||
<lib>sodium</lib>
|
<lib>sodium</lib>
|
||||||
@@ -92,6 +95,7 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
||||||
<command>OCA\NgBackup\Command\BackupUser</command>
|
<command>OCA\NgBackup\Command\BackupUser</command>
|
||||||
<command>OCA\NgBackup\Command\RestoreUser</command>
|
<command>OCA\NgBackup\Command\RestoreUser</command>
|
||||||
|
<command>OCA\NgBackup\Command\ListUserExports</command>
|
||||||
<command>OCA\NgBackup\Command\KeyImport</command>
|
<command>OCA\NgBackup\Command\KeyImport</command>
|
||||||
<command>OCA\NgBackup\Command\RestoreFull</command>
|
<command>OCA\NgBackup\Command\RestoreFull</command>
|
||||||
<command>OCA\NgBackup\Command\Verify</command>
|
<command>OCA\NgBackup\Command\Verify</command>
|
||||||
|
|||||||
@@ -114,10 +114,16 @@ Needs the `user_migration` app. NG Backup asks before installing it:
|
|||||||
|
|
||||||
```
|
```
|
||||||
occ backup:user:backup offsite <uid>
|
occ backup:user:backup offsite <uid>
|
||||||
|
occ backup:user:list offsite [<uid>] # the exports and their manifest
|
||||||
occ backup:user:restore offsite users/<uid>/<exportId> # replace (default)
|
occ backup:user:restore offsite users/<uid>/<exportId> # replace (default)
|
||||||
occ backup:user:restore offsite users/<uid>/<exportId> --mode=as-backup
|
occ backup:user:restore offsite users/<uid>/<exportId> --mode=as-backup
|
||||||
```
|
```
|
||||||
|
|
||||||
|
- `backup:user:restore` needs the manifest path that `backup:user:backup` prints at the end
|
||||||
|
(`users/<uid>/<exportId>`); `backup:user:list` shows it again. A user id alone also works when
|
||||||
|
that user has exactly one export on the location. With more (a `replace` restore adds a safety
|
||||||
|
export), the command lists them and you pass one.
|
||||||
|
|
||||||
- If `user_migration` is missing, the command asks to install and enable it. For non-interactive
|
- If `user_migration` is missing, the command asks to install and enable it. For non-interactive
|
||||||
use, add `--install-user-migration` to approve that explicitly. The command stops after the
|
use, add `--install-user-migration` to approve that explicitly. The command stops after the
|
||||||
installation; run it once more.
|
installation; run it once more.
|
||||||
|
|||||||
@@ -72,6 +72,10 @@ final class ExternalStorageFactory {
|
|||||||
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
|
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (is_string($options['private_key'] ?? null)) {
|
||||||
|
$options['private_key'] = self::normalizePrivateKey($options['private_key']);
|
||||||
|
}
|
||||||
|
|
||||||
$config = new StorageConfig();
|
$config = new StorageConfig();
|
||||||
$config->setBackend($backend);
|
$config->setBackend($backend);
|
||||||
$config->setAuthMechanism($auth);
|
$config->setAuthMechanism($auth);
|
||||||
@@ -86,4 +90,22 @@ final class ExternalStorageFactory {
|
|||||||
$storage = $backend->wrapStorage($storage);
|
$storage = $backend->wrapStorage($storage);
|
||||||
return $auth->wrapStorage($storage);
|
return $auth->wrapStorage($storage);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An OpenSSH-format private key with its line breaks restored. Up to Nextcloud 34
|
||||||
|
* (phpseclib 2) a key whose newlines were lost or turned into spaces, e.g. on its way through
|
||||||
|
* a shell or a form, still loaded; Nextcloud 35 (phpseclib 3) refuses it with "Unable to read
|
||||||
|
* key" (#24). The key itself is not changed, so it is fixed when it is used, not when stored.
|
||||||
|
* Other formats (PEM, PKCS#8) still load as they are and are returned unchanged.
|
||||||
|
*/
|
||||||
|
public static function normalizePrivateKey(string $key): string {
|
||||||
|
if (!preg_match('/-----BEGIN OPENSSH PRIVATE KEY-----(.*?)-----END OPENSSH PRIVATE KEY-----/s', $key, $m)) {
|
||||||
|
return $key;
|
||||||
|
}
|
||||||
|
$body = (string)preg_replace('/\s+/', '', $m[1]);
|
||||||
|
if (!preg_match('#^[A-Za-z0-9+/]+=*$#', $body)) {
|
||||||
|
return $key;
|
||||||
|
}
|
||||||
|
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,7 +7,9 @@ declare(strict_types=1);
|
|||||||
|
|
||||||
namespace OCA\NgBackup\Command;
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Format;
|
||||||
use OCA\NgBackup\Service\RestoreService;
|
use OCA\NgBackup\Service\RestoreService;
|
||||||
|
use OCA\NgBackup\Service\Schedule;
|
||||||
use OCA\NgBackup\Service\TargetService;
|
use OCA\NgBackup\Service\TargetService;
|
||||||
use Symfony\Component\Console\Command\Command;
|
use Symfony\Component\Console\Command\Command;
|
||||||
use Symfony\Component\Console\Helper\Table;
|
use Symfony\Component\Console\Helper\Table;
|
||||||
@@ -19,6 +21,7 @@ final class Browse extends Command {
|
|||||||
public function __construct(
|
public function __construct(
|
||||||
private TargetService $targets,
|
private TargetService $targets,
|
||||||
private RestoreService $restore,
|
private RestoreService $restore,
|
||||||
|
private Schedule $schedule,
|
||||||
) {
|
) {
|
||||||
parent::__construct();
|
parent::__construct();
|
||||||
}
|
}
|
||||||
@@ -43,9 +46,10 @@ final class Browse extends Command {
|
|||||||
$output->writeln('Nothing here.');
|
$output->writeln('Nothing here.');
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
$zone = $this->schedule->get()['timezone'];
|
||||||
$rows = array_map(fn ($i) => [$i['type'] === 'dir' ? $i['name'] . '/' : $i['name'], $this->size($i['size']),
|
$rows = array_map(fn ($i) => [$i['type'] === 'dir' ? $i['name'] . '/' : $i['name'], $this->size($i['size']),
|
||||||
$i['type'] === 'dir' ? $i['files'] . ' files' : date('Y-m-d H:i', (int)$i['mtime'])], $items);
|
$i['type'] === 'dir' ? $i['files'] . ' files' : Format::time((int)$i['mtime'], $zone)], $items);
|
||||||
(new Table($output))->setHeaders(['name', 'size', 'files / modified'])->setRows($rows)->render();
|
(new Table($output))->setHeaders(['name', 'size', "files / modified ($zone)"])->setRows($rows)->render();
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -8,6 +8,8 @@ declare(strict_types=1);
|
|||||||
namespace OCA\NgBackup\Command;
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
use OCA\NgBackup\Db\SnapshotMapper;
|
use OCA\NgBackup\Db\SnapshotMapper;
|
||||||
|
use OCA\NgBackup\Format;
|
||||||
|
use OCA\NgBackup\Service\Schedule;
|
||||||
use OCA\NgBackup\Service\TargetService;
|
use OCA\NgBackup\Service\TargetService;
|
||||||
use Symfony\Component\Console\Command\Command;
|
use Symfony\Component\Console\Command\Command;
|
||||||
use Symfony\Component\Console\Helper\Table;
|
use Symfony\Component\Console\Helper\Table;
|
||||||
@@ -19,6 +21,7 @@ final class ListSnapshots extends Command {
|
|||||||
public function __construct(
|
public function __construct(
|
||||||
private TargetService $targets,
|
private TargetService $targets,
|
||||||
private SnapshotMapper $snapshots,
|
private SnapshotMapper $snapshots,
|
||||||
|
private Schedule $schedule,
|
||||||
) {
|
) {
|
||||||
parent::__construct();
|
parent::__construct();
|
||||||
}
|
}
|
||||||
@@ -32,10 +35,11 @@ final class ListSnapshots extends Command {
|
|||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
$name = $input->getArgument('target');
|
$name = $input->getArgument('target');
|
||||||
$targets = $name === null ? $this->targets->list() : [$this->targets->get($name)];
|
$targets = $name === null ? $this->targets->list() : [$this->targets->get($name)];
|
||||||
|
$zone = $this->schedule->get()['timezone'];
|
||||||
$rows = [];
|
$rows = [];
|
||||||
foreach ($targets as $t) {
|
foreach ($targets as $t) {
|
||||||
foreach ($this->snapshots->findForTarget($t->getId()) as $s) {
|
foreach ($this->snapshots->findForTarget($t->getId()) as $s) {
|
||||||
$rows[] = [$t->getName(), $s->getSnapshotId(), date('Y-m-d H:i', $s->getCreatedAt()), $s->getKind(),
|
$rows[] = [$t->getName(), $s->getSnapshotId(), Format::time($s->getCreatedAt(), $zone), $s->getKind(),
|
||||||
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getLabel() ?? ''];
|
$s->getFiles(), sprintf('%.1f GiB', ($s->getBytes() ?? 0) / 1073741824), $s->getLabel() ?? ''];
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -43,7 +47,7 @@ final class ListSnapshots extends Command {
|
|||||||
$output->writeln('No snapshots yet (occ backup:run).');
|
$output->writeln('No snapshots yet (occ backup:run).');
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
(new Table($output))->setHeaders(['location', 'snapshot', 'date', 'kind', 'files', 'size', 'label'])->setRows($rows)->render();
|
(new Table($output))->setHeaders(['location', 'snapshot', "date ($zone)", 'kind', 'files', 'size', 'label'])->setRows($rows)->render();
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Helper\Table;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class ListUserExports extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:user:list')
|
||||||
|
->setDescription('List the user exports (backup:user:backup) on a location')
|
||||||
|
->addArgument('target', InputArgument::OPTIONAL, 'Location name or id (default: all)')
|
||||||
|
->addArgument('user', InputArgument::OPTIONAL, 'Only the exports of this user id')
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
The manifest column is what occ backup:user:restore expects. A safety export made by a restore
|
||||||
|
with --mode=replace is listed here as well.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
occ backup:user:list offsite alice
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$name = $input->getArgument('target');
|
||||||
|
$uid = $input->getArgument('user');
|
||||||
|
$rows = [];
|
||||||
|
try {
|
||||||
|
$targets = $name === null ? $this->targets->list() : [$this->targets->get($name)];
|
||||||
|
foreach ($targets as $t) {
|
||||||
|
foreach ($this->targets->repository($t)->listUserExports($uid) as $path) {
|
||||||
|
$parts = explode('/', $path, 3);
|
||||||
|
$user = $parts[1] ?? '';
|
||||||
|
$exportId = $parts[2] ?? '';
|
||||||
|
$time = \DateTimeImmutable::createFromFormat('Ymd\THis\Z', substr($exportId, 0, 16), new \DateTimeZone('UTC'));
|
||||||
|
$rows[] = [$t->getName(), $user, $time !== false ? $time->format('Y-m-d H:i') : '', $path];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($rows === []) {
|
||||||
|
$output->writeln('No user exports yet (occ backup:user:backup).');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
(new Table($output))->setHeaders(['location', 'user', 'date (UTC)', 'manifest'])->setRows($rows)->render();
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -33,7 +33,7 @@ final class RestoreUser extends Command {
|
|||||||
$this->setName('backup:user:restore')
|
$this->setName('backup:user:restore')
|
||||||
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
||||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>)')
|
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>), or a user id with exactly one export')
|
||||||
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
||||||
->addOption('force', null, InputOption::VALUE_NONE, 'Replace an existing account without asking (required when not interactive)')
|
->addOption('force', null, InputOption::VALUE_NONE, 'Replace an existing account without asking (required when not interactive)')
|
||||||
->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)')
|
->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)')
|
||||||
@@ -45,16 +45,31 @@ Never imports over a live account. Modes:
|
|||||||
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
||||||
the new name, never the existing account
|
the new name, never the existing account
|
||||||
|
|
||||||
|
occ backup:user:list shows the exports on a location. A user id alone works when that user
|
||||||
|
has exactly one export there; with more, the command lists them and you pass one.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
||||||
|
occ backup:user:restore offsite alice
|
||||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
||||||
HELP);
|
HELP);
|
||||||
}
|
}
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
$mode = (string)$input->getOption('mode');
|
$mode = (string)$input->getOption('mode');
|
||||||
$uid = explode('/', (string)$input->getArgument('manifest'))[1] ?? '';
|
try {
|
||||||
if ($mode === UserRestoreService::MODE_REPLACE && $uid !== '' && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
$target = $this->targets->get($input->getArgument('target'));
|
||||||
|
$ref = (string)$input->getArgument('manifest');
|
||||||
|
$manifest = $this->targets->repository($target)->resolveUserExport($ref);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($manifest !== $ref) {
|
||||||
|
$output->writeln("Using $manifest.");
|
||||||
|
}
|
||||||
|
$uid = explode('/', $manifest)[1];
|
||||||
|
if ($mode === UserRestoreService::MODE_REPLACE && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
||||||
// Replace deletes the live account (after a safety export). Shares to others and data of
|
// Replace deletes the live account (after a safety export). Shares to others and data of
|
||||||
// apps without a user_migration migrator do not come back, so never do it unasked.
|
// apps without a user_migration migrator do not come back, so never do it unasked.
|
||||||
if (!$input->isInteractive()) {
|
if (!$input->isInteractive()) {
|
||||||
@@ -69,9 +84,8 @@ HELP);
|
|||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
$output->writeln('<comment>Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.</comment>');
|
$output->writeln('<comment>Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.</comment>');
|
||||||
$target = $this->targets->get($input->getArgument('target'));
|
|
||||||
$this->installer->ensureEnabled($input, $output);
|
$this->installer->ensureEnabled($input, $output);
|
||||||
$user = $this->userRestore->restoreUser($target, (string)$input->getArgument('manifest'), $mode);
|
$user = $this->userRestore->restoreUser($target, $manifest, $mode);
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
return 1;
|
return 1;
|
||||||
|
|||||||
@@ -8,7 +8,9 @@ declare(strict_types=1);
|
|||||||
namespace OCA\NgBackup\Command;
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
use OCA\NgBackup\Db\RunMapper;
|
use OCA\NgBackup\Db\RunMapper;
|
||||||
|
use OCA\NgBackup\Format;
|
||||||
use OCA\NgBackup\Service\KeyService;
|
use OCA\NgBackup\Service\KeyService;
|
||||||
|
use OCA\NgBackup\Service\Schedule;
|
||||||
use OCA\NgBackup\Service\TargetService;
|
use OCA\NgBackup\Service\TargetService;
|
||||||
use OCA\NgBackup\Service\VerifyService;
|
use OCA\NgBackup\Service\VerifyService;
|
||||||
use Symfony\Component\Console\Command\Command;
|
use Symfony\Component\Console\Command\Command;
|
||||||
@@ -26,6 +28,7 @@ final class Status extends Command {
|
|||||||
private TargetService $targets,
|
private TargetService $targets,
|
||||||
private RunMapper $runs,
|
private RunMapper $runs,
|
||||||
private VerifyService $verify,
|
private VerifyService $verify,
|
||||||
|
private Schedule $schedule,
|
||||||
) {
|
) {
|
||||||
parent::__construct();
|
parent::__construct();
|
||||||
}
|
}
|
||||||
@@ -36,12 +39,13 @@ final class Status extends Command {
|
|||||||
}
|
}
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$zone = $this->schedule->get()['timezone'];
|
||||||
if (!$this->keys->isInitialized()) {
|
if (!$this->keys->isInitialized()) {
|
||||||
$output->writeln('Key: <error>not created</error> (occ backup:key:init)');
|
$output->writeln('Key: <error>not created</error> (occ backup:key:init)');
|
||||||
} else {
|
} else {
|
||||||
$c = $this->keys->recoveryKitConfirmation();
|
$c = $this->keys->recoveryKitConfirmation();
|
||||||
$output->writeln('Key: ' . $this->keys->fingerprint() . ', recovery kit '
|
$output->writeln('Key: ' . $this->keys->fingerprint() . ', recovery kit '
|
||||||
. ($c ? 'confirmed by ' . $c['uid'] . ' on ' . date('Y-m-d H:i', $c['time']) : '<error>not confirmed</error> (occ backup:key:confirm)'));
|
. ($c ? 'confirmed by ' . $c['uid'] . ' on ' . Format::time($c['time'], $zone) . " ($zone)" : '<error>not confirmed</error> (occ backup:key:confirm)'));
|
||||||
}
|
}
|
||||||
$names = [];
|
$names = [];
|
||||||
foreach ($this->targets->list() as $t) {
|
foreach ($this->targets->list() as $t) {
|
||||||
@@ -54,12 +58,12 @@ final class Status extends Command {
|
|||||||
foreach ($shown as $r) {
|
foreach ($shown as $r) {
|
||||||
$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
|
$s = json_decode($r->getStats() ?? '{}', true)['files'] ?? [];
|
||||||
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() . ($r->getStatus() === 'running' ? ' (' . $r->getPhase() . self::percent($r) . ')' : ''),
|
$rows[] = [$r->getId(), $names[$r->getTargetId()] ?? '#' . $r->getTargetId(), $r->getKind(), $r->getStatus() . ($r->getStatus() === 'running' ? ' (' . $r->getPhase() . self::percent($r) . ')' : ''),
|
||||||
date('Y-m-d H:i', $r->getStartedAt()), $r->getFinishedAt() ? \OCA\NgBackup\Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
|
Format::time($r->getStartedAt(), $zone), $r->getFinishedAt() ? Format::duration($r->getFinishedAt() - $r->getStartedAt()) : '',
|
||||||
$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
|
$s['files'] ?? '', isset($s['bytes']) ? self::size((int)$s['bytes']) : '',
|
||||||
isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
|
isset($s['uploaded']) ? self::size((int)$s['uploaded']) : '', mb_strimwidth((string)$r->getError(), 0, 50, '…')];
|
||||||
}
|
}
|
||||||
if ($rows !== []) {
|
if ($rows !== []) {
|
||||||
(new Table($output))->setHeaders(['run', 'location', 'kind', 'status', 'started', 'took', 'files', 'size', 'uploaded', 'error'])->setRows($rows)->render();
|
(new Table($output))->setHeaders(['run', 'location', 'kind', 'status', "started ($zone)", 'took', 'files', 'size', 'uploaded', 'error'])->setRows($rows)->render();
|
||||||
}
|
}
|
||||||
$older = $all ? 0 : $this->runs->countAll() - count($shown);
|
$older = $all ? 0 : $this->runs->countAll() - count($shown);
|
||||||
if ($older > 0) {
|
if ($older > 0) {
|
||||||
|
|||||||
@@ -7,7 +7,9 @@ declare(strict_types=1);
|
|||||||
|
|
||||||
namespace OCA\NgBackup\Command;
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Format;
|
||||||
use OCA\NgBackup\Service\KeyService;
|
use OCA\NgBackup\Service\KeyService;
|
||||||
|
use OCA\NgBackup\Service\Schedule;
|
||||||
use OCA\NgBackup\Service\TargetService;
|
use OCA\NgBackup\Service\TargetService;
|
||||||
use Symfony\Component\Console\Command\Command;
|
use Symfony\Component\Console\Command\Command;
|
||||||
use Symfony\Component\Console\Helper\Table;
|
use Symfony\Component\Console\Helper\Table;
|
||||||
@@ -19,6 +21,7 @@ final class Trash extends Command {
|
|||||||
public function __construct(
|
public function __construct(
|
||||||
private TargetService $targets,
|
private TargetService $targets,
|
||||||
private KeyService $keys,
|
private KeyService $keys,
|
||||||
|
private Schedule $schedule,
|
||||||
) {
|
) {
|
||||||
parent::__construct();
|
parent::__construct();
|
||||||
}
|
}
|
||||||
@@ -32,11 +35,14 @@ final class Trash extends Command {
|
|||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
$name = $input->getArgument('target');
|
$name = $input->getArgument('target');
|
||||||
$delay = $this->keys->deleteDelayDays() * 86400;
|
$delay = $this->keys->deleteDelayDays() * 86400;
|
||||||
|
$zone = $this->schedule->get()['timezone'];
|
||||||
$rows = [];
|
$rows = [];
|
||||||
foreach ($name === null ? $this->targets->list() : [$this->targets->get($name)] as $t) {
|
foreach ($name === null ? $this->targets->list() : [$this->targets->get($name)] as $t) {
|
||||||
foreach ($this->targets->repository($t)->trash() as $id => $info) {
|
foreach ($this->targets->repository($t)->trash() as $id => $info) {
|
||||||
$rows[] = [$t->getName(), $id, (string)($info['time'] ?? ''), $info['label'], date('Y-m-d H:i', $info['forgottenAt']), $info['by'],
|
$time = $info['time'] ?? null;
|
||||||
date('Y-m-d H:i', $info['forgottenAt'] + $delay)];
|
$made = is_string($time) ? strtotime($time) : false;
|
||||||
|
$rows[] = [$t->getName(), $id, $made !== false ? Format::time($made, $zone) : '', $info['label'], Format::time($info['forgottenAt'], $zone), $info['by'],
|
||||||
|
Format::time($info['forgottenAt'] + $delay, $zone)];
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if ($rows === []) {
|
if ($rows === []) {
|
||||||
@@ -44,6 +50,7 @@ final class Trash extends Command {
|
|||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
(new Table($output))->setHeaders(['location', 'snapshot', 'made', 'label', 'removed', 'by', 'deleted after'])->setRows($rows)->render();
|
(new Table($output))->setHeaders(['location', 'snapshot', 'made', 'label', 'removed', 'by', 'deleted after'])->setRows($rows)->render();
|
||||||
|
$output->writeln("Times in $zone. The retention policy keeps the newest snapshots and the last of each recent day, week and month; it moves the others here, and they are deleted permanently after " . $this->keys->deleteDelayDays() . ' days.');
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+7
-1
@@ -169,7 +169,7 @@ final class DbDumper {
|
|||||||
$foreignKeys = [];
|
$foreignKeys = [];
|
||||||
$this->db->beginTransaction();
|
$this->db->beginTransaction();
|
||||||
try {
|
try {
|
||||||
$foreignKeys = $this->suspendForeignKeys();
|
$foreignKeys = $this->suspendForeignKeys(); // first: on SQLite it also takes the write lock
|
||||||
foreach ($manifest['tables'] as $table => $info) {
|
foreach ($manifest['tables'] as $table => $info) {
|
||||||
if (in_array($table, $report['missing'], true)) {
|
if (in_array($table, $report['missing'], true)) {
|
||||||
continue;
|
continue;
|
||||||
@@ -412,6 +412,12 @@ final class DbDumper {
|
|||||||
return $checks;
|
return $checks;
|
||||||
}
|
}
|
||||||
if ($platform instanceof \Doctrine\DBAL\Platforms\SqlitePlatform) {
|
if ($platform instanceof \Doctrine\DBAL\Platforms\SqlitePlatform) {
|
||||||
|
// SQLite (WAL) starts a transaction as a reader. When another process (a cron job, the
|
||||||
|
// installer still finishing) writes between our first read and our first write, that
|
||||||
|
// write fails at once with "database is locked": no waiting helps a reader whose view
|
||||||
|
// is out of date. Take the write lock first, so we wait for the other writer instead
|
||||||
|
// and it waits for us after that.
|
||||||
|
$this->db->executeStatement('DELETE FROM ' . $this->db->quoteIdentifier($this->prefix . 'appconfig') . ' WHERE 0 = 1');
|
||||||
$this->db->executeStatement('PRAGMA defer_foreign_keys = ON'); // checked at commit
|
$this->db->executeStatement('PRAGMA defer_foreign_keys = ON'); // checked at commit
|
||||||
}
|
}
|
||||||
return [];
|
return [];
|
||||||
|
|||||||
@@ -15,4 +15,14 @@ final class Format {
|
|||||||
$time = gmdate('H:i:s', $seconds % 86400);
|
$time = gmdate('H:i:s', $seconds % 86400);
|
||||||
return $days > 0 ? $days . 'd ' . $time : $time;
|
return $days > 0 ? $days . 'd ' . $time : $time;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* "2026-10-29 04:30" in $zone. Nextcloud runs PHP in UTC; occ shows times in the zone of the
|
||||||
|
* schedule, the administrator's, as the web interface does.
|
||||||
|
*
|
||||||
|
* @param non-empty-string $zone
|
||||||
|
*/
|
||||||
|
public static function time(int $timestamp, string $zone): string {
|
||||||
|
return (new \DateTimeImmutable('@' . $timestamp))->setTimezone(new \DateTimeZone($zone))->format('Y-m-d H:i');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -214,7 +214,8 @@ final class Repository {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Restore a snapshot (or only paths starting with $prefix) into a local directory.
|
* Restore a snapshot (or only paths starting with $prefix) into a local directory. A $prefix
|
||||||
|
* "dir/file/" also restores the file "dir/file" itself.
|
||||||
*
|
*
|
||||||
* @param bool $stripPrefix write "$target/<path without $prefix>" instead of
|
* @param bool $stripPrefix write "$target/<path without $prefix>" instead of
|
||||||
* "$target/<full logical path>" (disaster recovery: restoring e.g. "data/" or
|
* "$target/<full logical path>" (disaster recovery: restoring e.g. "data/" or
|
||||||
@@ -253,7 +254,7 @@ final class Repository {
|
|||||||
$plan = new ReadAheadPlan(rtrim($target, '/') . '/.ngb-restore-spool-' . bin2hex(random_bytes(4)));
|
$plan = new ReadAheadPlan(rtrim($target, '/') . '/.ngb-restore-spool-' . bin2hex(random_bytes(4)));
|
||||||
try {
|
try {
|
||||||
foreach ($this->streamEntries($snapshotId) as $entry) {
|
foreach ($this->streamEntries($snapshotId) as $entry) {
|
||||||
if ($prefix !== '' && !str_starts_with($entry['p'], $prefix)) {
|
if (!self::underPrefix($entry['p'], $prefix)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
foreach ($entry['b'] as $id) {
|
foreach ($entry['b'] as $id) {
|
||||||
@@ -286,11 +287,15 @@ final class Repository {
|
|||||||
|
|
||||||
private function restoreEntries(string $snapshotId, string $target, string $prefix, bool $stripPrefix, ?callable $heartbeat, ?ReadAheadPlan $plan): int {
|
private function restoreEntries(string $snapshotId, string $target, string $prefix, bool $stripPrefix, ?callable $heartbeat, ?ReadAheadPlan $plan): int {
|
||||||
$count = 0;
|
$count = 0;
|
||||||
|
$cleaned = [];
|
||||||
foreach ($this->streamEntries($snapshotId) as $entry) {
|
foreach ($this->streamEntries($snapshotId) as $entry) {
|
||||||
if ($prefix !== '' && !str_starts_with($entry['p'], $prefix)) {
|
if (!self::underPrefix($entry['p'], $prefix)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
$relPath = ($stripPrefix && $prefix !== '') ? substr($entry['p'], strlen(rtrim($prefix, '/')) + 1) : $entry['p'];
|
$relPath = $entry['p'];
|
||||||
|
if ($stripPrefix && $prefix !== '') {
|
||||||
|
$relPath = $entry['p'] === rtrim($prefix, '/') ? basename($entry['p']) : substr($entry['p'], strlen($prefix));
|
||||||
|
}
|
||||||
$dest = rtrim($target, '/') . '/' . $relPath;
|
$dest = rtrim($target, '/') . '/' . $relPath;
|
||||||
if (!is_dir(dirname($dest))) {
|
if (!is_dir(dirname($dest))) {
|
||||||
mkdir(dirname($dest), 0700, true);
|
mkdir(dirname($dest), 0700, true);
|
||||||
@@ -301,6 +306,10 @@ final class Repository {
|
|||||||
if ($base === false || $dir === false || ($dir !== $base && !str_starts_with($dir, $base . '/')) || is_link($dest)) {
|
if ($base === false || $dir === false || ($dir !== $base && !str_starts_with($dir, $base . '/')) || is_link($dest)) {
|
||||||
throw new RepositoryException('Refusing to write outside the restore directory: ' . $entry['p']);
|
throw new RepositoryException('Refusing to write outside the restore directory: ' . $entry['p']);
|
||||||
}
|
}
|
||||||
|
if (!isset($cleaned[$dir])) {
|
||||||
|
self::removeStaleTemporaries($dir);
|
||||||
|
$cleaned[$dir] = true;
|
||||||
|
}
|
||||||
// Write next to the destination and rename after all blobs verified: at most one file
|
// Write next to the destination and rename after all blobs verified: at most one file
|
||||||
// is ever present twice.
|
// is ever present twice.
|
||||||
$tmp = $dest . '.ngb-restore-' . bin2hex(random_bytes(6));
|
$tmp = $dest . '.ngb-restore-' . bin2hex(random_bytes(6));
|
||||||
@@ -331,6 +340,30 @@ final class Repository {
|
|||||||
return $count;
|
return $count;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** $prefix is "" (everything) or ends with "/"; the path without that "/" is a file to restore too. */
|
||||||
|
private static function underPrefix(string $path, string $prefix): bool {
|
||||||
|
return $prefix === '' || str_starts_with($path, $prefix) || $path === rtrim($prefix, '/');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Remove what a restore that was stopped left in a directory this restore writes to:
|
||||||
|
* "<name>.ngb-restore-<hex>" from here, ".<name>.ngb-restore-<hex>" from a restore through
|
||||||
|
* Nextcloud. A files:scan would otherwise make them show up in Files.
|
||||||
|
*/
|
||||||
|
public static function removeStaleTemporaries(string $dir): void {
|
||||||
|
$names = scandir($dir);
|
||||||
|
foreach ($names === false ? [] : $names as $name) {
|
||||||
|
$path = $dir . '/' . $name;
|
||||||
|
if (self::isRestoreTemporary($name) && is_file($path) && !is_link($path)) {
|
||||||
|
@unlink($path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static function isRestoreTemporary(string $name): bool {
|
||||||
|
return preg_match('/.\.ngb-restore-[0-9a-f]{6}(?:[0-9a-f]{6})?$/', $name) === 1;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Start writing a generated stream (e.g. a table dump) as blobs. Call flushPacks() when the
|
* Start writing a generated stream (e.g. a table dump) as blobs. Call flushPacks() when the
|
||||||
* run is done, so the last pack is uploaded before anything references it.
|
* run is done, so the last pack is uploaded before anything references it.
|
||||||
@@ -561,6 +594,31 @@ final class Repository {
|
|||||||
return isset($this->catalog->userExports[$manifestPath]);
|
return isset($this->catalog->userExports[$manifestPath]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The recorded export $ref names: a manifest path (users/<uid>/<exportId>) as is, or a bare
|
||||||
|
* user id (also "users/<uid>") when that user has exactly one export here. Anything else
|
||||||
|
* throws, with the exports to choose from in the message.
|
||||||
|
*/
|
||||||
|
public function resolveUserExport(string $ref): string {
|
||||||
|
if ($this->isRecordedUserExport($ref)) {
|
||||||
|
return $ref;
|
||||||
|
}
|
||||||
|
$rest = trim($ref, '/');
|
||||||
|
if (str_starts_with($rest, 'users/')) {
|
||||||
|
$rest = substr($rest, strlen('users/'));
|
||||||
|
}
|
||||||
|
$uid = explode('/', $rest)[0];
|
||||||
|
$exports = $uid !== '' ? $this->listUserExports($uid) : [];
|
||||||
|
if (!str_contains($rest, '/') && count($exports) === 1) {
|
||||||
|
return $exports[0];
|
||||||
|
}
|
||||||
|
if ($exports === []) {
|
||||||
|
throw new RepositoryException("$ref is not a recorded user export for this location, and $uid has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports");
|
||||||
|
}
|
||||||
|
$which = str_contains($rest, '/') ? "$ref is not a recorded user export for this location" : "$uid has " . count($exports) . ' exports here';
|
||||||
|
throw new RepositoryException("$which; pass one of these (newest first):\n " . implode("\n ", $exports));
|
||||||
|
}
|
||||||
|
|
||||||
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
||||||
public function forgetUserExport(string $manifestPath): void {
|
public function forgetUserExport(string $manifestPath): void {
|
||||||
if (!isset($this->catalog->userExports[$manifestPath])) {
|
if (!isset($this->catalog->userExports[$manifestPath])) {
|
||||||
|
|||||||
@@ -34,6 +34,8 @@ use Psr\Log\LoggerInterface;
|
|||||||
*/
|
*/
|
||||||
final class BackupService {
|
final class BackupService {
|
||||||
public const KIND_FULL = 'full';
|
public const KIND_FULL = 'full';
|
||||||
|
/** Characters of a snapshot label (the column holds 255) */
|
||||||
|
public const LABEL_MAX = 100;
|
||||||
|
|
||||||
/** @var \WeakMap<Repository, string> the location settings each kept repository was opened with */
|
/** @var \WeakMap<Repository, string> the location settings each kept repository was opened with */
|
||||||
private \WeakMap $openedWith;
|
private \WeakMap $openedWith;
|
||||||
@@ -57,6 +59,10 @@ final class BackupService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public function start(Target $target, string $label = ''): Run {
|
public function start(Target $target, string $label = ''): Run {
|
||||||
|
$label = trim($label);
|
||||||
|
if (mb_strlen($label) > self::LABEL_MAX) {
|
||||||
|
throw new \InvalidArgumentException('A label has at most ' . self::LABEL_MAX . ' characters');
|
||||||
|
}
|
||||||
if (!$this->keys->everConfirmed()) {
|
if (!$this->keys->everConfirmed()) {
|
||||||
throw new \RuntimeException('Confirm the recovery kit first (occ backup:key:confirm)');
|
throw new \RuntimeException('Confirm the recovery kit first (occ backup:key:confirm)');
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -33,6 +33,9 @@ final class RestoreService {
|
|||||||
public const MODE_MERGE = 'merge';
|
public const MODE_MERGE = 'merge';
|
||||||
public const MODE_REPLACE = 'replace';
|
public const MODE_REPLACE = 'replace';
|
||||||
|
|
||||||
|
/** @var array<string, true> folders of the current restore already cleared of stale temporaries */
|
||||||
|
private array $cleaned = [];
|
||||||
|
|
||||||
public function __construct(
|
public function __construct(
|
||||||
private TargetService $targets,
|
private TargetService $targets,
|
||||||
private IRootFolder $root,
|
private IRootFolder $root,
|
||||||
@@ -133,6 +136,7 @@ final class RestoreService {
|
|||||||
$dest = $this->ensureFolder($userFolder, $parentRel);
|
$dest = $this->ensureFolder($userFolder, $parentRel);
|
||||||
$names = $single ? [basename($rel) => $entries['']] : $entries;
|
$names = $single ? [basename($rel) => $entries['']] : $entries;
|
||||||
|
|
||||||
|
$this->cleaned = [];
|
||||||
$restored = 0;
|
$restored = 0;
|
||||||
foreach ($names as $relPath => $entry) {
|
foreach ($names as $relPath => $entry) {
|
||||||
$this->writeFile($repo, $dest, (string)$relPath, $entry);
|
$this->writeFile($repo, $dest, (string)$relPath, $entry);
|
||||||
@@ -178,6 +182,10 @@ final class RestoreService {
|
|||||||
private function writeFile(Repository $repo, Folder $dest, string $relPath, array $entry): void {
|
private function writeFile(Repository $repo, Folder $dest, string $relPath, array $entry): void {
|
||||||
$dir = dirname($relPath);
|
$dir = dirname($relPath);
|
||||||
$folder = $dir === '.' ? $dest : $this->ensureFolder($dest, $dir);
|
$folder = $dir === '.' ? $dest : $this->ensureFolder($dest, $dir);
|
||||||
|
if (!isset($this->cleaned[$folder->getPath()])) {
|
||||||
|
$this->removeStaleTemporaries($folder);
|
||||||
|
$this->cleaned[$folder->getPath()] = true;
|
||||||
|
}
|
||||||
$name = basename($relPath);
|
$name = basename($relPath);
|
||||||
// Stage under a temporary name; only rename into place after every blob was verified.
|
// Stage under a temporary name; only rename into place after every blob was verified.
|
||||||
$tmpName = '.' . $name . '.ngb-restore-' . bin2hex(random_bytes(3));
|
$tmpName = '.' . $name . '.ngb-restore-' . bin2hex(random_bytes(3));
|
||||||
@@ -223,6 +231,19 @@ final class RestoreService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** See Repository::removeStaleTemporaries(): what a restore that was stopped left in this folder. */
|
||||||
|
private function removeStaleTemporaries(Folder $folder): void {
|
||||||
|
foreach ($folder->getDirectoryListing() as $node) {
|
||||||
|
if ($node instanceof File && Repository::isRestoreTemporary($node->getName())) {
|
||||||
|
try {
|
||||||
|
$node->delete();
|
||||||
|
} catch (\Throwable) {
|
||||||
|
// in the way of nothing: the new temporary gets a name of its own
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private function trashExtras(Folder $folder, string $prefix, array $keep): int {
|
private function trashExtras(Folder $folder, string $prefix, array $keep): int {
|
||||||
$count = 0;
|
$count = 0;
|
||||||
foreach ($folder->getDirectoryListing() as $node) {
|
foreach ($folder->getDirectoryListing() as $node) {
|
||||||
|
|||||||
@@ -110,11 +110,6 @@
|
|||||||
<code><![CDATA[ask]]></code>
|
<code><![CDATA[ask]]></code>
|
||||||
</UndefinedInterfaceMethod>
|
</UndefinedInterfaceMethod>
|
||||||
</file>
|
</file>
|
||||||
<file src="lib/Command/Trash.php">
|
|
||||||
<RedundantCastGivenDocblockType>
|
|
||||||
<code><![CDATA[(string)($info['time'] ?? '')]]></code>
|
|
||||||
</RedundantCastGivenDocblockType>
|
|
||||||
</file>
|
|
||||||
<file src="lib/Controller/AdminApiController.php">
|
<file src="lib/Controller/AdminApiController.php">
|
||||||
<InvalidOperand>
|
<InvalidOperand>
|
||||||
<code><![CDATA[$maxGb * 1073741824]]></code>
|
<code><![CDATA[$maxGb * 1073741824]]></code>
|
||||||
|
|||||||
+41
-2
@@ -7,7 +7,7 @@
|
|||||||
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
|
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
|
||||||
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
|
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
|
||||||
<p class="ngb-feedback">
|
<p class="ngb-feedback">
|
||||||
{{ t('ng_backup', 'NG Backup is in beta. We would like to hear how it works for you, also when everything went fine.') }}
|
{{ t('ng_backup', 'We would like to hear how NG Backup works for you, also when everything went fine.') }}
|
||||||
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
|
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
|
||||||
</p>
|
</p>
|
||||||
<NcLoadingIcon v-if="!status" :size="32" />
|
<NcLoadingIcon v-if="!status" :size="32" />
|
||||||
@@ -18,9 +18,18 @@
|
|||||||
<div class="ngb-tabs">
|
<div class="ngb-tabs">
|
||||||
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
|
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
|
||||||
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
|
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
|
||||||
|
<NcButton :variant="tab === 'docs' ? 'primary' : 'secondary'" @click="tab = 'docs'">{{ t('ng_backup', 'Docs') }}</NcButton>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<template v-if="tab === 'settings'">
|
<NcSettingsSection v-if="tab === 'docs'" :name="t('ng_backup', 'Documentation')">
|
||||||
|
<ul class="ngb-docs">
|
||||||
|
<li v-for="doc in docs" :key="doc.url">
|
||||||
|
<a :href="doc.url" target="_blank" rel="noopener noreferrer">{{ doc.name }} ↗</a>
|
||||||
|
<span>{{ doc.description }}</span>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</NcSettingsSection>
|
||||||
|
<template v-else-if="tab === 'settings'">
|
||||||
<KeySetup :status="status" @changed="refresh" />
|
<KeySetup :status="status" @changed="refresh" />
|
||||||
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
|
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
|
||||||
</template>
|
</template>
|
||||||
@@ -52,6 +61,25 @@ import RestoreBrowser from './components/RestoreBrowser.vue'
|
|||||||
// Opens the experience report form in .github/ISSUE_TEMPLATE
|
// Opens the experience report form in .github/ISSUE_TEMPLATE
|
||||||
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
|
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
|
||||||
|
|
||||||
|
const repoUrl = 'https://github.com/dillardblom/ng_backup'
|
||||||
|
const docs = [
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'How-to'),
|
||||||
|
description: t('ng_backup', 'Step by step: from first install to a tested restore.'),
|
||||||
|
url: repoUrl + '/blob/main/docs/HOWTO.md',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'Disaster recovery'),
|
||||||
|
description: t('ng_backup', 'Your server is gone: get Nextcloud back on a new server from a backup location.'),
|
||||||
|
url: repoUrl + '/blob/main/docs/DISASTER-RECOVERY.md',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'Roadmap'),
|
||||||
|
description: t('ng_backup', 'What is planned for the next versions.'),
|
||||||
|
url: repoUrl + '#roadmap',
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
const status = ref(null)
|
const status = ref(null)
|
||||||
const allRuns = ref(false) // runs older than the last days too
|
const allRuns = ref(false) // runs older than the last days too
|
||||||
const error = ref('')
|
const error = ref('')
|
||||||
@@ -102,6 +130,17 @@ onBeforeUnmount(() => clearTimeout(timer))
|
|||||||
color: var(--color-primary-element);
|
color: var(--color-primary-element);
|
||||||
text-decoration: underline;
|
text-decoration: underline;
|
||||||
}
|
}
|
||||||
|
.ngb-docs li {
|
||||||
|
margin-bottom: 8px;
|
||||||
|
}
|
||||||
|
.ngb-docs a {
|
||||||
|
color: var(--color-primary-element);
|
||||||
|
text-decoration: underline;
|
||||||
|
margin-right: 8px;
|
||||||
|
}
|
||||||
|
.ngb-docs span {
|
||||||
|
color: var(--color-text-maxcontrast);
|
||||||
|
}
|
||||||
.ngb-tabs {
|
.ngb-tabs {
|
||||||
display: flex;
|
display: flex;
|
||||||
gap: 8px;
|
gap: 8px;
|
||||||
|
|||||||
@@ -10,6 +10,8 @@
|
|||||||
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||||
</div>
|
</div>
|
||||||
<div class="ngb-row">
|
<div class="ngb-row">
|
||||||
|
<NcTextField v-model="label" class="ngb-label" :maxlength="100" :label="t('ng_backup', 'Label (optional)')"
|
||||||
|
:placeholder="t('ng_backup', 'e.g. before the upgrade to Nextcloud 35')" />
|
||||||
<NcButton v-for="target in status.targets" :key="target.id" :disabled="isRunning(target.id)" @click="start(target)">
|
<NcButton v-for="target in status.targets" :key="target.id" :disabled="isRunning(target.id)" @click="start(target)">
|
||||||
{{ isRunning(target.id) ? t('ng_backup', 'Backup to {name} running …', { name: target.name }) : t('ng_backup', 'Back up now to {name}', { name: target.name }) }}
|
{{ isRunning(target.id) ? t('ng_backup', 'Backup to {name} running …', { name: target.name }) : t('ng_backup', 'Back up now to {name}', { name: target.name }) }}
|
||||||
</NcButton>
|
</NcButton>
|
||||||
@@ -24,9 +26,23 @@
|
|||||||
<li v-for="item in trash[target.id]" :key="item.id">
|
<li v-for="item in trash[target.id]" :key="item.id">
|
||||||
{{ item.time ? formatDateTime(item.time) : item.id }}{{ item.label ? ' – ' + item.label : '' }}
|
{{ item.time ? formatDateTime(item.time) : item.id }}{{ item.label ? ' – ' + item.label : '' }}
|
||||||
<span class="ngb-muted">{{ t('ng_backup', 'removed by {by}, deleted after {date}', { by: item.by, date: formatDate(item.deleteAfter) }) }}</span>
|
<span class="ngb-muted">{{ t('ng_backup', 'removed by {by}, deleted after {date}', { by: item.by, date: formatDate(item.deleteAfter) }) }}</span>
|
||||||
|
<button v-if="item.by === RETENTION"
|
||||||
|
class="ngb-info"
|
||||||
|
:aria-expanded="retentionInfo ? 'true' : 'false'"
|
||||||
|
:aria-label="t('ng_backup', 'What does this mean?')"
|
||||||
|
:title="t('ng_backup', 'What does this mean?')"
|
||||||
|
@click="retentionInfo = !retentionInfo">
|
||||||
|
ⓘ
|
||||||
|
</button>
|
||||||
<NcButton variant="tertiary" @click="untrash(target, item)">{{ t('ng_backup', 'Restore') }}</NcButton>
|
<NcButton variant="tertiary" @click="untrash(target, item)">{{ t('ng_backup', 'Restore') }}</NcButton>
|
||||||
</li>
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
<NcNoteCard v-if="retentionInfo && trash[target.id].some(item => item.by === RETENTION)" type="info">
|
||||||
|
{{ n('ng_backup',
|
||||||
|
'The retention policy keeps the newest restore points, and for older days, weeks and months only the last restore point of each. It moves the others to this trash, where they stay for %n day before they are deleted permanently. Until then you can bring one back with "Restore". Nothing was lost by accident.',
|
||||||
|
'The retention policy keeps the newest restore points, and for older days, weeks and months only the last restore point of each. It moves the others to this trash, where they stay for %n days before they are deleted permanently. Until then you can bring one back with "Restore". Nothing was lost by accident.',
|
||||||
|
status.key.deleteDelayDays) }}
|
||||||
|
</NcNoteCard>
|
||||||
</div>
|
</div>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
@@ -93,6 +109,10 @@ const schedule = ref(shown(props.status.schedule))
|
|||||||
const error = ref('')
|
const error = ref('')
|
||||||
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
|
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
|
||||||
|
|
||||||
|
const label = ref('')
|
||||||
|
// "by" of a restore point the weekly cleanup removed (PruneService)
|
||||||
|
const RETENTION = 'retention policy'
|
||||||
|
const retentionInfo = ref(false)
|
||||||
const trash = reactive({})
|
const trash = reactive({})
|
||||||
async function loadTrash() {
|
async function loadTrash() {
|
||||||
for (const target of props.status.targets) {
|
for (const target of props.status.targets) {
|
||||||
@@ -125,7 +145,8 @@ const targetName = (id) => props.status.targets.find(tg => tg.id === id)?.name ?
|
|||||||
async function start(target) {
|
async function start(target) {
|
||||||
error.value = ''
|
error.value = ''
|
||||||
try {
|
try {
|
||||||
await api.startBackup(target.id, '')
|
await api.startBackup(target.id, label.value.trim())
|
||||||
|
label.value = ''
|
||||||
emit('changed')
|
emit('changed')
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
error.value = e.message
|
error.value = e.message
|
||||||
@@ -146,6 +167,8 @@ async function saveSchedule() {
|
|||||||
<style scoped>
|
<style scoped>
|
||||||
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
.ngb-row { display: flex; gap: 8px; align-items: flex-end; flex-wrap: wrap; margin-bottom: 12px; }
|
||||||
.ngb-time { max-width: 260px; }
|
.ngb-time { max-width: 260px; }
|
||||||
|
.ngb-label { max-width: 360px; }
|
||||||
|
.ngb-info { background: none; border: none; padding: 0 4px; min-height: auto; cursor: pointer; color: var(--color-primary-element); }
|
||||||
.ngb-table-wrap { overflow-x: auto; }
|
.ngb-table-wrap { overflow-x: auto; }
|
||||||
.ngb-table { width: 100%; border-collapse: collapse; }
|
.ngb-table { width: 100%; border-collapse: collapse; }
|
||||||
.ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); }
|
.ngb-table th, .ngb-table td { text-align: start; padding: 4px 8px; border-bottom: 1px solid var(--color-border); }
|
||||||
|
|||||||
@@ -0,0 +1,35 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Backend;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\ExternalStorageFactory;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** ExternalStorageFactory::normalizePrivateKey(): keys that phpseclib 2 read, for phpseclib 3 (#24). */
|
||||||
|
class ExternalStorageFactoryTest extends TestCase {
|
||||||
|
/** An OpenSSH-format key laid out as ssh-keygen writes it; the body is random, not a real key. */
|
||||||
|
private static function key(): string {
|
||||||
|
$body = base64_encode(random_bytes(1000));
|
||||||
|
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRestoresLostLineBreaks(): void {
|
||||||
|
$key = self::key();
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey($key));
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", ' ', $key)), 'newlines as spaces');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", '', $key)), 'no newlines');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", "\r\n", $key)), 'CRLF');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(rtrim($key)), 'no trailing newline');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(preg_replace('/^/m', ' ', $key)), 'indented');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testLeavesOtherFormatsAlone(): void {
|
||||||
|
$pem = "-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\nDEK-Info: DES-EDE3-CBC,FF44E0A94036766C\n\nAAAA\n-----END RSA PRIVATE KEY-----\n";
|
||||||
|
$this->assertSame($pem, ExternalStorageFactory::normalizePrivateKey($pem));
|
||||||
|
$this->assertSame('not a key', ExternalStorageFactory::normalizePrivateKey('not a key'));
|
||||||
|
$odd = "-----BEGIN OPENSSH PRIVATE KEY-----\\nAAAA\\n-----END OPENSSH PRIVATE KEY-----";
|
||||||
|
$this->assertSame($odd, ExternalStorageFactory::normalizePrivateKey($odd), 'not base64 inside: unchanged');
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -14,4 +14,11 @@ class FormatTest extends TestCase {
|
|||||||
$this->assertSame('1d 00:00:00', Format::duration(86400));
|
$this->assertSame('1d 00:00:00', Format::duration(86400));
|
||||||
$this->assertSame('1d 12:10:13', Format::duration(86400 + 12 * 3600 + 10 * 60 + 13));
|
$this->assertSame('1d 12:10:13', Format::duration(86400 + 12 * 3600 + 10 * 60 + 13));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testTimeInZone(): void {
|
||||||
|
$t = (int)strtotime('2026-10-19 13:12:00 UTC');
|
||||||
|
$this->assertSame('2026-10-19 13:12', Format::time($t, 'UTC'));
|
||||||
|
$this->assertSame('2026-10-19 15:12', Format::time($t, 'Europe/Berlin')); // summer time
|
||||||
|
$this->assertSame('2026-12-01 01:00', Format::time((int)strtotime('2026-12-01 00:00:00 UTC'), 'Europe/Amsterdam'));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -87,6 +87,46 @@ class RepositoryTest extends TestCase {
|
|||||||
$this->assertSame(10, $repo->restore($r['snapshot'], $target, 'docs/d1/'));
|
$this->assertSame(10, $repo->restore($r['snapshot'], $target, 'docs/d1/'));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testRestoreOfOneFile(): void {
|
||||||
|
$repo = Repository::init($this->backend, 'pw');
|
||||||
|
$r = $repo->backupDirectory($this->src);
|
||||||
|
$target = $this->tempDir();
|
||||||
|
// occ backup:restore --to-directory passes the path with a "/" appended
|
||||||
|
$this->assertSame(1, $repo->restore($r['snapshot'], $target, 'docs/d1/note-1.txt/'));
|
||||||
|
$this->assertSame(file_get_contents($this->src . '/docs/d1/note-1.txt'), file_get_contents($target . '/docs/d1/note-1.txt'));
|
||||||
|
$this->assertSame(0, $repo->restore($r['snapshot'], $this->tempDir(), 'docs/d1/note-1/'), 'a prefix of a name is not that file');
|
||||||
|
|
||||||
|
$stripped = $this->tempDir();
|
||||||
|
$this->assertSame(1, $repo->restore($r['snapshot'], $stripped, 'docs/d1/note-1.txt/', true));
|
||||||
|
$this->assertFileExists($stripped . '/note-1.txt');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRestoreRemovesTemporariesOfAStoppedRestore(): void {
|
||||||
|
$repo = Repository::init($this->backend, 'pw');
|
||||||
|
$r = $repo->backupDirectory($this->src);
|
||||||
|
$target = $this->tempDir();
|
||||||
|
mkdir($target . '/docs/d1', 0700, true);
|
||||||
|
mkdir($target . '/elsewhere', 0700, true);
|
||||||
|
$stale = [
|
||||||
|
$target . '/docs/d1/note-1.txt.ngb-restore-0123456789ab', // from a restore to a directory
|
||||||
|
$target . '/docs/d1/.note-4.txt.ngb-restore-a1b2c3', // from a restore through Nextcloud
|
||||||
|
];
|
||||||
|
$kept = [
|
||||||
|
$target . '/docs/d1/notes.ngb-restore-draft.txt', // not a temporary of ours
|
||||||
|
$target . '/elsewhere/x.ngb-restore-0123456789ab', // not a directory this restore writes to
|
||||||
|
];
|
||||||
|
foreach ([...$stale, ...$kept] as $f) {
|
||||||
|
file_put_contents($f, 'partial');
|
||||||
|
}
|
||||||
|
$repo->restore($r['snapshot'], $target, 'docs/');
|
||||||
|
foreach ($stale as $f) {
|
||||||
|
$this->assertFileDoesNotExist($f);
|
||||||
|
}
|
||||||
|
foreach ($kept as $f) {
|
||||||
|
$this->assertFileExists($f);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public function testWrongPassphrase(): void {
|
public function testWrongPassphrase(): void {
|
||||||
Repository::init($this->backend, 'pw');
|
Repository::init($this->backend, 'pw');
|
||||||
$this->expectException(CryptoException::class);
|
$this->expectException(CryptoException::class);
|
||||||
|
|||||||
@@ -106,4 +106,51 @@ class UserExportTest extends TestCase {
|
|||||||
$this->expectExceptionMessage('missing on the location');
|
$this->expectExceptionMessage('missing on the location');
|
||||||
$repo->listUserExports();
|
$repo->listUserExports();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testResolveTakesARecordedPathAsIs(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport($path));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveTakesAUserIdWithOneExport(): void {
|
||||||
|
// Issue #22: "backup:user:restore <location> New_User" was rejected as not a recorded export.
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
$this->putManifest($repo, 'users/alice2/20261005T120000Z-aaaa0002');
|
||||||
|
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport('alice'));
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport('users/alice/'));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveListsTheExportsOfAUserWithMore(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
$this->putManifest($repo, 'users/alice/20261006T120000Z-aaaa0002');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage("alice has 2 exports here; pass one of these (newest first):\n users/alice/20261006T120000Z-aaaa0002\n users/alice/20261005T120000Z-aaaa0001");
|
||||||
|
$repo->resolveUserExport('alice');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveListsTheExportsOfTheUserOfAWrongPath(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage("users/alice/typo is not a recorded user export for this location; pass one of these (newest first):\n users/alice/20261005T120000Z-aaaa0001");
|
||||||
|
$repo->resolveUserExport('users/alice/typo');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveNamesTheExpectedFormForAUserWithoutExports(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage('bob has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports');
|
||||||
|
$repo->resolveUserExport('bob');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -173,6 +173,20 @@ if [ "$LEGACY" = legacy ] && [ "$DB" != sqlite ]; then
|
|||||||
ok "older backup refused without --skip-missing-tables, missing tables named"
|
ok "older backup refused without --skip-missing-tables, missing tables named"
|
||||||
RESTORE_OPTS+=(--skip-missing-tables)
|
RESTORE_OPTS+=(--skip-missing-tables)
|
||||||
fi
|
fi
|
||||||
|
if [ "$DB" = sqlite ]; then
|
||||||
|
# Another process writing while the restore starts (a cron job that was already running, or
|
||||||
|
# the installer still finishing after it says "installed"): the restore must wait for it, not
|
||||||
|
# stop with "database is locked". Short write transactions on a table of its own, until the
|
||||||
|
# restore is done (/tmp/frt-side-stop; Nextcloud's own commands after it are not under test).
|
||||||
|
docker exec -d -u www-data ngb-frt-restore php -r '
|
||||||
|
$p = new PDO("sqlite:/var/www/html/data/nextcloud.db"); $p->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
||||||
|
$p->exec("CREATE TABLE IF NOT EXISTS frt_side_writer (t REAL)");
|
||||||
|
for ($end = microtime(true) + 300; microtime(true) < $end && !file_exists("/tmp/frt-side-stop");) {
|
||||||
|
$p->beginTransaction(); $p->exec("INSERT INTO frt_side_writer VALUES (" . microtime(true) . ")"); usleep(300000); $p->commit(); usleep(20000);
|
||||||
|
}
|
||||||
|
touch("/tmp/frt-side-done");'
|
||||||
|
sleep 1
|
||||||
|
fi
|
||||||
sftp_logins() { docker logs ngb-frt-sftp 2>&1 | grep -c "Accepted password" || true; }
|
sftp_logins() { docker logs ngb-frt-sftp 2>&1 | grep -c "Accepted password" || true; }
|
||||||
[ "$MODE" = sftp ] && LOGINS_BEFORE=$(sftp_logins)
|
[ "$MODE" = sftp ] && LOGINS_BEFORE=$(sftp_logins)
|
||||||
OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" "${RESTORE_OPTS[@]}" 2>&1) || {
|
OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" "${RESTORE_OPTS[@]}" 2>&1) || {
|
||||||
@@ -181,14 +195,22 @@ OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" "${RESTORE_OPTS
|
|||||||
fail "restore:full"
|
fail "restore:full"
|
||||||
}
|
}
|
||||||
printf '%s\n' "$OUT" | grep -q "maintenance mode while the restore runs" || fail "the restore did not say it runs in maintenance mode"
|
printf '%s\n' "$OUT" | grep -q "maintenance mode while the restore runs" || fail "the restore did not say it runs in maintenance mode"
|
||||||
run_on ngb-frt-restore status 2>&1 | grep -q "maintenance: false" || fail "maintenance mode is still on after the restore"
|
run_on ngb-frt-restore status 2>&1 | grep "maintenance: false" >/dev/null || fail "maintenance mode is still on after the restore"
|
||||||
run_on ngb-frt-restore config:app:get ng_backup restore_full_started >/dev/null 2>&1 && fail "the restore left its 'started' marker behind"
|
run_on ngb-frt-restore config:app:get ng_backup restore_full_started >/dev/null 2>&1 && fail "the restore left its 'started' marker behind"
|
||||||
ok "backup:restore:full completed, in maintenance mode and out of it again"
|
ok "backup:restore:full completed, in maintenance mode and out of it again"
|
||||||
|
if [ "$DB" = sqlite ]; then
|
||||||
|
docker exec ngb-frt-restore touch /tmp/frt-side-stop
|
||||||
|
for i in $(seq 1 20); do docker exec ngb-frt-restore test -f /tmp/frt-side-done && break; sleep 0.5; done
|
||||||
|
WRITES=$(docker exec -u www-data ngb-frt-restore php -r 'echo (new PDO("sqlite:/var/www/html/data/nextcloud.db"))->query("SELECT COUNT(*) FROM frt_side_writer")->fetchColumn();')
|
||||||
|
[ "${WRITES:-0}" -gt 3 ] || fail "the other writer did not run alongside the restore ($WRITES writes)"
|
||||||
|
ok "the restore waited for another process writing to SQLite ($WRITES writes alongside)"
|
||||||
|
fi
|
||||||
|
|
||||||
# Everything below must work in a FRESH occ process (a new PHP process per docker exec), since
|
# Everything below must work in a FRESH occ process (a new PHP process per docker exec), since
|
||||||
# earlier bugs here only showed up after config.php's secret was rewritten mid-restore.
|
# earlier bugs here only showed up after config.php's secret was rewritten mid-restore.
|
||||||
run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after restore (own key/target undecryptable?)"
|
run_on ngb-frt-restore backup:status >/dev/null || fail "backup:status fails after restore (own key/target undecryptable?)"
|
||||||
run_on ngb-frt-restore user:list | grep -q frtest || fail "the backed-up user is missing from the restored database"
|
# not grep -q: it stops reading at the first match, and occ, still writing, then fails (pipefail)
|
||||||
|
run_on ngb-frt-restore user:list | grep frtest >/dev/null || fail "the backed-up user is missing from the restored database"
|
||||||
docker exec ngb-frt-restore test -f /var/www/html/data/frtest/files/Docs/note.txt || fail "restored file is missing"
|
docker exec ngb-frt-restore test -f /var/www/html/data/frtest/files/Docs/note.txt || fail "restored file is missing"
|
||||||
[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt)" = "hello disaster recovery" ] || fail "restored file content differs"
|
[ "$(docker exec ngb-frt-restore cat /var/www/html/data/frtest/files/Docs/note.txt)" = "hello disaster recovery" ] || fail "restored file content differs"
|
||||||
run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after restore"
|
run_on ngb-frt-restore files:scan frtest >/dev/null || fail "files:scan fails after restore"
|
||||||
|
|||||||
@@ -49,9 +49,20 @@ SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xa
|
|||||||
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
$X sh -c "ls $DATA/$U/files_trashbin/files/ | grep -q '^extra.txt'" || fail "extra file not in trash"
|
||||||
ok "restore new-folder and replace: files identical, extra file in the trash"
|
ok "restore new-folder and replace: files identical, extra file in the trash"
|
||||||
|
|
||||||
|
# One file to a local directory, where a restore that was stopped left its temporary.
|
||||||
|
ONE=/tmp/ngb-rt-one/data/$U/files/Docs
|
||||||
|
$X sh -c "mkdir -p $ONE && echo partial > $ONE/f2.bin.ngb-restore-0123456789ab"
|
||||||
|
occ backup:restore ci "$SNAP" "data/$U/files/Docs/f2.bin" -d /tmp/ngb-rt-one | grep "Restored 1 files" >/dev/null || fail "restore of one file to a directory"
|
||||||
|
$X sh -c "cmp -s $ONE/f2.bin $DATA/$U/files/Docs/f2.bin" || fail "the file restored to a directory differs"
|
||||||
|
$X sh -c "ls -A $ONE | grep -q ngb-restore" && fail "the temporary of a stopped restore was left"
|
||||||
|
ok "restore of one file to a directory; the temporary of a stopped restore removed"
|
||||||
|
# not grep -q: it stops reading at the first match, and occ, still writing, then fails (pipefail)
|
||||||
|
occ backup:status | grep "started (" >/dev/null || fail "backup:status does not name the time zone"
|
||||||
|
ok "backup:status names the time zone of its times"
|
||||||
|
|
||||||
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
|
# Checksum audit: shallow (index presence) and deep (download, decrypt, verify every blob).
|
||||||
occ backup:verify ci "$SNAP" | grep -q "^OK$" || fail "shallow verify"
|
occ backup:verify ci "$SNAP" | grep "^OK$" >/dev/null || fail "shallow verify"
|
||||||
occ backup:verify ci "$SNAP" --deep | grep -q "^OK$" || fail "deep verify"
|
occ backup:verify ci "$SNAP" --deep | grep "^OK$" >/dev/null || fail "deep verify"
|
||||||
ok "shallow and deep checksum audit"
|
ok "shallow and deep checksum audit"
|
||||||
|
|
||||||
# Database dump/restore round trip (all tables, checksums, sequences on PostgreSQL).
|
# Database dump/restore round trip (all tables, checksums, sequences on PostgreSQL).
|
||||||
|
|||||||
@@ -36,8 +36,14 @@ MANIFEST=$(occ backup:user:backup ci "$U" | sed -n 's/.* to \(.*\)\.$/\1/p')
|
|||||||
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
||||||
ok "backed up $U to $MANIFEST"
|
ok "backed up $U to $MANIFEST"
|
||||||
|
|
||||||
# as-backup: restore into <uid>-bak; the original account must stay untouched.
|
# not grep -q: it stops reading at the first match, and occ, still writing, then fails (pipefail)
|
||||||
occ backup:user:restore ci "$MANIFEST" --mode=as-backup >/dev/null || fail "restore as-backup"
|
occ backup:user:list ci "$U" | grep -F "$MANIFEST" >/dev/null || fail "backup:user:list does not show $MANIFEST"
|
||||||
|
ok "backup:user:list shows the export"
|
||||||
|
|
||||||
|
# as-backup: restore into <uid>-bak; the original account must stay untouched. $U has one export,
|
||||||
|
# so the bare user id names it (#22).
|
||||||
|
OUT=$(occ backup:user:restore ci "$U" --mode=as-backup) || fail "restore as-backup by user id"
|
||||||
|
echo "$OUT" | grep -qF "Using $MANIFEST." || fail "restore by user id did not say which export it used"
|
||||||
occ files:scan "$U-bak" >/dev/null
|
occ files:scan "$U-bak" >/dev/null
|
||||||
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||||
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
||||||
@@ -55,6 +61,11 @@ SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xa
|
|||||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
||||||
ok "replace restores the original content under the same uid, byte-identical"
|
ok "replace restores the original content under the same uid, byte-identical"
|
||||||
|
|
||||||
|
# The replace above added a safety export: a bare user id is now ambiguous and must list both.
|
||||||
|
if OUT=$(occ backup:user:restore ci "$U" --mode=as-backup 2>&1); then fail "restore by an ambiguous user id ran"; fi
|
||||||
|
echo "$OUT" | grep -qF "$U has 2 exports here" && echo "$OUT" | grep -qF "$MANIFEST" || fail "an ambiguous user id did not list the exports"
|
||||||
|
ok "an ambiguous user id lists the exports instead of picking one"
|
||||||
|
|
||||||
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
||||||
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
||||||
# must forget exactly the 2 oldest.
|
# must forget exactly the 2 oldest.
|
||||||
|
|||||||
Reference in New Issue
Block a user