Compare commits
33 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4847ce293d | |||
| 90dd97977a | |||
| 08534975d7 | |||
| 95e3dfcf6d | |||
| 7f4cae0e42 | |||
| 965440d1f7 | |||
| 3a0f882a38 | |||
| 632d2a2838 | |||
| 9e9234e2a8 | |||
| 3ac3298313 | |||
| 384b2d2792 | |||
| 8032411789 | |||
| b4f20cb070 | |||
| 343389b98b | |||
| 133f5f9d34 | |||
| cf4bd9db7f | |||
| 67cc350b8c | |||
| fb5b234e8a | |||
| fd12c586b6 | |||
| 6ffe979ea7 | |||
| a03b7354b3 | |||
| be1c628f07 | |||
| a8355df12e | |||
| cd251cffb3 | |||
| 0b4872599d | |||
| b7d09c9784 | |||
| 24686897c3 | |||
| 315e83b847 | |||
| b728db36c7 | |||
| 6ed0a555cf | |||
| 221ed9dc17 | |||
| 285153a968 | |||
| 6c8a78f6bb |
@@ -41,6 +41,20 @@ jobs:
|
|||||||
- run: composer install --no-interaction --no-progress
|
- run: composer install --no-interaction --no-progress
|
||||||
- run: composer psalm
|
- run: composer psalm
|
||||||
|
|
||||||
|
frontend:
|
||||||
|
# Builds the admin page, so a broken Vue component fails CI instead of showing up after a release.
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: '22'
|
||||||
|
cache: npm
|
||||||
|
- run: npm ci --no-audit --no-fund
|
||||||
|
- run: npm run build
|
||||||
|
|
||||||
integration:
|
integration:
|
||||||
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
|
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
|
||||||
# restore, retention, trash) on a fresh Nextcloud per version and database.
|
# restore, retention, trash) on a fresh Nextcloud per version and database.
|
||||||
|
|||||||
@@ -7,6 +7,56 @@ the app store accepts a `-beta` suffix, but Nextcloud servers then only offer th
|
|||||||
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
||||||
instead carried by the app description, the git tag, the release, and this file.
|
instead carried by the app description, the git tag, the release, and this file.
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- `occ backup:user:list [location] [user]` shows the user exports on a location, with the manifest
|
||||||
|
path `backup:user:restore` expects. Before, that path was only printed once by
|
||||||
|
`backup:user:backup` (#22).
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- `occ backup:user:restore` also takes a user id instead of the manifest path, when that user has
|
||||||
|
exactly one export on the location. With more, and for a manifest path that is not on the
|
||||||
|
location, the error lists that user's exports; for a user without exports it names the
|
||||||
|
expected form (#22). Before, `backup:user:restore <location> <uid>` only said "is not a
|
||||||
|
recorded user export for this location".
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- An SFTP location with an RSA key in OpenSSH format ("BEGIN OPENSSH PRIVATE KEY") whose line
|
||||||
|
breaks were lost, e.g. turned into spaces on the way in, stopped working after an upgrade to
|
||||||
|
Nextcloud 35 with "Unable to read key": Nextcloud 35 reads keys with a stricter library. The
|
||||||
|
line breaks are now restored before the key is used, so the stored location works again without
|
||||||
|
re-entering it (#24).
|
||||||
|
|
||||||
|
## [1.0.0-rc.6] - 2026-10-24
|
||||||
|
|
||||||
|
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
||||||
|
simulations and a user report. If no problems are reported for about a week, 1.0.0 will be this
|
||||||
|
same code.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
||||||
|
Berlin (#19). The time is now saved with the administrator's time zone and shown with it. A time
|
||||||
|
saved before is converted once to the zone of the server or an administrator, so it shows as it
|
||||||
|
effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer) and nothing moves.
|
||||||
|
- A full restore over SMB stopped after 17 hours with "Decryption failed": a read over a long
|
||||||
|
line handed out wrong bytes (the backup itself was fine, the same files restored identically
|
||||||
|
afterwards). A blob that does not decrypt as first read, of a file or of the database, is now
|
||||||
|
read once more on a request of its own, and only a second failure stops the restore.
|
||||||
|
- A full restore onto MySQL/MariaDB stopped with "Incorrect string value" when it created a table
|
||||||
|
(of an app that is not installed here) whose rows hold emoji or other 4-byte characters: the
|
||||||
|
table got 3-byte utf8. Tables the restore creates now get this installation's character set and
|
||||||
|
collation (utf8mb4). Found in a disaster recovery simulation onto MariaDB.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- The admin page uses more of the screen (sections up to 1400 pixels instead of 900), so the
|
||||||
|
list of backup runs fits without scrolling sideways.
|
||||||
|
- README: a roadmap, the tested disaster recoveries, measurements for OVH S3, the Storage Box over
|
||||||
|
SFTP and SMB, how to build the SMB extension when PECL is down, and what works without shell
|
||||||
|
access. The guide lists shell access among what you need.
|
||||||
|
- The full-restore integration test checks a 4-byte character in a table the restore creates, and
|
||||||
|
retries app installs from the app store.
|
||||||
|
|
||||||
## [1.0.0-rc.5] - 2026-10-22
|
## [1.0.0-rc.5] - 2026-10-22
|
||||||
|
|
||||||
Fifth release candidate for 1.0.0; the app version is 0.9.11. A full restore of 1 TB over SFTP
|
Fifth release candidate for 1.0.0; the app version is 0.9.11. A full restore of 1 TB over SFTP
|
||||||
|
|||||||
@@ -395,11 +395,73 @@ in README "Known limitations":
|
|||||||
- **1.0.1:**
|
- **1.0.1:**
|
||||||
1. (Naar rc4: herstelsnelheid over SFTP.) S3 leest al 16 tegelijk bij index en tree, maar een
|
1. (Naar rc4: herstelsnelheid over SFTP.) S3 leest al 16 tegelijk bij index en tree, maar een
|
||||||
restore leest de packs nog één voor één: ook vooruitlezen via de S3-client.
|
restore leest de packs nog één voor één: ook vooruitlezen via de S3-client.
|
||||||
2. Waarschuwing in de beheerpagina en `backup:status` als achtergrondjobs niet op systeem-cron
|
2. Achtergrondjobs zonder systeem-cron (issue #13; besluit 2026-10-19). De PR die dit
|
||||||
staan, en "waiting for background jobs" als een lopende backup lang geen stap zette (issue
|
oplost zegt "Fixes #13" (bevestigd: #13 was de cron; besluit 2026-10-21):
|
||||||
#13). Cron zelf aanzetten kan de app niet: zonder echte cron-regel stopt dan alles.
|
a. **Waarschuwen** in de beheerpagina en `backup:status`: modus uit `core backgroundjobs_mode`
|
||||||
|
(`ajax`/`webcron`/`cron`) en de laatste run uit `core lastcron`. Niet op `cron`, of de
|
||||||
|
laatste run te lang geleden: melding met de cron-regel erbij ("Backups only move while
|
||||||
|
someone has Nextcloud open"), en "waiting for background jobs" als een lopende backup lang
|
||||||
|
geen stap zette.
|
||||||
|
b. **De pagina laat de backup lopen** zolang ze openstaat: in `ajax`/`webcron` voert de
|
||||||
|
statusaanroep die de pagina toch al herhaalt ook een korte, begrensde stap uit (zoals de
|
||||||
|
korte stappen die AJAX-cron nu al krijgt). "Back up now" werkt dan ook zonder cron zolang
|
||||||
|
het tabblad openstaat; de melding zegt dat geplande backups echte cron nodig hebben.
|
||||||
|
De melding zegt (zonder werkende systeem-cron): backups hebben systeem-cron nodig, ook de
|
||||||
|
gewone nachtelijke; zo stel je cron in (link naar de Nextcloud-documentatie over
|
||||||
|
achtergrondjobs voor de versie van de server, nu
|
||||||
|
https://docs.nextcloud.com/server/34/admin_manual/configuration_server/background_jobs_configuration.html,
|
||||||
|
versienummer uit de server, niet vast); zolang er geen cron is, loopt een backup die je
|
||||||
|
met de knop start alleen door terwijl de NG Backup-pagina openstaat.
|
||||||
|
c. **Voortgang in de databasefase** (issue #13): nu staat er alleen "running (db)", wat er
|
||||||
|
hetzelfde uitziet als vastzitten. Toon "running (db: 47/183 tables)" zoals de bestandsfase
|
||||||
|
een percentage toont, en "waiting for background jobs (last ran 12 min ago)" als een run
|
||||||
|
een tijd geen stap zette, ook bij een cron die is blijven staan.
|
||||||
|
Niet doen: de cron-modus vanuit de app omzetten (zonder echte cron-regel stopt alles, en
|
||||||
|
`cron.php` zet de modus zelf op `cron`), en geen `exec php occ backup:run &` vanuit de knop
|
||||||
|
(vaak uitgeschakeld, niet op hosted, buiten Nextcloud om lastig te volgen en te stoppen).
|
||||||
3. Volledige DR-test met een backup die al de tabelstructuur bevat (gemaakt met rc2 of later),
|
3. Volledige DR-test met een backup die al de tabelstructuur bevat (gemaakt met rc2 of later),
|
||||||
zonder `--skip-missing-tables`: dat pad is op 1 TB nog niet getest.
|
zonder `--skip-missing-tables`: dat pad is op 1 TB nog niet getest.
|
||||||
|
4. **Label bij "Back up now"** (besluit 2026-10-19): een optioneel tekstveld naast de knop; de
|
||||||
|
tekst gaat als label van de snapshot mee (nu kan dat alleen met `occ backup:run -l`; de knop
|
||||||
|
stuurt een leeg label, geplande runs krijgen "scheduled"). Raakt alleen de UI en een
|
||||||
|
bestaande parameter.
|
||||||
|
5. **`--salvage`** voor `backup:restore` en `backup:restore:full` (besluit 2026-10-22): terughalen
|
||||||
|
wat kan, een bestand dat niet te lezen of te ontsleutelen is overslaan en aan het eind
|
||||||
|
opsommen, in plaats van stoppen.
|
||||||
|
6. **Tijdelijke bestanden van een afgebroken restore opruimen** (gevonden in de SMB-DR van
|
||||||
|
rc6, 2026-10-24): de rc5-poging die stopte liet `VID_0009.mp4.ngb-restore-<id>` staan; de
|
||||||
|
geslaagde restore daarna haalde het niet weg, en `files:scan` maakte het zichtbaar in Files.
|
||||||
|
Een restore ruimt `*.ngb-restore-*` in de mappen die hij terugzet op.
|
||||||
|
7. **Eén bestand terugzetten met `backup:restore`**: een pad naar een bestand zet 0 bestanden
|
||||||
|
terug (er komt een `/` achter); nu werkt alleen een map.
|
||||||
|
8. **Uitleg (i) bij de prullenbak van herstelpunten** (besluit 2026-10-25): bij "removed by
|
||||||
|
retention policy" een infoknop met wat het betekent: het bewaarbeleid houdt voor oudere dagen
|
||||||
|
één herstelpunt per dag (het laatste), weken en maanden; de rest gaat naar de prullenbak en
|
||||||
|
wordt pas na de ingestelde wachttijd (`delete_delay_days`, standaard 30) echt verwijderd, en is
|
||||||
|
tot die datum terug te zetten. Aanleiding: de beheerder zag "19-10 13:12 – after rc4 upgrade
|
||||||
|
removed by retention policy" en dacht dat er iets was weggegooid.
|
||||||
|
9. **Tijdzone in `occ backup:status`**: de tabel toont de servertijd (UTC), de webinterface de
|
||||||
|
tijd van de beheerder; dat leest als een verkeerde starttijd. Zet de zone in de kolomkop
|
||||||
|
("started (UTC)") of toon de tijden in de zone van het schema.
|
||||||
|
10. **User-restore op gebruikersnaam** (issue #22, 2026-10-27): `backup:user:restore <locatie>
|
||||||
|
New_User` gaf alleen "is not a recorded user export"; het argument moet het manifestpad
|
||||||
|
zijn, dat alleen `backup:user:backup` eenmalig toonde. Nu: `backup:user:list` toont de
|
||||||
|
exports, een kale uid werkt bij precies één export, anders een lijst om uit te kiezen —
|
||||||
|
**klaar** (2026-10-27).
|
||||||
|
11. **Tabblad "Docs" in de beheerpagina** (besluit 2026-10-27): een derde knop naast "Backup and
|
||||||
|
restore" en "Settings", met links naar de HOWTO (`docs/HOWTO.md`), de DR-gids
|
||||||
|
(`docs/DISASTER-RECOVERY.md`) en de roadmap (README, `#roadmap`) op GitHub, elk met een
|
||||||
|
regel wat erin staat. Nu zijn ze alleen via de README te vinden. Zichtbaar ook zonder
|
||||||
|
sleutel, zodat de gids vindbaar is vóór de eerste installatie-stap. "NG Backup is in beta"
|
||||||
|
gaat uit de feedbackregel — **klaar** (2026-10-27).
|
||||||
|
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
|
||||||
|
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
|
||||||
|
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
|
||||||
|
(MariaDB-bron, 138 GB) nogmaals naar een MariaDB-doel en de SMB-simulatie naar een SQLite-doel;
|
||||||
|
**na 1.0.0** de 1 TB van nc01 (PostgreSQL) naar een MariaDB-doel.
|
||||||
|
- **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3,
|
||||||
|
4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan
|
||||||
|
een paar bestanden.
|
||||||
- **1.1.0:**
|
- **1.1.0:**
|
||||||
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
|
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
|
||||||
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
||||||
@@ -412,6 +474,31 @@ in README "Known limitations":
|
|||||||
3. Snelheid: lezen en uploaden laten overlappen, meerdere bestanden tegelijk lezen, eventueel
|
3. Snelheid: lezen en uploaden laten overlappen, meerdere bestanden tegelijk lezen, eventueel
|
||||||
een eigen worker (`occ background-job:worker`). Gemeten op nc01: 55% wachten op de schijf,
|
een eigen worker (`occ background-job:worker`). Gemeten op nc01: 55% wachten op de schijf,
|
||||||
33% op de SMB-upload, ~7% CPU. Vergt extra PHP-processen; botst mogelijk met uitgangspunt 1.
|
33% op de SMB-upload, ~7% CPU. Vergt extra PHP-processen; botst mogelijk met uitgangspunt 1.
|
||||||
|
4. **Schema per locatie** (besluit 2026-10-19): in de instellingen van een locatie "uit",
|
||||||
|
"dagelijks om HH:MM" of "elke N uur (vanaf HH:MM)". Nu is er één `schedule_time` voor alle
|
||||||
|
locaties, één keer per dag. Bij de update wordt de huidige tijd het schema van elke
|
||||||
|
bestaande locatie (voor bestaande gebruikers verandert er niets).
|
||||||
|
5. **Locaties na elkaar laten starten** in plaats van tegelijk: twee runs die dezelfde schijven
|
||||||
|
lezen zijn samen niet sneller (gezien op nc01); wachtende runs zichtbaar als "queued".
|
||||||
|
7. **SMB sneller** (gemeten 2026-10-21): lokaal netwerk zonder vertraging (Samba-container op de
|
||||||
|
laptop, 1,8 GB) back-up 24,9 MB/s, restore 27,6 MiB/s; vanaf Canada naar de Storage Box (130 ms)
|
||||||
|
2,6 MiB/s, omdat SMB per blok op het antwoord wacht. Grotere schrijfblokken, en vooruitlezen
|
||||||
|
bij een restore zoals SFTP (IReadsAhead) of meerdere packs tegelijk. Een NAS met 1 Gbit haalt
|
||||||
|
~110 MB/s: daar is winst.
|
||||||
|
Ook de index: `target:add` over SMB vanaf Canada haalt ~1 indexbestand per seconde (4.927
|
||||||
|
bestanden ~85 min), want SMB krijgt geen `getMany()` met verzoeken tegelijk zoals S3 en SFTP.
|
||||||
|
6. **Notitie achteraf bij een restorepunt**, te wijzigen in de lijst. Het label in de snapshot
|
||||||
|
is versleuteld en ligt vast; de notitie apart op de locatie bewaren (bijv. in de catalogus),
|
||||||
|
zodat een nieuwe server na een DR hem ook ziet.
|
||||||
|
8. **Herstelpunt vastpinnen** (besluit 2026-10-25): een herstelpunt dat het bewaarbeleid
|
||||||
|
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
||||||
|
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
||||||
|
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
||||||
|
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
|
||||||
|
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
||||||
|
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
|
||||||
|
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
|
||||||
|
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
|
||||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||||
|
|||||||
@@ -146,10 +146,31 @@ What you should do on the location (strongly recommended):
|
|||||||
- **Files and folders** of a user: in the web interface or `occ backup:restore`, into a new folder
|
- **Files and folders** of a user: in the web interface or `occ backup:restore`, into a new folder
|
||||||
("Restored <date>"), merged into the original place, or replacing it.
|
("Restored <date>"), merged into the original place, or replacing it.
|
||||||
- **Raw restore** of any path (for example `config/`) to a local directory: `occ backup:restore ... --to-directory=DIR`.
|
- **Raw restore** of any path (for example `config/`) to a local directory: `occ backup:restore ... --to-directory=DIR`.
|
||||||
- **Whole server** (disaster recovery onto a fresh installation): `occ backup:restore:full`, see the how-to.
|
- **Whole server** (disaster recovery onto a fresh installation): `occ backup:restore:full`, see
|
||||||
|
[docs/DISASTER-RECOVERY.md](docs/DISASTER-RECOVERY.md). It runs in maintenance mode, so that
|
||||||
|
nobody and no background job writes into the instance while it is being replaced.
|
||||||
- **One user** (account, settings, files, calendars, contacts via user_migration): `occ backup:user:backup` and
|
- **One user** (account, settings, files, calendars, contacts via user_migration): `occ backup:user:backup` and
|
||||||
`occ backup:user:restore`, replacing the account or restoring it as `<uid>-bak`. See the how-to.
|
`occ backup:user:restore`, replacing the account or restoring it as `<uid>-bak`. See the how-to.
|
||||||
|
|
||||||
|
**Without shell access** (a hosted Nextcloud where you cannot run `occ`): files and folders can be
|
||||||
|
restored in the web interface. Restoring a whole user or the whole server needs `occ`, so on such
|
||||||
|
hosting it is something your provider (or whoever has shell access) runs. A restore mode that
|
||||||
|
works without maintenance mode, and so from the web interface, is on the roadmap.
|
||||||
|
|
||||||
|
## Tested disaster recoveries
|
||||||
|
|
||||||
|
Full restores onto a fresh server, following only [the guide](docs/DISASTER-RECOVERY.md):
|
||||||
|
|
||||||
|
| Backup | Location | Database | Result |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Production, about 1 TB, 150,000 files (Nextcloud All-in-One) | Hetzner Storage Box (SFTP), Germany → Canada | PostgreSQL → PostgreSQL | 150,622 files restored in about 25 h (13-14 MiB/s), sample identical |
|
||||||
|
| 138 GB simulation: 20 users, calendars, contacts, mail, shares, team folders with an ACL, versions, trash | OVH S3, Warsaw → Canada | MariaDB → PostgreSQL | all 15,772 files and every database count identical |
|
||||||
|
| The same | OVH S3 | MariaDB → MariaDB | all 15,772 files and every database count identical |
|
||||||
|
| The same | Hetzner Storage Box (SMB), Germany → Canada | MariaDB → SQLite | all 15,772 files and every database count identical (27.6 h, about 1.5 MB/s) |
|
||||||
|
| 51 GB test instance | OVH S3 | PostgreSQL → PostgreSQL | 23,519/23,519 checksums identical |
|
||||||
|
|
||||||
|
The integration tests restore across every pair of PostgreSQL, MariaDB and SQLite.
|
||||||
|
|
||||||
## Storage backend experiences
|
## Storage backend experiences
|
||||||
|
|
||||||
Notes from actually running a backend, beyond what's in files_external's own documentation.
|
Notes from actually running a backend, beyond what's in files_external's own documentation.
|
||||||
@@ -206,12 +227,70 @@ Contributions welcome via PR — add what you ran into with your own provider.
|
|||||||
can be read on more connections: `occ config:app:set ng_backup restore_sftp_connections
|
can be read on more connections: `occ config:app:set ng_backup restore_sftp_connections
|
||||||
--value=4` (at most 8; the server must allow that many plus one at once for the user).
|
--value=4` (at most 8; the server must allow that many plus one at once for the user).
|
||||||
|
|
||||||
|
- **OVH Object Storage (S3, Warsaw, 2026-10):** works with the `amazons3` backend (path style,
|
||||||
|
region `waw`). A full backup of 1 TB from Germany ran at about 7 MiB/s, from Canada about
|
||||||
|
6 MiB/s with BBR; a check of a 1 TB backup took 3.5 minutes (index files are fetched 16 at a
|
||||||
|
time); a restore of 138 GB from Warsaw to Canada took about 6 hours. Turn on versioning and
|
||||||
|
Object Lock on the bucket (see above); OVH charges no egress.
|
||||||
|
|
||||||
|
- **Hetzner Storage Box over SFTP, full restore:** about 13-14 MiB/s from Germany to Canada
|
||||||
|
(122 ms) with the read-ahead of 1.0.0-rc.4; without it about 2.8 MiB/s. A Storage Box gets
|
||||||
|
slower with several connections reading at once (4-6 MiB/s for 2-4 connections, it reads from
|
||||||
|
disks), so one extra connection is the default. A Storage Box drops an SFTP connection that
|
||||||
|
stayed quiet for a while without telling the client; NG Backup checks and replaces it.
|
||||||
|
|
||||||
|
- **SMB speed:** on a local network (Samba, no delay) a backup ran at about 25 MB/s and a
|
||||||
|
restore at about 28 MiB/s. Over a long line SMB waits for an answer to each request: from
|
||||||
|
Canada to a Storage Box in Germany (130 ms) about 2.6 MiB/s for a backup and 1.6 MiB/s for a
|
||||||
|
restore, and connecting a location with an existing backup reads its index at about one file a
|
||||||
|
second. SMB is meant for a local network; use SFTP or S3 for a location far away. When
|
||||||
|
`pecl install smbclient` fails (pecl.php.net was unreachable for hours on 2026-10-21), the
|
||||||
|
extension builds from its source: `libsmbclient-php` on GitHub, `phpize && ./configure && make
|
||||||
|
&& make install`, then `docker-php-ext-enable smbclient`.
|
||||||
|
|
||||||
- **NFS is not a files_external backend at all.** Mount the NFS export on the server yourself
|
- **NFS is not a files_external backend at all.** Mount the NFS export on the server yourself
|
||||||
(e.g. via `/etc/fstab` or a systemd `.mount` unit) and point ng_backup's `local` backend at the
|
(e.g. via `/etc/fstab` or a systemd `.mount` unit) and point ng_backup's `local` backend at the
|
||||||
mount point; ng_backup never mounts or unmounts anything on its own, so an NFS mount dropping
|
mount point; ng_backup never mounts or unmounts anything on its own, so an NFS mount dropping
|
||||||
mid-run behaves like any other local-path failure (the run resumes). See PLAN.md 3.5 for why NFS
|
mid-run behaves like any other local-path failure (the run resumes). See PLAN.md 3.5 for why NFS
|
||||||
can't be driven from PHP directly.
|
can't be driven from PHP directly.
|
||||||
|
|
||||||
|
## Roadmap
|
||||||
|
|
||||||
|
What is planned, without dates. Ideas and votes are welcome in the
|
||||||
|
[issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
|
||||||
|
a 👍 on its issue helps decide what comes first.
|
||||||
|
|
||||||
|
**1.0.1**
|
||||||
|
- A warning on the admin page and in `occ backup:status` when background jobs do not run on a
|
||||||
|
system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
|
||||||
|
while the NG Backup page is open ([#13](https://github.com/dillardblom/ng_backup/issues/13)).
|
||||||
|
- An optional label for a backup started with "Back up now".
|
||||||
|
- An explanation in the trash of restore points of what "removed by retention policy" means and
|
||||||
|
until when a restore point can be brought back.
|
||||||
|
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
|
||||||
|
- A "Docs" tab on the admin page with links to the how-to, the disaster recovery guide and this
|
||||||
|
roadmap.
|
||||||
|
|
||||||
|
**1.1.0**
|
||||||
|
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
||||||
|
your passphrases.
|
||||||
|
- A schedule per location, also "every N hours".
|
||||||
|
- Locations backed up one after the other instead of at the same time.
|
||||||
|
- A note on a restore point that you can change afterwards.
|
||||||
|
- Pinning a restore point, so the retention policy never removes it (for example the one from
|
||||||
|
before an upgrade).
|
||||||
|
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
||||||
|
Generator) while a backup runs.
|
||||||
|
- Faster backups: reading and uploading at the same time.
|
||||||
|
|
||||||
|
**Later**
|
||||||
|
- A full restore onto the existing installation, without first installing a fresh Nextcloud, in a
|
||||||
|
restore mode of its own instead of maintenance mode, so that it can also run from the web
|
||||||
|
interface (for hosted Nextcloud without shell access).
|
||||||
|
- Users restoring their own files from the Files app.
|
||||||
|
- Primary object storage and External Storage mounts (see Known limitations).
|
||||||
|
- An option not to keep the backup key on the location (the recovery kit is then the only copy).
|
||||||
|
|
||||||
## Known limitations
|
## Known limitations
|
||||||
|
|
||||||
On the backlog; none of these block the beta or 1.0.0, but know them before you rely on it:
|
On the backlog; none of these block the beta or 1.0.0, but know them before you rely on it:
|
||||||
@@ -236,8 +315,11 @@ On the backlog; none of these block the beta or 1.0.0, but know them before you
|
|||||||
- Location passwords passed with `-o password=...` are visible in the process list and shell
|
- Location passwords passed with `-o password=...` are visible in the process list and shell
|
||||||
history while the command runs.
|
history while the command runs.
|
||||||
- Temporary files of a restore into a user's files end up in that user's trash.
|
- Temporary files of a restore into a user's files end up in that user's trash.
|
||||||
- Restoring the database does not order tables by foreign keys; apps with cascading foreign keys
|
- **Restoring a whole user or the whole server needs `occ`** (shell access); see "Without shell
|
||||||
could lose rows (Nextcloud core does not use them).
|
access" above.
|
||||||
|
- **`--install-apps` depends on the app store.** Nextcloud downloads the whole app list (about
|
||||||
|
38 MB) and gives up after 120 seconds; on a slow day apps are not installed. The restore then
|
||||||
|
lists them: install them afterwards with `occ app:install`, their data is restored already.
|
||||||
- A pack whose index upload failed stays on the location until removed by hand.
|
- A pack whose index upload failed stays on the location until removed by hand.
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
@@ -254,4 +336,4 @@ On the backlog; none of these block the beta or 1.0.0, but know them before you
|
|||||||
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
||||||
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
||||||
| `backup:key:import` | import a recovery kit on a fresh installation |
|
| `backup:key:import` | import a recovery kit on a fresh installation |
|
||||||
| `backup:user:backup`, `backup:user:restore` | one user via user_migration (asks before installing it) |
|
| `backup:user:backup`, `backup:user:list`, `backup:user:restore` | one user via user_migration (asks before installing it); list the exports |
|
||||||
|
|||||||
+5
-4
@@ -42,7 +42,7 @@ welcome: use the "Share your experience" link on the admin page.
|
|||||||
|
|
||||||
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
||||||
you need help with restoring a backup after a crash.]]></description>
|
you need help with restoring a backup after a crash.]]></description>
|
||||||
<version>0.9.11</version>
|
<version>0.9.12</version>
|
||||||
<licence>agpl</licence>
|
<licence>agpl</licence>
|
||||||
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
||||||
<namespace>NgBackup</namespace>
|
<namespace>NgBackup</namespace>
|
||||||
@@ -53,9 +53,9 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<website>https://github.com/dillardblom/ng_backup</website>
|
<website>https://github.com/dillardblom/ng_backup</website>
|
||||||
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
||||||
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.5/screenshots/1-backups.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/1-backups.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.5/screenshots/2-restore.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/2-restore.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.5/screenshots/3-settings.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/3-settings.png</screenshot>
|
||||||
<dependencies>
|
<dependencies>
|
||||||
<php min-version="8.1"/>
|
<php min-version="8.1"/>
|
||||||
<lib>sodium</lib>
|
<lib>sodium</lib>
|
||||||
@@ -92,6 +92,7 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
||||||
<command>OCA\NgBackup\Command\BackupUser</command>
|
<command>OCA\NgBackup\Command\BackupUser</command>
|
||||||
<command>OCA\NgBackup\Command\RestoreUser</command>
|
<command>OCA\NgBackup\Command\RestoreUser</command>
|
||||||
|
<command>OCA\NgBackup\Command\ListUserExports</command>
|
||||||
<command>OCA\NgBackup\Command\KeyImport</command>
|
<command>OCA\NgBackup\Command\KeyImport</command>
|
||||||
<command>OCA\NgBackup\Command\RestoreFull</command>
|
<command>OCA\NgBackup\Command\RestoreFull</command>
|
||||||
<command>OCA\NgBackup\Command\Verify</command>
|
<command>OCA\NgBackup\Command\Verify</command>
|
||||||
|
|||||||
@@ -10,6 +10,8 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
|||||||
- [ ] A new server with the **same Nextcloud major version** as the old one (for example 34),
|
- [ ] A new server with the **same Nextcloud major version** as the old one (for example 34),
|
||||||
a database (PostgreSQL, MySQL/MariaDB or SQLite; it may differ from the old one) and room for
|
a database (PostgreSQL, MySQL/MariaDB or SQLite; it may differ from the old one) and room for
|
||||||
the whole backup plus a margin.
|
the whole backup plus a margin.
|
||||||
|
- [ ] **Shell access** to the new server, to run `occ` (on a hosted Nextcloud without it, your
|
||||||
|
provider runs these steps).
|
||||||
- [ ] A list of the apps that are **not in the app store** (your own builds, apps copied in by
|
- [ ] A list of the apps that are **not in the app store** (your own builds, apps copied in by
|
||||||
hand). Keep their packages somewhere safe beforehand.
|
hand). Keep their packages somewhere safe beforehand.
|
||||||
|
|
||||||
|
|||||||
@@ -114,10 +114,16 @@ Needs the `user_migration` app. NG Backup asks before installing it:
|
|||||||
|
|
||||||
```
|
```
|
||||||
occ backup:user:backup offsite <uid>
|
occ backup:user:backup offsite <uid>
|
||||||
|
occ backup:user:list offsite [<uid>] # the exports and their manifest
|
||||||
occ backup:user:restore offsite users/<uid>/<exportId> # replace (default)
|
occ backup:user:restore offsite users/<uid>/<exportId> # replace (default)
|
||||||
occ backup:user:restore offsite users/<uid>/<exportId> --mode=as-backup
|
occ backup:user:restore offsite users/<uid>/<exportId> --mode=as-backup
|
||||||
```
|
```
|
||||||
|
|
||||||
|
- `backup:user:restore` needs the manifest path that `backup:user:backup` prints at the end
|
||||||
|
(`users/<uid>/<exportId>`); `backup:user:list` shows it again. A user id alone also works when
|
||||||
|
that user has exactly one export on the location. With more (a `replace` restore adds a safety
|
||||||
|
export), the command lists them and you pass one.
|
||||||
|
|
||||||
- If `user_migration` is missing, the command asks to install and enable it. For non-interactive
|
- If `user_migration` is missing, the command asks to install and enable it. For non-interactive
|
||||||
use, add `--install-user-migration` to approve that explicitly. The command stops after the
|
use, add `--install-user-migration` to approve that explicitly. The command stops after the
|
||||||
installation; run it once more.
|
installation; run it once more.
|
||||||
|
|||||||
@@ -72,6 +72,10 @@ final class ExternalStorageFactory {
|
|||||||
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
|
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (is_string($options['private_key'] ?? null)) {
|
||||||
|
$options['private_key'] = self::normalizePrivateKey($options['private_key']);
|
||||||
|
}
|
||||||
|
|
||||||
$config = new StorageConfig();
|
$config = new StorageConfig();
|
||||||
$config->setBackend($backend);
|
$config->setBackend($backend);
|
||||||
$config->setAuthMechanism($auth);
|
$config->setAuthMechanism($auth);
|
||||||
@@ -86,4 +90,22 @@ final class ExternalStorageFactory {
|
|||||||
$storage = $backend->wrapStorage($storage);
|
$storage = $backend->wrapStorage($storage);
|
||||||
return $auth->wrapStorage($storage);
|
return $auth->wrapStorage($storage);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An OpenSSH-format private key with its line breaks restored. Up to Nextcloud 34
|
||||||
|
* (phpseclib 2) a key whose newlines were lost or turned into spaces, e.g. on its way through
|
||||||
|
* a shell or a form, still loaded; Nextcloud 35 (phpseclib 3) refuses it with "Unable to read
|
||||||
|
* key" (#24). The key itself is not changed, so it is fixed when it is used, not when stored.
|
||||||
|
* Other formats (PEM, PKCS#8) still load as they are and are returned unchanged.
|
||||||
|
*/
|
||||||
|
public static function normalizePrivateKey(string $key): string {
|
||||||
|
if (!preg_match('/-----BEGIN OPENSSH PRIVATE KEY-----(.*?)-----END OPENSSH PRIVATE KEY-----/s', $key, $m)) {
|
||||||
|
return $key;
|
||||||
|
}
|
||||||
|
$body = (string)preg_replace('/\s+/', '', $m[1]);
|
||||||
|
if (!preg_match('#^[A-Za-z0-9+/]+=*$#', $body)) {
|
||||||
|
return $key;
|
||||||
|
}
|
||||||
|
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -32,6 +32,7 @@ final class BackupJob extends TimedJob {
|
|||||||
private RunMapper $runs,
|
private RunMapper $runs,
|
||||||
private IAppConfig $appConfig,
|
private IAppConfig $appConfig,
|
||||||
private LoggerInterface $logger,
|
private LoggerInterface $logger,
|
||||||
|
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||||
) {
|
) {
|
||||||
parent::__construct($time);
|
parent::__construct($time);
|
||||||
// Well below the usual 5-minute cron interval: the last run is stamped a moment after cron
|
// Well below the usual 5-minute cron interval: the last run is stamped a moment after cron
|
||||||
@@ -101,14 +102,10 @@ final class BackupJob extends TimedJob {
|
|||||||
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
|
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Daily schedule: appconfig ng_backup/schedule_time "HH:MM" (server time); empty = no schedule. */
|
/** Daily schedule: "HH:MM" in the schedule's time zone (see Schedule); empty = no schedule. */
|
||||||
private function startScheduled(): void {
|
private function startScheduled(): void {
|
||||||
$at = $this->appConfig->getValueString(Application::APP_ID, 'schedule_time', '');
|
$todayAt = $this->schedule->todayAt();
|
||||||
if (!preg_match('/^([01]\d|2[0-3]):[0-5]\d$/', $at)) {
|
if ($todayAt === null || time() < $todayAt) {
|
||||||
return;
|
|
||||||
}
|
|
||||||
$todayAt = (int)strtotime(date('Y-m-d') . ' ' . $at);
|
|
||||||
if (time() < $todayAt) {
|
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
foreach ($this->targets->list() as $t) {
|
foreach ($this->targets->list() as $t) {
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Helper\Table;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class ListUserExports extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:user:list')
|
||||||
|
->setDescription('List the user exports (backup:user:backup) on a location')
|
||||||
|
->addArgument('target', InputArgument::OPTIONAL, 'Location name or id (default: all)')
|
||||||
|
->addArgument('user', InputArgument::OPTIONAL, 'Only the exports of this user id')
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
The manifest column is what occ backup:user:restore expects. A safety export made by a restore
|
||||||
|
with --mode=replace is listed here as well.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
occ backup:user:list offsite alice
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$name = $input->getArgument('target');
|
||||||
|
$uid = $input->getArgument('user');
|
||||||
|
$rows = [];
|
||||||
|
try {
|
||||||
|
$targets = $name === null ? $this->targets->list() : [$this->targets->get($name)];
|
||||||
|
foreach ($targets as $t) {
|
||||||
|
foreach ($this->targets->repository($t)->listUserExports($uid) as $path) {
|
||||||
|
$parts = explode('/', $path, 3);
|
||||||
|
$user = $parts[1] ?? '';
|
||||||
|
$exportId = $parts[2] ?? '';
|
||||||
|
$time = \DateTimeImmutable::createFromFormat('Ymd\THis\Z', substr($exportId, 0, 16), new \DateTimeZone('UTC'));
|
||||||
|
$rows[] = [$t->getName(), $user, $time !== false ? $time->format('Y-m-d H:i') : '', $path];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($rows === []) {
|
||||||
|
$output->writeln('No user exports yet (occ backup:user:backup).');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
(new Table($output))->setHeaders(['location', 'user', 'date (UTC)', 'manifest'])->setRows($rows)->render();
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -33,7 +33,7 @@ final class RestoreUser extends Command {
|
|||||||
$this->setName('backup:user:restore')
|
$this->setName('backup:user:restore')
|
||||||
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
||||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>)')
|
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>), or a user id with exactly one export')
|
||||||
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
||||||
->addOption('force', null, InputOption::VALUE_NONE, 'Replace an existing account without asking (required when not interactive)')
|
->addOption('force', null, InputOption::VALUE_NONE, 'Replace an existing account without asking (required when not interactive)')
|
||||||
->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)')
|
->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)')
|
||||||
@@ -45,16 +45,31 @@ Never imports over a live account. Modes:
|
|||||||
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
||||||
the new name, never the existing account
|
the new name, never the existing account
|
||||||
|
|
||||||
|
occ backup:user:list shows the exports on a location. A user id alone works when that user
|
||||||
|
has exactly one export there; with more, the command lists them and you pass one.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
||||||
|
occ backup:user:restore offsite alice
|
||||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
||||||
HELP);
|
HELP);
|
||||||
}
|
}
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
$mode = (string)$input->getOption('mode');
|
$mode = (string)$input->getOption('mode');
|
||||||
$uid = explode('/', (string)$input->getArgument('manifest'))[1] ?? '';
|
try {
|
||||||
if ($mode === UserRestoreService::MODE_REPLACE && $uid !== '' && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
$target = $this->targets->get($input->getArgument('target'));
|
||||||
|
$ref = (string)$input->getArgument('manifest');
|
||||||
|
$manifest = $this->targets->repository($target)->resolveUserExport($ref);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($manifest !== $ref) {
|
||||||
|
$output->writeln("Using $manifest.");
|
||||||
|
}
|
||||||
|
$uid = explode('/', $manifest)[1];
|
||||||
|
if ($mode === UserRestoreService::MODE_REPLACE && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
||||||
// Replace deletes the live account (after a safety export). Shares to others and data of
|
// Replace deletes the live account (after a safety export). Shares to others and data of
|
||||||
// apps without a user_migration migrator do not come back, so never do it unasked.
|
// apps without a user_migration migrator do not come back, so never do it unasked.
|
||||||
if (!$input->isInteractive()) {
|
if (!$input->isInteractive()) {
|
||||||
@@ -69,9 +84,8 @@ HELP);
|
|||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
$output->writeln('<comment>Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.</comment>');
|
$output->writeln('<comment>Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.</comment>');
|
||||||
$target = $this->targets->get($input->getArgument('target'));
|
|
||||||
$this->installer->ensureEnabled($input, $output);
|
$this->installer->ensureEnabled($input, $output);
|
||||||
$user = $this->userRestore->restoreUser($target, (string)$input->getArgument('manifest'), $mode);
|
$user = $this->userRestore->restoreUser($target, $manifest, $mode);
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
return 1;
|
return 1;
|
||||||
|
|||||||
@@ -47,6 +47,7 @@ final class AdminApiController extends Controller {
|
|||||||
private \OCA\NgBackup\Service\AlertService $alerts,
|
private \OCA\NgBackup\Service\AlertService $alerts,
|
||||||
private \OCA\NgBackup\Service\KeySlotService $slots,
|
private \OCA\NgBackup\Service\KeySlotService $slots,
|
||||||
private \OCA\NgBackup\Service\PruneService $prune,
|
private \OCA\NgBackup\Service\PruneService $prune,
|
||||||
|
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||||
) {
|
) {
|
||||||
parent::__construct(Application::APP_ID, $request);
|
parent::__construct(Application::APP_ID, $request);
|
||||||
}
|
}
|
||||||
@@ -72,7 +73,8 @@ final class AdminApiController extends Controller {
|
|||||||
'runs' => $runs,
|
'runs' => $runs,
|
||||||
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
|
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
|
||||||
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
|
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
|
||||||
'schedule' => $this->appConfig->getValueString(Application::APP_ID, 'schedule_time', ''),
|
'schedule' => $this->schedule->get()['time'],
|
||||||
|
'scheduleTimezone' => $this->schedule->get()['timezone'],
|
||||||
]);
|
]);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -240,12 +242,14 @@ final class AdminApiController extends Controller {
|
|||||||
return new JSONResponse($this->runJson($run));
|
return new JSONResponse($this->runJson($run));
|
||||||
}
|
}
|
||||||
|
|
||||||
public function setSchedule(string $time): JSONResponse {
|
/** @param string $timezone the administrator's zone (the browser's); empty: keep the current one */
|
||||||
if ($time !== '' && !preg_match('/^([01]\d|2[0-3]):[0-5]\d$/', $time)) {
|
public function setSchedule(string $time, string $timezone = ''): JSONResponse {
|
||||||
return new JSONResponse(['error' => 'Use HH:MM or empty'], Http::STATUS_BAD_REQUEST);
|
try {
|
||||||
|
$this->schedule->set($time, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
||||||
|
} catch (\InvalidArgumentException $e) {
|
||||||
|
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||||
}
|
}
|
||||||
$this->appConfig->setValueString(Application::APP_ID, 'schedule_time', $time);
|
return new JSONResponse(['schedule' => $time, 'scheduleTimezone' => $this->schedule->get()['timezone']]);
|
||||||
return new JSONResponse(['schedule' => $time]);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public function snapshots(int $targetId): JSONResponse {
|
public function snapshots(int $targetId): JSONResponse {
|
||||||
|
|||||||
+26
-3
@@ -159,7 +159,7 @@ final class DbDumper {
|
|||||||
// everywhere); tables it does not know are left out, as asked.
|
// everywhere); tables it does not know are left out, as asked.
|
||||||
foreach ($missing as $table) {
|
foreach ($missing as $table) {
|
||||||
if (isset($manifest['tables'][$table]['schema'])) {
|
if (isset($manifest['tables'][$table]['schema'])) {
|
||||||
$this->db->createSchemaManager()->createTable(self::tableFromStructure($table, $manifest['tables'][$table]['schema']));
|
$this->db->createSchemaManager()->createTable(self::tableFromStructure($table, $manifest['tables'][$table]['schema'], $this->tableOptions()));
|
||||||
$report['created'][] = $table;
|
$report['created'][] = $table;
|
||||||
} else {
|
} else {
|
||||||
$report['missing'][] = $table;
|
$report['missing'][] = $table;
|
||||||
@@ -319,8 +319,31 @@ final class DbDumper {
|
|||||||
'foreignKeys' => $foreignKeys];
|
'foreignKeys' => $foreignKeys];
|
||||||
}
|
}
|
||||||
|
|
||||||
private static function tableFromStructure(string $name, array $structure): \Doctrine\DBAL\Schema\Table {
|
/**
|
||||||
$table = new \Doctrine\DBAL\Schema\Table($name);
|
* Table options for a table the restore creates: those of this installation's own tables.
|
||||||
|
* On MySQL/MariaDB that is the character set and collation, as Nextcloud's tables have them
|
||||||
|
* (utf8mb4, so emoji and other 4-byte characters fit); DBAL alone would use 3-byte utf8.
|
||||||
|
*
|
||||||
|
* @return array<string, mixed>
|
||||||
|
* @psalm-suppress UndefinedClass the DBAL classes come with Nextcloud, not with this app
|
||||||
|
*/
|
||||||
|
private function tableOptions(): array {
|
||||||
|
$options = $this->db->createSchemaManager()->createSchemaConfig()->getDefaultTableOptions();
|
||||||
|
if ($this->db->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\AbstractMySQLPlatform) {
|
||||||
|
$collation = $this->db->fetchOne('SELECT TABLE_COLLATION FROM information_schema.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = ?',
|
||||||
|
[$this->prefix . 'appconfig']);
|
||||||
|
if (is_string($collation) && $collation !== '') {
|
||||||
|
$options['collation'] = $collation;
|
||||||
|
$options['charset'] = explode('_', $collation, 2)[0];
|
||||||
|
unset($options['collate']);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return $options;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param array<string, mixed> $options table options (see tableOptions()) */
|
||||||
|
private static function tableFromStructure(string $name, array $structure, array $options = []): \Doctrine\DBAL\Schema\Table {
|
||||||
|
$table = new \Doctrine\DBAL\Schema\Table($name, [], [], [], [], $options);
|
||||||
foreach ($structure['columns'] as $c) {
|
foreach ($structure['columns'] as $c) {
|
||||||
$table->addColumn($c['name'], $c['type'], array_filter([
|
$table->addColumn($c['name'], $c['type'], array_filter([
|
||||||
'length' => $c['length'], 'precision' => $c['precision'], 'scale' => $c['scale'],
|
'length' => $c['length'], 'precision' => $c['precision'], 'scale' => $c['scale'],
|
||||||
|
|||||||
@@ -561,6 +561,31 @@ final class Repository {
|
|||||||
return isset($this->catalog->userExports[$manifestPath]);
|
return isset($this->catalog->userExports[$manifestPath]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The recorded export $ref names: a manifest path (users/<uid>/<exportId>) as is, or a bare
|
||||||
|
* user id (also "users/<uid>") when that user has exactly one export here. Anything else
|
||||||
|
* throws, with the exports to choose from in the message.
|
||||||
|
*/
|
||||||
|
public function resolveUserExport(string $ref): string {
|
||||||
|
if ($this->isRecordedUserExport($ref)) {
|
||||||
|
return $ref;
|
||||||
|
}
|
||||||
|
$rest = trim($ref, '/');
|
||||||
|
if (str_starts_with($rest, 'users/')) {
|
||||||
|
$rest = substr($rest, strlen('users/'));
|
||||||
|
}
|
||||||
|
$uid = explode('/', $rest)[0];
|
||||||
|
$exports = $uid !== '' ? $this->listUserExports($uid) : [];
|
||||||
|
if (!str_contains($rest, '/') && count($exports) === 1) {
|
||||||
|
return $exports[0];
|
||||||
|
}
|
||||||
|
if ($exports === []) {
|
||||||
|
throw new RepositoryException("$ref is not a recorded user export for this location, and $uid has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports");
|
||||||
|
}
|
||||||
|
$which = str_contains($rest, '/') ? "$ref is not a recorded user export for this location" : "$uid has " . count($exports) . ' exports here';
|
||||||
|
throw new RepositoryException("$which; pass one of these (newest first):\n " . implode("\n ", $exports));
|
||||||
|
}
|
||||||
|
|
||||||
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
||||||
public function forgetUserExport(string $manifestPath): void {
|
public function forgetUserExport(string $manifestPath): void {
|
||||||
if (!isset($this->catalog->userExports[$manifestPath])) {
|
if (!isset($this->catalog->userExports[$manifestPath])) {
|
||||||
@@ -845,7 +870,15 @@ final class Repository {
|
|||||||
try {
|
try {
|
||||||
foreach ($this->backend->getRanges($ranges) as $i => $data) {
|
foreach ($this->backend->getRanges($ranges) as $i => $data) {
|
||||||
foreach ($parts[$i] as [$id, $at, $length]) {
|
foreach ($parts[$i] as [$id, $at, $length]) {
|
||||||
yield $this->decodeBlob($id, substr($data, $at, $length));
|
try {
|
||||||
|
$blob = $this->decodeBlob($id, substr($data, $at, $length));
|
||||||
|
} catch (\Exception) {
|
||||||
|
// a read that went wrong on the way (see loadBlob()): this blob on its own
|
||||||
|
$blob = $this->loadBlob($id);
|
||||||
|
// its pack stream is closed before the next range is read on the same connection
|
||||||
|
$this->packReader->close();
|
||||||
|
}
|
||||||
|
yield $blob;
|
||||||
$done++;
|
$done++;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -895,8 +928,46 @@ final class Repository {
|
|||||||
} catch (BackendException) {
|
} catch (BackendException) {
|
||||||
throw $e;
|
throw $e;
|
||||||
}
|
}
|
||||||
|
$pack = $moved;
|
||||||
}
|
}
|
||||||
return $this->decodeBlob($id, $encrypted);
|
try {
|
||||||
|
return $this->decodeBlob($id, $encrypted);
|
||||||
|
} catch (\Exception $damaged) {
|
||||||
|
// Bytes that do not decrypt: damaged on the location, or a read that went wrong on the
|
||||||
|
// way (a stream that lost its place: an SMB restore stopped after 17 hours like this).
|
||||||
|
// Read exactly this blob once more, on a request of its own; only a second failure
|
||||||
|
// means the data itself is damaged.
|
||||||
|
$this->packReader->close();
|
||||||
|
try {
|
||||||
|
$again = $this->backend->getRange('packs/' . substr($pack, 0, 2) . '/' . $pack, $offset, $length);
|
||||||
|
} catch (\Exception) {
|
||||||
|
throw $damaged;
|
||||||
|
}
|
||||||
|
$data = $this->decodeBlob($id, $again);
|
||||||
|
if (function_exists('OCP\Log\logger')) {
|
||||||
|
\OCP\Log\logger('ng_backup')->warning('NG Backup: blob ' . $id . ' did not decrypt as first read from pack ' . $pack . ' at ' . $offset . '; read again, it is fine. First read: ' . self::howItDiffered($encrypted, $again));
|
||||||
|
}
|
||||||
|
return $data;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What a read that went wrong looked like next to the right bytes, for the log: where it starts
|
||||||
|
* to differ, and whether the rest is zeros or the right bytes from another place (a stream that
|
||||||
|
* lost its place).
|
||||||
|
*/
|
||||||
|
private static function howItDiffered(string $wrong, string $right): string {
|
||||||
|
$at = strspn($wrong ^ $right, "\0");
|
||||||
|
$text = strlen($wrong) . ' bytes, differ from byte ' . $at . ' of ' . strlen($right);
|
||||||
|
$tail = substr($wrong, $at, 64);
|
||||||
|
if ($tail === '') {
|
||||||
|
return $text . ' (too short)';
|
||||||
|
}
|
||||||
|
if (trim($tail, "\0") === '') {
|
||||||
|
return $text . ' (zeros)';
|
||||||
|
}
|
||||||
|
$found = strpos($right, $tail);
|
||||||
|
return $text . ($found !== false ? ' (the right bytes from ' . ($found - $at) . ' bytes further on)' : ' (other bytes)');
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Decrypts and checks a blob read from its pack; throws when it is corrupted or tampered with. */
|
/** Decrypts and checks a blob read from its pack; throws when it is corrupted or tampered with. */
|
||||||
|
|||||||
@@ -0,0 +1,110 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\AppInfo\Application;
|
||||||
|
use OCP\IAppConfig;
|
||||||
|
use OCP\IConfig;
|
||||||
|
use OCP\IGroupManager;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The daily backup time: "HH:MM" in a time zone. Nextcloud runs PHP in UTC, so a time without a
|
||||||
|
* zone meant UTC (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone
|
||||||
|
* is converted once to the zone of the server or of an administrator, at today's offset: it then
|
||||||
|
* shows as it effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
||||||
|
*/
|
||||||
|
final class Schedule {
|
||||||
|
public const TIME_KEY = 'schedule_time';
|
||||||
|
public const ZONE_KEY = 'schedule_timezone';
|
||||||
|
|
||||||
|
public function __construct(
|
||||||
|
private IAppConfig $appConfig,
|
||||||
|
private IConfig $config,
|
||||||
|
private IGroupManager $groups,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @return array{time: string, timezone: non-empty-string} time "" = no schedule */
|
||||||
|
public function get(): array {
|
||||||
|
$time = $this->appConfig->getValueString(Application::APP_ID, self::TIME_KEY, '');
|
||||||
|
$zone = $this->appConfig->getValueString(Application::APP_ID, self::ZONE_KEY, '');
|
||||||
|
if ($zone === '' || !self::validZone($zone)) {
|
||||||
|
$zone = $this->guessZone();
|
||||||
|
if ($time !== '' && self::validTime($time)) {
|
||||||
|
$time = self::convert($time, 'UTC', $zone);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
|
||||||
|
}
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
|
||||||
|
}
|
||||||
|
return ['time' => $time, 'timezone' => $zone];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @throws \InvalidArgumentException for a malformed time or an unknown zone */
|
||||||
|
public function set(string $time, string $zone): void {
|
||||||
|
if ($time !== '' && !self::validTime($time)) {
|
||||||
|
throw new \InvalidArgumentException('Use HH:MM or empty');
|
||||||
|
}
|
||||||
|
if (!self::validZone($zone)) {
|
||||||
|
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
||||||
|
}
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
|
||||||
|
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Today's scheduled moment as a Unix time, in the schedule's zone; null without a schedule. */
|
||||||
|
public function todayAt(?int $now = null): ?int {
|
||||||
|
['time' => $time, 'timezone' => $zone] = $this->get();
|
||||||
|
if ($time === '' || !self::validTime($time)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
$tz = new \DateTimeZone($zone);
|
||||||
|
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone($tz)->format('Y-m-d');
|
||||||
|
return (new \DateTimeImmutable($day . ' ' . $time, $tz))->getTimestamp();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* "HH:MM" in zone $from as "HH:MM" in zone $to, at today's offsets.
|
||||||
|
*
|
||||||
|
* @param non-empty-string $from
|
||||||
|
* @param non-empty-string $to
|
||||||
|
*/
|
||||||
|
public static function convert(string $time, string $from, string $to, ?int $now = null): string {
|
||||||
|
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone(new \DateTimeZone($from))->format('Y-m-d');
|
||||||
|
return (new \DateTimeImmutable($day . ' ' . $time, new \DateTimeZone($from)))->setTimezone(new \DateTimeZone($to))->format('H:i');
|
||||||
|
}
|
||||||
|
|
||||||
|
public static function validTime(string $time): bool {
|
||||||
|
return preg_match('/^([01]\d|2[0-3]):[0-5]\d$/', $time) === 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @psalm-assert-if-true non-empty-string $zone */
|
||||||
|
public static function validZone(string $zone): bool {
|
||||||
|
return in_array($zone, \DateTimeZone::listIdentifiers(), true);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The server's default_timezone when set, else the first administrator's, else UTC. A server
|
||||||
|
* zone of UTC is passed over on purpose: it is mostly what an image or installer left there,
|
||||||
|
* while the administrator who set the time read it as their own (#19).
|
||||||
|
*
|
||||||
|
* @return non-empty-string
|
||||||
|
*/
|
||||||
|
private function guessZone(): string {
|
||||||
|
$zone = $this->config->getSystemValueString('default_timezone', '');
|
||||||
|
if ($zone !== '' && $zone !== 'UTC' && self::validZone($zone)) {
|
||||||
|
return $zone;
|
||||||
|
}
|
||||||
|
foreach ($this->groups->get('admin')?->getUsers() ?? [] as $user) {
|
||||||
|
$zone = $this->config->getUserValue($user->getUID(), 'core', 'timezone', '');
|
||||||
|
if ($zone !== '' && self::validZone($zone)) {
|
||||||
|
return $zone;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 'UTC';
|
||||||
|
}
|
||||||
|
}
|
||||||
+44
-3
@@ -7,7 +7,7 @@
|
|||||||
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
|
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
|
||||||
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
|
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
|
||||||
<p class="ngb-feedback">
|
<p class="ngb-feedback">
|
||||||
{{ t('ng_backup', 'NG Backup is in beta. We would like to hear how it works for you, also when everything went fine.') }}
|
{{ t('ng_backup', 'We would like to hear how NG Backup works for you, also when everything went fine.') }}
|
||||||
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
|
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
|
||||||
</p>
|
</p>
|
||||||
<NcLoadingIcon v-if="!status" :size="32" />
|
<NcLoadingIcon v-if="!status" :size="32" />
|
||||||
@@ -18,9 +18,18 @@
|
|||||||
<div class="ngb-tabs">
|
<div class="ngb-tabs">
|
||||||
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
|
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
|
||||||
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
|
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
|
||||||
|
<NcButton :variant="tab === 'docs' ? 'primary' : 'secondary'" @click="tab = 'docs'">{{ t('ng_backup', 'Docs') }}</NcButton>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<template v-if="tab === 'settings'">
|
<NcSettingsSection v-if="tab === 'docs'" :name="t('ng_backup', 'Documentation')">
|
||||||
|
<ul class="ngb-docs">
|
||||||
|
<li v-for="doc in docs" :key="doc.url">
|
||||||
|
<a :href="doc.url" target="_blank" rel="noopener noreferrer">{{ doc.name }} ↗</a>
|
||||||
|
<span>{{ doc.description }}</span>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</NcSettingsSection>
|
||||||
|
<template v-else-if="tab === 'settings'">
|
||||||
<KeySetup :status="status" @changed="refresh" />
|
<KeySetup :status="status" @changed="refresh" />
|
||||||
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
|
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
|
||||||
</template>
|
</template>
|
||||||
@@ -52,6 +61,25 @@ import RestoreBrowser from './components/RestoreBrowser.vue'
|
|||||||
// Opens the experience report form in .github/ISSUE_TEMPLATE
|
// Opens the experience report form in .github/ISSUE_TEMPLATE
|
||||||
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
|
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
|
||||||
|
|
||||||
|
const repoUrl = 'https://github.com/dillardblom/ng_backup'
|
||||||
|
const docs = [
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'How-to'),
|
||||||
|
description: t('ng_backup', 'Step by step: from first install to a tested restore.'),
|
||||||
|
url: repoUrl + '/blob/main/docs/HOWTO.md',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'Disaster recovery'),
|
||||||
|
description: t('ng_backup', 'Your server is gone: get Nextcloud back on a new server from a backup location.'),
|
||||||
|
url: repoUrl + '/blob/main/docs/DISASTER-RECOVERY.md',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'Roadmap'),
|
||||||
|
description: t('ng_backup', 'What is planned for the next versions.'),
|
||||||
|
url: repoUrl + '#roadmap',
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
const status = ref(null)
|
const status = ref(null)
|
||||||
const allRuns = ref(false) // runs older than the last days too
|
const allRuns = ref(false) // runs older than the last days too
|
||||||
const error = ref('')
|
const error = ref('')
|
||||||
@@ -89,8 +117,10 @@ onBeforeUnmount(() => clearTimeout(timer))
|
|||||||
</script>
|
</script>
|
||||||
|
|
||||||
<style scoped>
|
<style scoped>
|
||||||
|
/* NcSettingsSection sets its own width (at most 900px): the runs table needs more */
|
||||||
.ng-backup :deep(.settings-section) {
|
.ng-backup :deep(.settings-section) {
|
||||||
max-width: 900px;
|
width: min(1400px, 100% - var(--default-grid-baseline) * 7 * 2);
|
||||||
|
max-width: none;
|
||||||
}
|
}
|
||||||
.ngb-feedback {
|
.ngb-feedback {
|
||||||
color: var(--color-text-maxcontrast);
|
color: var(--color-text-maxcontrast);
|
||||||
@@ -100,6 +130,17 @@ onBeforeUnmount(() => clearTimeout(timer))
|
|||||||
color: var(--color-primary-element);
|
color: var(--color-primary-element);
|
||||||
text-decoration: underline;
|
text-decoration: underline;
|
||||||
}
|
}
|
||||||
|
.ngb-docs li {
|
||||||
|
margin-bottom: 8px;
|
||||||
|
}
|
||||||
|
.ngb-docs a {
|
||||||
|
color: var(--color-primary-element);
|
||||||
|
text-decoration: underline;
|
||||||
|
margin-right: 8px;
|
||||||
|
}
|
||||||
|
.ngb-docs span {
|
||||||
|
color: var(--color-text-maxcontrast);
|
||||||
|
}
|
||||||
.ngb-tabs {
|
.ngb-tabs {
|
||||||
display: flex;
|
display: flex;
|
||||||
gap: 8px;
|
gap: 8px;
|
||||||
|
|||||||
+1
-1
@@ -33,7 +33,7 @@ export default {
|
|||||||
testTarget: (id) => call('post', `/targets/${id}/test`),
|
testTarget: (id) => call('post', `/targets/${id}/test`),
|
||||||
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
|
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
|
||||||
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
|
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
|
||||||
setSchedule: (time) => call('put', '/schedule', { time }),
|
setSchedule: (time, timezone) => call('put', '/schedule', { time, timezone }),
|
||||||
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
|
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
|
||||||
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
|
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
|
||||||
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
|
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
|
||||||
|
|||||||
@@ -6,8 +6,8 @@
|
|||||||
<NcSettingsSection :name="t('ng_backup', 'Backups')"
|
<NcSettingsSection :name="t('ng_backup', 'Backups')"
|
||||||
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
|
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
|
||||||
<div class="ngb-row">
|
<div class="ngb-row">
|
||||||
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, empty = off)')" />
|
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, {zone}, empty = off)', { zone: browserZone })" />
|
||||||
<NcButton :disabled="schedule === status.schedule" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||||
</div>
|
</div>
|
||||||
<div class="ngb-row">
|
<div class="ngb-row">
|
||||||
<NcButton v-for="target in status.targets" :key="target.id" :disabled="isRunning(target.id)" @click="start(target)">
|
<NcButton v-for="target in status.targets" :key="target.id" :disabled="isRunning(target.id)" @click="start(target)">
|
||||||
@@ -79,15 +79,19 @@ import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
|||||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||||
import api from '../api.js'
|
import api from '../api.js'
|
||||||
import { formatDateTime, formatDate, formatSize } from '../format.js'
|
import { formatDateTime, formatDate, formatSize, toZone } from '../format.js'
|
||||||
import { confirmPassword } from '@nextcloud/password-confirmation'
|
import { confirmPassword } from '@nextcloud/password-confirmation'
|
||||||
import { reactive } from 'vue'
|
import { reactive } from 'vue'
|
||||||
|
|
||||||
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
|
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
|
||||||
const emit = defineEmits(['changed', 'show-all-runs'])
|
const emit = defineEmits(['changed', 'show-all-runs'])
|
||||||
const schedule = ref(props.status.schedule)
|
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
|
||||||
|
// zone is shown in this one, so it reads as the moment it runs here.
|
||||||
|
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
||||||
|
const shown = (s) => (s && props.status.scheduleTimezone && props.status.scheduleTimezone !== browserZone) ? toZone(s, props.status.scheduleTimezone, browserZone) : s
|
||||||
|
const schedule = ref(shown(props.status.schedule))
|
||||||
const error = ref('')
|
const error = ref('')
|
||||||
watch(() => props.status.schedule, (s) => { schedule.value = s })
|
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
|
||||||
|
|
||||||
const trash = reactive({})
|
const trash = reactive({})
|
||||||
async function loadTrash() {
|
async function loadTrash() {
|
||||||
@@ -131,7 +135,7 @@ async function start(target) {
|
|||||||
async function saveSchedule() {
|
async function saveSchedule() {
|
||||||
error.value = ''
|
error.value = ''
|
||||||
try {
|
try {
|
||||||
await api.setSchedule(schedule.value.trim())
|
await api.setSchedule(schedule.value.trim(), browserZone)
|
||||||
emit('changed')
|
emit('changed')
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
error.value = e.message
|
error.value = e.message
|
||||||
|
|||||||
@@ -26,3 +26,30 @@ export function formatSize(bytes) {
|
|||||||
}
|
}
|
||||||
return i === 0 ? `${b} B` : `${b.toFixed(1)} ${units[i]}`
|
return i === 0 ? `${b} B` : `${b.toFixed(1)} ${units[i]}`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A time of day "HH:MM" in zone `from`, as "HH:MM" in zone `to`, at today's offsets.
|
||||||
|
*
|
||||||
|
* @param {string} hhmm the time
|
||||||
|
* @param {string} from IANA zone it is in
|
||||||
|
* @param {string} to IANA zone to show it in
|
||||||
|
* @param {Date} now the day to convert on (today)
|
||||||
|
* @return {string}
|
||||||
|
*/
|
||||||
|
export function toZone(hhmm, from, to, now = new Date()) {
|
||||||
|
const [h, m] = hhmm.split(':').map(Number)
|
||||||
|
const parts = (zone, date) => Object.fromEntries(new Intl.DateTimeFormat('en-GB', {
|
||||||
|
timeZone: zone, hourCycle: 'h23', year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit',
|
||||||
|
}).formatToParts(date).map((p) => [p.type, p.value]))
|
||||||
|
const offsetAt = (at) => {
|
||||||
|
const seen = parts(from, new Date(at))
|
||||||
|
return Date.UTC(+seen.year, +seen.month - 1, +seen.day, +seen.hour, +seen.minute) - at
|
||||||
|
}
|
||||||
|
const today = parts(from, now)
|
||||||
|
const wall = Date.UTC(+today.year, +today.month - 1, +today.day, h, m)
|
||||||
|
// the offset at the wall time itself: on a day the clocks change it differs from the one at
|
||||||
|
// midnight, so it is looked up a second time at the moment found first
|
||||||
|
const at = wall - offsetAt(wall - offsetAt(wall))
|
||||||
|
const there = parts(to, new Date(at))
|
||||||
|
return `${there.hour}:${there.minute}`
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,35 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Backend;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\ExternalStorageFactory;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** ExternalStorageFactory::normalizePrivateKey(): keys that phpseclib 2 read, for phpseclib 3 (#24). */
|
||||||
|
class ExternalStorageFactoryTest extends TestCase {
|
||||||
|
/** An OpenSSH-format key laid out as ssh-keygen writes it; the body is random, not a real key. */
|
||||||
|
private static function key(): string {
|
||||||
|
$body = base64_encode(random_bytes(1000));
|
||||||
|
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRestoresLostLineBreaks(): void {
|
||||||
|
$key = self::key();
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey($key));
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", ' ', $key)), 'newlines as spaces');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", '', $key)), 'no newlines');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", "\r\n", $key)), 'CRLF');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(rtrim($key)), 'no trailing newline');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(preg_replace('/^/m', ' ', $key)), 'indented');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testLeavesOtherFormatsAlone(): void {
|
||||||
|
$pem = "-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\nDEK-Info: DES-EDE3-CBC,FF44E0A94036766C\n\nAAAA\n-----END RSA PRIVATE KEY-----\n";
|
||||||
|
$this->assertSame($pem, ExternalStorageFactory::normalizePrivateKey($pem));
|
||||||
|
$this->assertSame('not a key', ExternalStorageFactory::normalizePrivateKey('not a key'));
|
||||||
|
$odd = "-----BEGIN OPENSSH PRIVATE KEY-----\\nAAAA\\n-----END OPENSSH PRIVATE KEY-----";
|
||||||
|
$this->assertSame($odd, ExternalStorageFactory::normalizePrivateKey($odd), 'not base64 inside: unchanged');
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,133 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\IBackend;
|
||||||
|
use OCA\NgBackup\Backend\LocalBackend;
|
||||||
|
use OCA\NgBackup\Crypto\KeyRing;
|
||||||
|
use OCA\NgBackup\Repository\Repository;
|
||||||
|
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A pack stream that hands out wrong bytes once (an SMB restore stopped after 17 hours with
|
||||||
|
* "Decryption failed"): the blob is read once more on a request of its own; only data that is
|
||||||
|
* damaged on the location stops the restore.
|
||||||
|
*/
|
||||||
|
class DamagedReadTest extends TestCase {
|
||||||
|
use TempDirTrait;
|
||||||
|
|
||||||
|
private string $root;
|
||||||
|
private KeyRing $keys;
|
||||||
|
private string $src;
|
||||||
|
private string $snapshot;
|
||||||
|
|
||||||
|
protected function setUp(): void {
|
||||||
|
$this->root = $this->tempDir();
|
||||||
|
$this->keys = KeyRing::generate();
|
||||||
|
$this->src = $this->tempDir();
|
||||||
|
for ($i = 0; $i < 6; $i++) {
|
||||||
|
file_put_contents("$this->src/f$i", random_bytes(3 * 1048576));
|
||||||
|
}
|
||||||
|
$repo = Repository::initWithKey(new LocalBackend($this->root), $this->keys, $this->keys->wrap('pw'));
|
||||||
|
$this->snapshot = $repo->backupDirectory($this->src)['snapshot'];
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testABlobThatCameInDamagedOnceIsReadAgain(): void {
|
||||||
|
$backend = new GarblingBackend(new LocalBackend($this->root), streams: 1);
|
||||||
|
$target = $this->tempDir();
|
||||||
|
|
||||||
|
$this->assertSame(6, Repository::openWithKey($backend, $this->keys)->restore($this->snapshot, $target));
|
||||||
|
|
||||||
|
$this->assertSame(self::hashTree($this->src), self::hashTree($target));
|
||||||
|
$this->assertSame(0, $backend->streams, 'the damaged read happened');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testATableDumpThatCameInDamagedOnceIsReadAgain(): void {
|
||||||
|
$repo = Repository::openWithKey(new LocalBackend($this->root), $this->keys);
|
||||||
|
$stats = [];
|
||||||
|
$writer = $repo->blobWriter($stats);
|
||||||
|
$lines = [];
|
||||||
|
for ($i = 0; $i < 20000; $i++) {
|
||||||
|
$lines[] = json_encode(['i' => $i, 'pad' => str_repeat('x', $i % 400)]);
|
||||||
|
}
|
||||||
|
$writer->write(implode("\n", $lines) . "\n");
|
||||||
|
$blobs = $writer->finish();
|
||||||
|
$repo->flushPacks();
|
||||||
|
$backend = new GarblingBackend(new LocalBackend($this->root), ranges: 1);
|
||||||
|
|
||||||
|
$this->assertSame($lines, iterator_to_array(Repository::openWithKey($backend, $this->keys)->readLines($blobs), false));
|
||||||
|
$this->assertSame(0, $backend->ranges, 'the damaged read happened');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testDataDamagedOnTheLocationStillStopsTheRestore(): void {
|
||||||
|
$backend = new GarblingBackend(new LocalBackend($this->root), streams: 1000, ranges: 1000);
|
||||||
|
|
||||||
|
$this->expectException(\Exception::class);
|
||||||
|
Repository::openWithKey($backend, $this->keys)->restore($this->snapshot, $this->tempDir());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pack reads that flip a byte: the first $streams pack streams and $ranges range reads. */
|
||||||
|
final class GarblingBackend implements IBackend {
|
||||||
|
use \OCA\NgBackup\Backend\GetsOneByOne;
|
||||||
|
|
||||||
|
public function __construct(
|
||||||
|
private IBackend $inner,
|
||||||
|
public int $streams = 0,
|
||||||
|
public int $ranges = 0,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
public function put(string $path, $stream): void {
|
||||||
|
$this->inner->put($path, $stream);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function get(string $path) {
|
||||||
|
$fh = $this->inner->get($path);
|
||||||
|
if (!str_starts_with($path, 'packs/') || $this->streams <= 0) {
|
||||||
|
return $fh;
|
||||||
|
}
|
||||||
|
$this->streams--;
|
||||||
|
$data = stream_get_contents($fh);
|
||||||
|
fclose($fh);
|
||||||
|
for ($k = 100; $k < strlen($data); $k += 65536) {
|
||||||
|
$data[$k] = chr(ord($data[$k]) ^ 0x55);
|
||||||
|
}
|
||||||
|
$out = fopen('php://temp', 'w+b');
|
||||||
|
fwrite($out, $data);
|
||||||
|
rewind($out);
|
||||||
|
return $out;
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getRange(string $path, int $offset, int $length): string {
|
||||||
|
$data = $this->inner->getRange($path, $offset, $length);
|
||||||
|
if (str_starts_with($path, 'packs/') && $this->ranges > 0) {
|
||||||
|
$this->ranges--;
|
||||||
|
$data[0] = chr(ord($data[0]) ^ 0x55);
|
||||||
|
}
|
||||||
|
return $data;
|
||||||
|
}
|
||||||
|
|
||||||
|
public function exists(string $path): bool {
|
||||||
|
return $this->inner->exists($path);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function move(string $from, string $to): void {
|
||||||
|
$this->inner->move($from, $to);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function list(string $prefix): array {
|
||||||
|
return $this->inner->list($prefix);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function listFiles(string $dir): array {
|
||||||
|
return $this->inner->listFiles($dir);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function delete(string $path): void {
|
||||||
|
$this->inner->delete($path);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -106,4 +106,51 @@ class UserExportTest extends TestCase {
|
|||||||
$this->expectExceptionMessage('missing on the location');
|
$this->expectExceptionMessage('missing on the location');
|
||||||
$repo->listUserExports();
|
$repo->listUserExports();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testResolveTakesARecordedPathAsIs(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport($path));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveTakesAUserIdWithOneExport(): void {
|
||||||
|
// Issue #22: "backup:user:restore <location> New_User" was rejected as not a recorded export.
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
$this->putManifest($repo, 'users/alice2/20261005T120000Z-aaaa0002');
|
||||||
|
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport('alice'));
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport('users/alice/'));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveListsTheExportsOfAUserWithMore(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
$this->putManifest($repo, 'users/alice/20261006T120000Z-aaaa0002');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage("alice has 2 exports here; pass one of these (newest first):\n users/alice/20261006T120000Z-aaaa0002\n users/alice/20261005T120000Z-aaaa0001");
|
||||||
|
$repo->resolveUserExport('alice');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveListsTheExportsOfTheUserOfAWrongPath(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage("users/alice/typo is not a recorded user export for this location; pass one of these (newest first):\n users/alice/20261005T120000Z-aaaa0001");
|
||||||
|
$repo->resolveUserExport('users/alice/typo');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveNamesTheExpectedFormForAUserWithoutExports(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage('bob has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports');
|
||||||
|
$repo->resolveUserExport('bob');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\Schedule;
|
||||||
|
use OCP\IAppConfig;
|
||||||
|
use OCP\IConfig;
|
||||||
|
use OCP\IGroup;
|
||||||
|
use OCP\IGroupManager;
|
||||||
|
use OCP\IUser;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** The daily time in a time zone (issue #19), and the conversion of a time saved without one. */
|
||||||
|
class ScheduleTest extends TestCase {
|
||||||
|
private array $values = [];
|
||||||
|
|
||||||
|
private function schedule(string $serverZone = '', string $adminZone = ''): Schedule {
|
||||||
|
$app = $this->createMock(IAppConfig::class);
|
||||||
|
$app->method('getValueString')->willReturnCallback(fn (string $a, string $k, string $d = '') => $this->values[$k] ?? $d);
|
||||||
|
$app->method('setValueString')->willReturnCallback(function (string $a, string $k, string $v): bool {
|
||||||
|
$this->values[$k] = $v;
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
$config = $this->createMock(IConfig::class);
|
||||||
|
$config->method('getSystemValueString')->willReturn($serverZone);
|
||||||
|
$config->method('getUserValue')->willReturn($adminZone);
|
||||||
|
$user = $this->createMock(IUser::class);
|
||||||
|
$user->method('getUID')->willReturn('admin');
|
||||||
|
$group = $this->createMock(IGroup::class);
|
||||||
|
$group->method('getUsers')->willReturn([$user]);
|
||||||
|
$groups = $this->createMock(IGroupManager::class);
|
||||||
|
$groups->method('get')->willReturn($group);
|
||||||
|
return new Schedule($app, $config, $groups);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testATimeSavedWithoutAZoneShowsAsItRanInTheAdministratorsZone(): void {
|
||||||
|
$this->values = [Schedule::TIME_KEY => '02:30'];
|
||||||
|
|
||||||
|
$s = $this->schedule(adminZone: 'Europe/Berlin')->get();
|
||||||
|
|
||||||
|
// it ran at 02:30 UTC: in summer that is 04:30 in Berlin, in winter 03:30
|
||||||
|
$summer = Schedule::convert('02:30', 'UTC', 'Europe/Berlin', strtotime('2026-07-01 12:00 UTC'));
|
||||||
|
$this->assertSame('04:30', $summer);
|
||||||
|
$this->assertSame('Europe/Berlin', $s['timezone']);
|
||||||
|
$this->assertSame(Schedule::convert('02:30', 'UTC', 'Europe/Berlin'), $s['time']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testTheServersDefaultZoneComesFirst(): void {
|
||||||
|
$this->values = [Schedule::TIME_KEY => '01:00'];
|
||||||
|
$this->assertSame('America/Toronto', $this->schedule('America/Toronto', 'Europe/Berlin')->get()['timezone']);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testWithoutAnyZoneItStaysUtc(): void {
|
||||||
|
$this->values = [Schedule::TIME_KEY => '02:30'];
|
||||||
|
$this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get());
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testTodayAtIsTheMomentInTheSchedulesZone(): void {
|
||||||
|
$this->values = [Schedule::TIME_KEY => '03:00', Schedule::ZONE_KEY => 'Europe/Berlin'];
|
||||||
|
$at = $this->schedule()->todayAt(strtotime('2026-10-22 12:00 UTC'));
|
||||||
|
$this->assertSame(strtotime('2026-10-22 01:00 UTC'), $at, '03:00 in Berlin (summer time) is 01:00 UTC');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testAnUnknownZoneOrABadTimeIsRefused(): void {
|
||||||
|
$s = $this->schedule();
|
||||||
|
$this->expectException(\InvalidArgumentException::class);
|
||||||
|
$s->set('03:00', 'Mars/Olympus');
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -47,6 +47,12 @@ if [ "$LEGACY" = legacy ]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
|
run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
|
||||||
|
# The app store sends the whole app list (~38 MB) and Nextcloud gives up after 120 s: on a slow day
|
||||||
|
# an install fails for reasons outside this test. Up to three tries.
|
||||||
|
install_app() { # container app
|
||||||
|
for i in 1 2 3; do run_on "$1" app:install "$2" >/dev/null 2>&1 && return 0; sleep 20; done
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
# Database settings for an instance: SQLite inside the container, or its own PostgreSQL container
|
# Database settings for an instance: SQLite inside the container, or its own PostgreSQL container
|
||||||
db_env() {
|
db_env() {
|
||||||
@@ -109,7 +115,7 @@ if [ "$DB" != sqlite ]; then
|
|||||||
# while the restore loads tables in name order, accounts before mailboxes.
|
# while the restore loads tables in name order, accounts before mailboxes.
|
||||||
docker run -d --name ngb-frt-imap --network "$NET" -e MAILNAME=mail.domain.tld -e MAIL_ADDRESS=user@domain.tld \
|
docker run -d --name ngb-frt-imap --network "$NET" -e MAILNAME=mail.domain.tld -e MAIL_ADDRESS=user@domain.tld \
|
||||||
-e MAIL_PASS=mypassword ghcr.io/christophwurst/docker-imap-devel:latest >/dev/null
|
-e MAIL_PASS=mypassword ghcr.io/christophwurst/docker-imap-devel:latest >/dev/null
|
||||||
run_on ngb-frt-origin app:install mail >/dev/null || fail "install mail on the origin"
|
install_app ngb-frt-origin mail || fail "install mail on the origin (app store unreachable?)"
|
||||||
run_on ngb-frt-origin config:system:set app.mail.verify-tls-peer --type=boolean --value=false >/dev/null
|
run_on ngb-frt-origin config:system:set app.mail.verify-tls-peer --type=boolean --value=false >/dev/null
|
||||||
sleep 10
|
sleep 10
|
||||||
run_on ngb-frt-origin mail:account:create frtest Test user@domain.tld ngb-frt-imap 993 ssl user@domain.tld mypassword \
|
run_on ngb-frt-origin mail:account:create frtest Test user@domain.tld ngb-frt-imap 993 ssl user@domain.tld mypassword \
|
||||||
@@ -118,7 +124,9 @@ if [ "$DB" != sqlite ]; then
|
|||||||
[ "$(sql origin 'select count(*) from oc_mail_accounts where drafts_mailbox_id is not null')" = 1 ] \
|
[ "$(sql origin 'select count(*) from oc_mail_accounts where drafts_mailbox_id is not null')" = 1 ] \
|
||||||
|| fail "the mail account does not point at its mailboxes"
|
|| fail "the mail account does not point at its mailboxes"
|
||||||
# Deck installed and then switched off: its tables stay in the database and in the backup
|
# Deck installed and then switched off: its tables stay in the database and in the backup
|
||||||
run_on ngb-frt-origin app:install deck >/dev/null || fail "install deck on the origin"
|
install_app ngb-frt-origin deck || fail "install deck on the origin (app store unreachable?)"
|
||||||
|
# a 4-byte character (emoji) in a table the restore has to create: utf8mb4 on MySQL/MariaDB
|
||||||
|
sql origin "insert into oc_deck_boards (title, owner, color, archived, deleted_at, last_modified) values ('Plan 📌', 'frtest', '0087C5', false, 0, 0)" >/dev/null
|
||||||
run_on ngb-frt-origin app:disable deck >/dev/null
|
run_on ngb-frt-origin app:disable deck >/dev/null
|
||||||
ok "origin has a synced Mail account and the tables of a disabled app"
|
ok "origin has a synced Mail account and the tables of a disabled app"
|
||||||
fi
|
fi
|
||||||
@@ -202,6 +210,7 @@ if [ "$ORIGIN_DBTYPE" != sqlite ] && [ "$DB" != sqlite ] && [ "$LEGACY" = legacy
|
|||||||
ok "older backup: tables of the disabled app left out, Mail restored"
|
ok "older backup: tables of the disabled app left out, Mail restored"
|
||||||
elif [ "$ORIGIN_DBTYPE" != sqlite ] && [ "$DB" != sqlite ]; then
|
elif [ "$ORIGIN_DBTYPE" != sqlite ] && [ "$DB" != sqlite ]; then
|
||||||
[ "$(sql restore "select count(*) from information_schema.tables where table_name = 'oc_deck_boards'")" = 1 ] || fail "the tables of the disabled app were not created"
|
[ "$(sql restore "select count(*) from information_schema.tables where table_name = 'oc_deck_boards'")" = 1 ] || fail "the tables of the disabled app were not created"
|
||||||
|
[ "$(sql restore "select title from oc_deck_boards")" = "Plan 📌" ] || fail "a 4-byte character did not survive in a created table"
|
||||||
[ "$(sql restore "select count(*) from oc_mail_accounts where drafts_mailbox_id is not null")" = 1 ] || fail "the mail account was not restored"
|
[ "$(sql restore "select count(*) from oc_mail_accounts where drafts_mailbox_id is not null")" = 1 ] || fail "the mail account was not restored"
|
||||||
[ "$(sql restore "select count(*) from information_schema.table_constraints where constraint_type = 'FOREIGN KEY' and table_name like 'oc_mail_%'")" -gt 0 ] \
|
[ "$(sql restore "select count(*) from information_schema.table_constraints where constraint_type = 'FOREIGN KEY' and table_name like 'oc_mail_%'")" -gt 0 ] \
|
||||||
|| fail "the foreign keys of Mail are missing after the restore"
|
|| fail "the foreign keys of Mail are missing after the restore"
|
||||||
|
|||||||
@@ -36,8 +36,13 @@ MANIFEST=$(occ backup:user:backup ci "$U" | sed -n 's/.* to \(.*\)\.$/\1/p')
|
|||||||
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
||||||
ok "backed up $U to $MANIFEST"
|
ok "backed up $U to $MANIFEST"
|
||||||
|
|
||||||
# as-backup: restore into <uid>-bak; the original account must stay untouched.
|
occ backup:user:list ci "$U" | grep -qF "$MANIFEST" || fail "backup:user:list does not show $MANIFEST"
|
||||||
occ backup:user:restore ci "$MANIFEST" --mode=as-backup >/dev/null || fail "restore as-backup"
|
ok "backup:user:list shows the export"
|
||||||
|
|
||||||
|
# as-backup: restore into <uid>-bak; the original account must stay untouched. $U has one export,
|
||||||
|
# so the bare user id names it (#22).
|
||||||
|
OUT=$(occ backup:user:restore ci "$U" --mode=as-backup) || fail "restore as-backup by user id"
|
||||||
|
echo "$OUT" | grep -qF "Using $MANIFEST." || fail "restore by user id did not say which export it used"
|
||||||
occ files:scan "$U-bak" >/dev/null
|
occ files:scan "$U-bak" >/dev/null
|
||||||
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||||
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
||||||
@@ -55,6 +60,11 @@ SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xa
|
|||||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
||||||
ok "replace restores the original content under the same uid, byte-identical"
|
ok "replace restores the original content under the same uid, byte-identical"
|
||||||
|
|
||||||
|
# The replace above added a safety export: a bare user id is now ambiguous and must list both.
|
||||||
|
if OUT=$(occ backup:user:restore ci "$U" --mode=as-backup 2>&1); then fail "restore by an ambiguous user id ran"; fi
|
||||||
|
echo "$OUT" | grep -qF "$U has 2 exports here" && echo "$OUT" | grep -qF "$MANIFEST" || fail "an ambiguous user id did not list the exports"
|
||||||
|
ok "an ambiguous user id lists the exports instead of picking one"
|
||||||
|
|
||||||
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
||||||
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
||||||
# must forget exactly the 2 oldest.
|
# must forget exactly the 2 oldest.
|
||||||
|
|||||||
Reference in New Issue
Block a user