Compare commits
12 Commits
rc6
...
5cde57bc1b
| Author | SHA1 | Date | |
|---|---|---|---|
| 5cde57bc1b | |||
| 3eec2e9554 | |||
| cdf0b70e25 | |||
| 66615c2a88 | |||
| cc75c7128c | |||
| a0c4abb60a | |||
| 4847ce293d | |||
| 90dd97977a | |||
| 08534975d7 | |||
| 95e3dfcf6d | |||
| 7f4cae0e42 | |||
| 965440d1f7 |
@@ -41,6 +41,20 @@ jobs:
|
|||||||
- run: composer install --no-interaction --no-progress
|
- run: composer install --no-interaction --no-progress
|
||||||
- run: composer psalm
|
- run: composer psalm
|
||||||
|
|
||||||
|
frontend:
|
||||||
|
# Builds the admin page, so a broken Vue component fails CI instead of showing up after a release.
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: '22'
|
||||||
|
cache: npm
|
||||||
|
- run: npm ci --no-audit --no-fund
|
||||||
|
- run: npm run build
|
||||||
|
|
||||||
integration:
|
integration:
|
||||||
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
|
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
|
||||||
# restore, retention, trash) on a fresh Nextcloud per version and database.
|
# restore, retention, trash) on a fresh Nextcloud per version and database.
|
||||||
|
|||||||
+37
-2
@@ -7,11 +7,46 @@ the app store accepts a `-beta` suffix, but Nextcloud servers then only offer th
|
|||||||
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
||||||
instead carried by the app description, the git tag, the release, and this file.
|
instead carried by the app description, the git tag, the release, and this file.
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- `occ backup:user:list [location] [user]` shows the user exports on a location, with the manifest
|
||||||
|
path `backup:user:restore` expects. Before, that path was only printed once by
|
||||||
|
`backup:user:backup` (#22).
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- `occ backup:user:restore` also takes a user id instead of the manifest path, when that user has
|
||||||
|
exactly one export on the location. With more, and for a manifest path that is not on the
|
||||||
|
location, the error lists that user's exports; for a user without exports it names the
|
||||||
|
expected form (#22). Before, `backup:user:restore <location> <uid>` only said "is not a
|
||||||
|
recorded user export for this location".
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- An SFTP location with an RSA key in OpenSSH format ("BEGIN OPENSSH PRIVATE KEY") stopped
|
||||||
|
working after an upgrade to Nextcloud 35 with "Unable to read key", and such a key could not be
|
||||||
|
added there at all (#24). The "Private key" field on the admin page is a single line, so a
|
||||||
|
pasted key lost its line breaks; Nextcloud 34 still read it, Nextcloud 35 reads keys with a
|
||||||
|
stricter library. The line breaks are now restored before the key is used, so a stored location
|
||||||
|
works again without re-entering it.
|
||||||
|
- A full restore onto SQLite could stop at once with "database is locked" when another process
|
||||||
|
wrote to the database just as the restore began (a background job that was already running).
|
||||||
|
The restore now takes the write lock first and waits for the other process instead. The
|
||||||
|
full-restore test on SQLite runs with a second process writing alongside.
|
||||||
|
|
||||||
|
## [1.0.0] - 2026-10-29
|
||||||
|
|
||||||
|
First stable release. NG Backup makes encrypted, incremental backups of your whole Nextcloud
|
||||||
|
(files, database and configuration) to S3, SFTP, WebDAV, SMB and other storage, and restores them
|
||||||
|
from the web interface or with occ. Disaster recovery has been tested on real installations; see
|
||||||
|
"Tested disaster recoveries" in the README, and read "Known limitations" before you rely on it.
|
||||||
|
|
||||||
|
The code is the same as 1.0.0-rc.6 (app version 0.9.12): updating from it changes nothing in your
|
||||||
|
backups or settings.
|
||||||
|
|
||||||
## [1.0.0-rc.6] - 2026-10-24
|
## [1.0.0-rc.6] - 2026-10-24
|
||||||
|
|
||||||
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
||||||
simulations and a user report. If no problems are reported for about a week, 1.0.0 will be this
|
simulations and a user report.
|
||||||
same code.
|
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
||||||
|
|||||||
@@ -434,6 +434,40 @@ in README "Known limitations":
|
|||||||
Een restore ruimt `*.ngb-restore-*` in de mappen die hij terugzet op.
|
Een restore ruimt `*.ngb-restore-*` in de mappen die hij terugzet op.
|
||||||
7. **Eén bestand terugzetten met `backup:restore`**: een pad naar een bestand zet 0 bestanden
|
7. **Eén bestand terugzetten met `backup:restore`**: een pad naar een bestand zet 0 bestanden
|
||||||
terug (er komt een `/` achter); nu werkt alleen een map.
|
terug (er komt een `/` achter); nu werkt alleen een map.
|
||||||
|
8. **Uitleg (i) bij de prullenbak van herstelpunten** (besluit 2026-10-25): bij "removed by
|
||||||
|
retention policy" een infoknop met wat het betekent: het bewaarbeleid houdt voor oudere dagen
|
||||||
|
één herstelpunt per dag (het laatste), weken en maanden; de rest gaat naar de prullenbak en
|
||||||
|
wordt pas na de ingestelde wachttijd (`delete_delay_days`, standaard 30) echt verwijderd, en is
|
||||||
|
tot die datum terug te zetten. Aanleiding: de beheerder zag "19-10 13:12 – after rc4 upgrade
|
||||||
|
removed by retention policy" en dacht dat er iets was weggegooid.
|
||||||
|
9. **Tijdzone in `occ backup:status`**: de tabel toont de servertijd (UTC), de webinterface de
|
||||||
|
tijd van de beheerder; dat leest als een verkeerde starttijd. Zet de zone in de kolomkop
|
||||||
|
("started (UTC)") of toon de tijden in de zone van het schema.
|
||||||
|
10. **User-restore op gebruikersnaam** (issue #22, 2026-10-27): `backup:user:restore <locatie>
|
||||||
|
New_User` gaf alleen "is not a recorded user export"; het argument moet het manifestpad
|
||||||
|
zijn, dat alleen `backup:user:backup` eenmalig toonde. Nu: `backup:user:list` toont de
|
||||||
|
exports, een kale uid werkt bij precies één export, anders een lijst om uit te kiezen —
|
||||||
|
**klaar** (2026-10-27).
|
||||||
|
11. **Tabblad "Docs" in de beheerpagina** (besluit 2026-10-27): een derde knop naast "Backup and
|
||||||
|
restore" en "Settings", met links naar de HOWTO (`docs/HOWTO.md`), de DR-gids
|
||||||
|
(`docs/DISASTER-RECOVERY.md`) en de roadmap (README, `#roadmap`) op GitHub, elk met een
|
||||||
|
regel wat erin staat. Nu zijn ze alleen via de README te vinden. Zichtbaar ook zonder
|
||||||
|
sleutel, zodat de gids vindbaar is vóór de eerste installatie-stap. "NG Backup is in beta"
|
||||||
|
gaat uit de feedbackregel — **klaar** (2026-10-27).
|
||||||
|
12. **SFTP-sleutel zonder regeleinden** (issue #24, 2026-10-28): het veld "Private key" op de
|
||||||
|
beheerpagina is één regel, dus een geplakte sleutel verliest zijn regeleinden. NC34
|
||||||
|
(phpseclib 2) las dat nog, NC35 (phpseclib 3) weigert een OpenSSH-sleutel dan met "Unable
|
||||||
|
to read key". De regeleinden worden nu teruggezet vlak voor gebruik — **klaar** (2026-10-28).
|
||||||
|
13. **"database is locked" bij het begin van `backup:restore:full` op SQLite** (CI, 2026-10-26
|
||||||
|
bij #22 en 2026-10-29 op `main` na de merge van 1.0.0): de restore stopt meteen na het
|
||||||
|
aanzetten van de onderhoudsmodus. Niet altijd, dus een tweede proces dat op hetzelfde moment
|
||||||
|
de database gebruikt (cron, een webverzoek of de opstart van de container). Kan ook bij een
|
||||||
|
echte restore naar SQLite gebeuren. Oorzaak (2026-10-29): SQLite (WAL) begint een transactie
|
||||||
|
als lezer; schrijft een ander proces tussen ons eerste lezen en ons eerste schrijven, dan
|
||||||
|
faalt dat schrijven direct (geen wachttijd helpt). In CI was dat de installer, die
|
||||||
|
"installed" zet en daarna nog seconden doorschrijft. Nu neemt de restore eerst het
|
||||||
|
schrijfslot en wacht hij op de ander; de SQLite-test draait met een tweede schrijver
|
||||||
|
ernaast — **klaar** (2026-10-29).
|
||||||
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
|
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
|
||||||
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
|
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
|
||||||
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
|
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
|
||||||
@@ -470,6 +504,15 @@ in README "Known limitations":
|
|||||||
6. **Notitie achteraf bij een restorepunt**, te wijzigen in de lijst. Het label in de snapshot
|
6. **Notitie achteraf bij een restorepunt**, te wijzigen in de lijst. Het label in de snapshot
|
||||||
is versleuteld en ligt vast; de notitie apart op de locatie bewaren (bijv. in de catalogus),
|
is versleuteld en ligt vast; de notitie apart op de locatie bewaren (bijv. in de catalogus),
|
||||||
zodat een nieuwe server na een DR hem ook ziet.
|
zodat een nieuwe server na een DR hem ook ziet.
|
||||||
|
8. **Herstelpunt vastpinnen** (besluit 2026-10-25): een herstelpunt dat het bewaarbeleid
|
||||||
|
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
||||||
|
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
||||||
|
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
||||||
|
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
|
||||||
|
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
||||||
|
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
|
||||||
|
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
|
||||||
|
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
|
||||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||||
|
|||||||
@@ -7,8 +7,8 @@ installations and hosted Nextcloud. (Choosing SMB as a storage backend is the on
|
|||||||
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
see "Storage backend experiences" below — that requirement comes from files_external, not from
|
||||||
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
ng_backup, and applies to any Nextcloud app that writes to an SMB share.)
|
||||||
|
|
||||||
**Status: release candidate for 1.0.0 (1.0.0-rc.1, app version 0.9.7).** Tested thoroughly, usable
|
**Status: 1.0.0, stable.** Disaster recovery has been tested on real installations (see "Tested
|
||||||
alongside your current backup, but not yet a drop-in replacement for it. See "Known limitations" below, `PLAN.md` for the design and
|
disaster recoveries" below). See "Known limitations" below, `PLAN.md` for the design and
|
||||||
`docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
|
`docs/HOWTO.md` for a step-by-step guide from setup to a tested restore. Server lost? See
|
||||||
`docs/DISASTER-RECOVERY.md`.
|
`docs/DISASTER-RECOVERY.md`.
|
||||||
|
|
||||||
@@ -265,7 +265,11 @@ a 👍 on its issue helps decide what comes first.
|
|||||||
system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
|
system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
|
||||||
while the NG Backup page is open ([#13](https://github.com/dillardblom/ng_backup/issues/13)).
|
while the NG Backup page is open ([#13](https://github.com/dillardblom/ng_backup/issues/13)).
|
||||||
- An optional label for a backup started with "Back up now".
|
- An optional label for a backup started with "Back up now".
|
||||||
|
- An explanation in the trash of restore points of what "removed by retention policy" means and
|
||||||
|
until when a restore point can be brought back.
|
||||||
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
|
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
|
||||||
|
- A "Docs" tab on the admin page with links to the how-to, the disaster recovery guide and this
|
||||||
|
roadmap.
|
||||||
|
|
||||||
**1.1.0**
|
**1.1.0**
|
||||||
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
||||||
@@ -273,6 +277,8 @@ a 👍 on its issue helps decide what comes first.
|
|||||||
- A schedule per location, also "every N hours".
|
- A schedule per location, also "every N hours".
|
||||||
- Locations backed up one after the other instead of at the same time.
|
- Locations backed up one after the other instead of at the same time.
|
||||||
- A note on a restore point that you can change afterwards.
|
- A note on a restore point that you can change afterwards.
|
||||||
|
- Pinning a restore point, so the retention policy never removes it (for example the one from
|
||||||
|
before an upgrade).
|
||||||
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
||||||
Generator) while a backup runs.
|
Generator) while a backup runs.
|
||||||
- Faster backups: reading and uploading at the same time.
|
- Faster backups: reading and uploading at the same time.
|
||||||
@@ -287,7 +293,7 @@ a 👍 on its issue helps decide what comes first.
|
|||||||
|
|
||||||
## Known limitations
|
## Known limitations
|
||||||
|
|
||||||
On the backlog; none of these block the beta or 1.0.0, but know them before you rely on it:
|
On the backlog; know them before you rely on NG Backup:
|
||||||
|
|
||||||
- **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
|
- **Only the data directory is backed up.** Object storage as primary storage (`objectstore` in
|
||||||
`config.php`) is not supported: a backup refuses to start there. Folders mounted through the
|
`config.php`) is not supported: a backup refuses to start there. Folders mounted through the
|
||||||
@@ -330,4 +336,4 @@ On the backlog; none of these block the beta or 1.0.0, but know them before you
|
|||||||
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
||||||
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
||||||
| `backup:key:import` | import a recovery kit on a fresh installation |
|
| `backup:key:import` | import a recovery kit on a fresh installation |
|
||||||
| `backup:user:backup`, `backup:user:restore` | one user via user_migration (asks before installing it) |
|
| `backup:user:backup`, `backup:user:list`, `backup:user:restore` | one user via user_migration (asks before installing it); list the exports |
|
||||||
|
|||||||
+5
-4
@@ -42,7 +42,7 @@ welcome: use the "Share your experience" link on the admin page.
|
|||||||
|
|
||||||
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
||||||
you need help with restoring a backup after a crash.]]></description>
|
you need help with restoring a backup after a crash.]]></description>
|
||||||
<version>0.9.12</version>
|
<version>1.0.0</version>
|
||||||
<licence>agpl</licence>
|
<licence>agpl</licence>
|
||||||
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
||||||
<namespace>NgBackup</namespace>
|
<namespace>NgBackup</namespace>
|
||||||
@@ -53,9 +53,9 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<website>https://github.com/dillardblom/ng_backup</website>
|
<website>https://github.com/dillardblom/ng_backup</website>
|
||||||
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
||||||
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/1-backups.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0/screenshots/1-backups.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/2-restore.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0/screenshots/2-restore.png</screenshot>
|
||||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/3-settings.png</screenshot>
|
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0/screenshots/3-settings.png</screenshot>
|
||||||
<dependencies>
|
<dependencies>
|
||||||
<php min-version="8.1"/>
|
<php min-version="8.1"/>
|
||||||
<lib>sodium</lib>
|
<lib>sodium</lib>
|
||||||
@@ -92,6 +92,7 @@ you need help with restoring a backup after a crash.]]></description>
|
|||||||
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
||||||
<command>OCA\NgBackup\Command\BackupUser</command>
|
<command>OCA\NgBackup\Command\BackupUser</command>
|
||||||
<command>OCA\NgBackup\Command\RestoreUser</command>
|
<command>OCA\NgBackup\Command\RestoreUser</command>
|
||||||
|
<command>OCA\NgBackup\Command\ListUserExports</command>
|
||||||
<command>OCA\NgBackup\Command\KeyImport</command>
|
<command>OCA\NgBackup\Command\KeyImport</command>
|
||||||
<command>OCA\NgBackup\Command\RestoreFull</command>
|
<command>OCA\NgBackup\Command\RestoreFull</command>
|
||||||
<command>OCA\NgBackup\Command\Verify</command>
|
<command>OCA\NgBackup\Command\Verify</command>
|
||||||
|
|||||||
@@ -114,10 +114,16 @@ Needs the `user_migration` app. NG Backup asks before installing it:
|
|||||||
|
|
||||||
```
|
```
|
||||||
occ backup:user:backup offsite <uid>
|
occ backup:user:backup offsite <uid>
|
||||||
|
occ backup:user:list offsite [<uid>] # the exports and their manifest
|
||||||
occ backup:user:restore offsite users/<uid>/<exportId> # replace (default)
|
occ backup:user:restore offsite users/<uid>/<exportId> # replace (default)
|
||||||
occ backup:user:restore offsite users/<uid>/<exportId> --mode=as-backup
|
occ backup:user:restore offsite users/<uid>/<exportId> --mode=as-backup
|
||||||
```
|
```
|
||||||
|
|
||||||
|
- `backup:user:restore` needs the manifest path that `backup:user:backup` prints at the end
|
||||||
|
(`users/<uid>/<exportId>`); `backup:user:list` shows it again. A user id alone also works when
|
||||||
|
that user has exactly one export on the location. With more (a `replace` restore adds a safety
|
||||||
|
export), the command lists them and you pass one.
|
||||||
|
|
||||||
- If `user_migration` is missing, the command asks to install and enable it. For non-interactive
|
- If `user_migration` is missing, the command asks to install and enable it. For non-interactive
|
||||||
use, add `--install-user-migration` to approve that explicitly. The command stops after the
|
use, add `--install-user-migration` to approve that explicitly. The command stops after the
|
||||||
installation; run it once more.
|
installation; run it once more.
|
||||||
|
|||||||
@@ -72,6 +72,10 @@ final class ExternalStorageFactory {
|
|||||||
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
|
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (is_string($options['private_key'] ?? null)) {
|
||||||
|
$options['private_key'] = self::normalizePrivateKey($options['private_key']);
|
||||||
|
}
|
||||||
|
|
||||||
$config = new StorageConfig();
|
$config = new StorageConfig();
|
||||||
$config->setBackend($backend);
|
$config->setBackend($backend);
|
||||||
$config->setAuthMechanism($auth);
|
$config->setAuthMechanism($auth);
|
||||||
@@ -86,4 +90,22 @@ final class ExternalStorageFactory {
|
|||||||
$storage = $backend->wrapStorage($storage);
|
$storage = $backend->wrapStorage($storage);
|
||||||
return $auth->wrapStorage($storage);
|
return $auth->wrapStorage($storage);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An OpenSSH-format private key with its line breaks restored. Up to Nextcloud 34
|
||||||
|
* (phpseclib 2) a key whose newlines were lost or turned into spaces, e.g. on its way through
|
||||||
|
* a shell or a form, still loaded; Nextcloud 35 (phpseclib 3) refuses it with "Unable to read
|
||||||
|
* key" (#24). The key itself is not changed, so it is fixed when it is used, not when stored.
|
||||||
|
* Other formats (PEM, PKCS#8) still load as they are and are returned unchanged.
|
||||||
|
*/
|
||||||
|
public static function normalizePrivateKey(string $key): string {
|
||||||
|
if (!preg_match('/-----BEGIN OPENSSH PRIVATE KEY-----(.*?)-----END OPENSSH PRIVATE KEY-----/s', $key, $m)) {
|
||||||
|
return $key;
|
||||||
|
}
|
||||||
|
$body = (string)preg_replace('/\s+/', '', $m[1]);
|
||||||
|
if (!preg_match('#^[A-Za-z0-9+/]+=*$#', $body)) {
|
||||||
|
return $key;
|
||||||
|
}
|
||||||
|
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Command;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Service\TargetService;
|
||||||
|
use Symfony\Component\Console\Command\Command;
|
||||||
|
use Symfony\Component\Console\Helper\Table;
|
||||||
|
use Symfony\Component\Console\Input\InputArgument;
|
||||||
|
use Symfony\Component\Console\Input\InputInterface;
|
||||||
|
use Symfony\Component\Console\Output\OutputInterface;
|
||||||
|
|
||||||
|
final class ListUserExports extends Command {
|
||||||
|
public function __construct(
|
||||||
|
private TargetService $targets,
|
||||||
|
) {
|
||||||
|
parent::__construct();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function configure(): void {
|
||||||
|
$this->setName('backup:user:list')
|
||||||
|
->setDescription('List the user exports (backup:user:backup) on a location')
|
||||||
|
->addArgument('target', InputArgument::OPTIONAL, 'Location name or id (default: all)')
|
||||||
|
->addArgument('user', InputArgument::OPTIONAL, 'Only the exports of this user id')
|
||||||
|
->setHelp(<<<'HELP'
|
||||||
|
The manifest column is what occ backup:user:restore expects. A safety export made by a restore
|
||||||
|
with --mode=replace is listed here as well.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
occ backup:user:list offsite alice
|
||||||
|
HELP);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
|
$name = $input->getArgument('target');
|
||||||
|
$uid = $input->getArgument('user');
|
||||||
|
$rows = [];
|
||||||
|
try {
|
||||||
|
$targets = $name === null ? $this->targets->list() : [$this->targets->get($name)];
|
||||||
|
foreach ($targets as $t) {
|
||||||
|
foreach ($this->targets->repository($t)->listUserExports($uid) as $path) {
|
||||||
|
$parts = explode('/', $path, 3);
|
||||||
|
$user = $parts[1] ?? '';
|
||||||
|
$exportId = $parts[2] ?? '';
|
||||||
|
$time = \DateTimeImmutable::createFromFormat('Ymd\THis\Z', substr($exportId, 0, 16), new \DateTimeZone('UTC'));
|
||||||
|
$rows[] = [$t->getName(), $user, $time !== false ? $time->format('Y-m-d H:i') : '', $path];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($rows === []) {
|
||||||
|
$output->writeln('No user exports yet (occ backup:user:backup).');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
(new Table($output))->setHeaders(['location', 'user', 'date (UTC)', 'manifest'])->setRows($rows)->render();
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -33,7 +33,7 @@ final class RestoreUser extends Command {
|
|||||||
$this->setName('backup:user:restore')
|
$this->setName('backup:user:restore')
|
||||||
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
||||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||||
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>)')
|
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>), or a user id with exactly one export')
|
||||||
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
||||||
->addOption('force', null, InputOption::VALUE_NONE, 'Replace an existing account without asking (required when not interactive)')
|
->addOption('force', null, InputOption::VALUE_NONE, 'Replace an existing account without asking (required when not interactive)')
|
||||||
->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)')
|
->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)')
|
||||||
@@ -45,16 +45,31 @@ Never imports over a live account. Modes:
|
|||||||
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
||||||
the new name, never the existing account
|
the new name, never the existing account
|
||||||
|
|
||||||
|
occ backup:user:list shows the exports on a location. A user id alone works when that user
|
||||||
|
has exactly one export there; with more, the command lists them and you pass one.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
||||||
|
occ backup:user:restore offsite alice
|
||||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
||||||
HELP);
|
HELP);
|
||||||
}
|
}
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||||
$mode = (string)$input->getOption('mode');
|
$mode = (string)$input->getOption('mode');
|
||||||
$uid = explode('/', (string)$input->getArgument('manifest'))[1] ?? '';
|
try {
|
||||||
if ($mode === UserRestoreService::MODE_REPLACE && $uid !== '' && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
$target = $this->targets->get($input->getArgument('target'));
|
||||||
|
$ref = (string)$input->getArgument('manifest');
|
||||||
|
$manifest = $this->targets->repository($target)->resolveUserExport($ref);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if ($manifest !== $ref) {
|
||||||
|
$output->writeln("Using $manifest.");
|
||||||
|
}
|
||||||
|
$uid = explode('/', $manifest)[1];
|
||||||
|
if ($mode === UserRestoreService::MODE_REPLACE && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
||||||
// Replace deletes the live account (after a safety export). Shares to others and data of
|
// Replace deletes the live account (after a safety export). Shares to others and data of
|
||||||
// apps without a user_migration migrator do not come back, so never do it unasked.
|
// apps without a user_migration migrator do not come back, so never do it unasked.
|
||||||
if (!$input->isInteractive()) {
|
if (!$input->isInteractive()) {
|
||||||
@@ -69,9 +84,8 @@ HELP);
|
|||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
$output->writeln('<comment>Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.</comment>');
|
$output->writeln('<comment>Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.</comment>');
|
||||||
$target = $this->targets->get($input->getArgument('target'));
|
|
||||||
$this->installer->ensureEnabled($input, $output);
|
$this->installer->ensureEnabled($input, $output);
|
||||||
$user = $this->userRestore->restoreUser($target, (string)$input->getArgument('manifest'), $mode);
|
$user = $this->userRestore->restoreUser($target, $manifest, $mode);
|
||||||
} catch (\Throwable $e) {
|
} catch (\Throwable $e) {
|
||||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||||
return 1;
|
return 1;
|
||||||
|
|||||||
+7
-1
@@ -169,7 +169,7 @@ final class DbDumper {
|
|||||||
$foreignKeys = [];
|
$foreignKeys = [];
|
||||||
$this->db->beginTransaction();
|
$this->db->beginTransaction();
|
||||||
try {
|
try {
|
||||||
$foreignKeys = $this->suspendForeignKeys();
|
$foreignKeys = $this->suspendForeignKeys(); // first: on SQLite it also takes the write lock
|
||||||
foreach ($manifest['tables'] as $table => $info) {
|
foreach ($manifest['tables'] as $table => $info) {
|
||||||
if (in_array($table, $report['missing'], true)) {
|
if (in_array($table, $report['missing'], true)) {
|
||||||
continue;
|
continue;
|
||||||
@@ -412,6 +412,12 @@ final class DbDumper {
|
|||||||
return $checks;
|
return $checks;
|
||||||
}
|
}
|
||||||
if ($platform instanceof \Doctrine\DBAL\Platforms\SqlitePlatform) {
|
if ($platform instanceof \Doctrine\DBAL\Platforms\SqlitePlatform) {
|
||||||
|
// SQLite (WAL) starts a transaction as a reader. When another process (a cron job, the
|
||||||
|
// installer still finishing) writes between our first read and our first write, that
|
||||||
|
// write fails at once with "database is locked": no waiting helps a reader whose view
|
||||||
|
// is out of date. Take the write lock first, so we wait for the other writer instead
|
||||||
|
// and it waits for us after that.
|
||||||
|
$this->db->executeStatement('DELETE FROM ' . $this->db->quoteIdentifier($this->prefix . 'appconfig') . ' WHERE 0 = 1');
|
||||||
$this->db->executeStatement('PRAGMA defer_foreign_keys = ON'); // checked at commit
|
$this->db->executeStatement('PRAGMA defer_foreign_keys = ON'); // checked at commit
|
||||||
}
|
}
|
||||||
return [];
|
return [];
|
||||||
|
|||||||
@@ -561,6 +561,31 @@ final class Repository {
|
|||||||
return isset($this->catalog->userExports[$manifestPath]);
|
return isset($this->catalog->userExports[$manifestPath]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The recorded export $ref names: a manifest path (users/<uid>/<exportId>) as is, or a bare
|
||||||
|
* user id (also "users/<uid>") when that user has exactly one export here. Anything else
|
||||||
|
* throws, with the exports to choose from in the message.
|
||||||
|
*/
|
||||||
|
public function resolveUserExport(string $ref): string {
|
||||||
|
if ($this->isRecordedUserExport($ref)) {
|
||||||
|
return $ref;
|
||||||
|
}
|
||||||
|
$rest = trim($ref, '/');
|
||||||
|
if (str_starts_with($rest, 'users/')) {
|
||||||
|
$rest = substr($rest, strlen('users/'));
|
||||||
|
}
|
||||||
|
$uid = explode('/', $rest)[0];
|
||||||
|
$exports = $uid !== '' ? $this->listUserExports($uid) : [];
|
||||||
|
if (!str_contains($rest, '/') && count($exports) === 1) {
|
||||||
|
return $exports[0];
|
||||||
|
}
|
||||||
|
if ($exports === []) {
|
||||||
|
throw new RepositoryException("$ref is not a recorded user export for this location, and $uid has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports");
|
||||||
|
}
|
||||||
|
$which = str_contains($rest, '/') ? "$ref is not a recorded user export for this location" : "$uid has " . count($exports) . ' exports here';
|
||||||
|
throw new RepositoryException("$which; pass one of these (newest first):\n " . implode("\n ", $exports));
|
||||||
|
}
|
||||||
|
|
||||||
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
||||||
public function forgetUserExport(string $manifestPath): void {
|
public function forgetUserExport(string $manifestPath): void {
|
||||||
if (!isset($this->catalog->userExports[$manifestPath])) {
|
if (!isset($this->catalog->userExports[$manifestPath])) {
|
||||||
|
|||||||
+41
-2
@@ -7,7 +7,7 @@
|
|||||||
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
|
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
|
||||||
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
|
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
|
||||||
<p class="ngb-feedback">
|
<p class="ngb-feedback">
|
||||||
{{ t('ng_backup', 'NG Backup is in beta. We would like to hear how it works for you, also when everything went fine.') }}
|
{{ t('ng_backup', 'We would like to hear how NG Backup works for you, also when everything went fine.') }}
|
||||||
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
|
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
|
||||||
</p>
|
</p>
|
||||||
<NcLoadingIcon v-if="!status" :size="32" />
|
<NcLoadingIcon v-if="!status" :size="32" />
|
||||||
@@ -18,9 +18,18 @@
|
|||||||
<div class="ngb-tabs">
|
<div class="ngb-tabs">
|
||||||
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
|
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
|
||||||
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
|
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
|
||||||
|
<NcButton :variant="tab === 'docs' ? 'primary' : 'secondary'" @click="tab = 'docs'">{{ t('ng_backup', 'Docs') }}</NcButton>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<template v-if="tab === 'settings'">
|
<NcSettingsSection v-if="tab === 'docs'" :name="t('ng_backup', 'Documentation')">
|
||||||
|
<ul class="ngb-docs">
|
||||||
|
<li v-for="doc in docs" :key="doc.url">
|
||||||
|
<a :href="doc.url" target="_blank" rel="noopener noreferrer">{{ doc.name }} ↗</a>
|
||||||
|
<span>{{ doc.description }}</span>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</NcSettingsSection>
|
||||||
|
<template v-else-if="tab === 'settings'">
|
||||||
<KeySetup :status="status" @changed="refresh" />
|
<KeySetup :status="status" @changed="refresh" />
|
||||||
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
|
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
|
||||||
</template>
|
</template>
|
||||||
@@ -52,6 +61,25 @@ import RestoreBrowser from './components/RestoreBrowser.vue'
|
|||||||
// Opens the experience report form in .github/ISSUE_TEMPLATE
|
// Opens the experience report form in .github/ISSUE_TEMPLATE
|
||||||
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
|
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
|
||||||
|
|
||||||
|
const repoUrl = 'https://github.com/dillardblom/ng_backup'
|
||||||
|
const docs = [
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'How-to'),
|
||||||
|
description: t('ng_backup', 'Step by step: from first install to a tested restore.'),
|
||||||
|
url: repoUrl + '/blob/main/docs/HOWTO.md',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'Disaster recovery'),
|
||||||
|
description: t('ng_backup', 'Your server is gone: get Nextcloud back on a new server from a backup location.'),
|
||||||
|
url: repoUrl + '/blob/main/docs/DISASTER-RECOVERY.md',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: t('ng_backup', 'Roadmap'),
|
||||||
|
description: t('ng_backup', 'What is planned for the next versions.'),
|
||||||
|
url: repoUrl + '#roadmap',
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
const status = ref(null)
|
const status = ref(null)
|
||||||
const allRuns = ref(false) // runs older than the last days too
|
const allRuns = ref(false) // runs older than the last days too
|
||||||
const error = ref('')
|
const error = ref('')
|
||||||
@@ -102,6 +130,17 @@ onBeforeUnmount(() => clearTimeout(timer))
|
|||||||
color: var(--color-primary-element);
|
color: var(--color-primary-element);
|
||||||
text-decoration: underline;
|
text-decoration: underline;
|
||||||
}
|
}
|
||||||
|
.ngb-docs li {
|
||||||
|
margin-bottom: 8px;
|
||||||
|
}
|
||||||
|
.ngb-docs a {
|
||||||
|
color: var(--color-primary-element);
|
||||||
|
text-decoration: underline;
|
||||||
|
margin-right: 8px;
|
||||||
|
}
|
||||||
|
.ngb-docs span {
|
||||||
|
color: var(--color-text-maxcontrast);
|
||||||
|
}
|
||||||
.ngb-tabs {
|
.ngb-tabs {
|
||||||
display: flex;
|
display: flex;
|
||||||
gap: 8px;
|
gap: 8px;
|
||||||
|
|||||||
@@ -0,0 +1,35 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace OCA\NgBackup\Tests\Unit\Backend;
|
||||||
|
|
||||||
|
use OCA\NgBackup\Backend\ExternalStorageFactory;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
/** ExternalStorageFactory::normalizePrivateKey(): keys that phpseclib 2 read, for phpseclib 3 (#24). */
|
||||||
|
class ExternalStorageFactoryTest extends TestCase {
|
||||||
|
/** An OpenSSH-format key laid out as ssh-keygen writes it; the body is random, not a real key. */
|
||||||
|
private static function key(): string {
|
||||||
|
$body = base64_encode(random_bytes(1000));
|
||||||
|
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testRestoresLostLineBreaks(): void {
|
||||||
|
$key = self::key();
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey($key));
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", ' ', $key)), 'newlines as spaces');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", '', $key)), 'no newlines');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", "\r\n", $key)), 'CRLF');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(rtrim($key)), 'no trailing newline');
|
||||||
|
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(preg_replace('/^/m', ' ', $key)), 'indented');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testLeavesOtherFormatsAlone(): void {
|
||||||
|
$pem = "-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\nDEK-Info: DES-EDE3-CBC,FF44E0A94036766C\n\nAAAA\n-----END RSA PRIVATE KEY-----\n";
|
||||||
|
$this->assertSame($pem, ExternalStorageFactory::normalizePrivateKey($pem));
|
||||||
|
$this->assertSame('not a key', ExternalStorageFactory::normalizePrivateKey('not a key'));
|
||||||
|
$odd = "-----BEGIN OPENSSH PRIVATE KEY-----\\nAAAA\\n-----END OPENSSH PRIVATE KEY-----";
|
||||||
|
$this->assertSame($odd, ExternalStorageFactory::normalizePrivateKey($odd), 'not base64 inside: unchanged');
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -106,4 +106,51 @@ class UserExportTest extends TestCase {
|
|||||||
$this->expectExceptionMessage('missing on the location');
|
$this->expectExceptionMessage('missing on the location');
|
||||||
$repo->listUserExports();
|
$repo->listUserExports();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function testResolveTakesARecordedPathAsIs(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport($path));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveTakesAUserIdWithOneExport(): void {
|
||||||
|
// Issue #22: "backup:user:restore <location> New_User" was rejected as not a recorded export.
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||||
|
$this->putManifest($repo, $path);
|
||||||
|
$this->putManifest($repo, 'users/alice2/20261005T120000Z-aaaa0002');
|
||||||
|
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport('alice'));
|
||||||
|
$this->assertSame($path, $repo->resolveUserExport('users/alice/'));
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveListsTheExportsOfAUserWithMore(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
$this->putManifest($repo, 'users/alice/20261006T120000Z-aaaa0002');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage("alice has 2 exports here; pass one of these (newest first):\n users/alice/20261006T120000Z-aaaa0002\n users/alice/20261005T120000Z-aaaa0001");
|
||||||
|
$repo->resolveUserExport('alice');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveListsTheExportsOfTheUserOfAWrongPath(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage("users/alice/typo is not a recorded user export for this location; pass one of these (newest first):\n users/alice/20261005T120000Z-aaaa0001");
|
||||||
|
$repo->resolveUserExport('users/alice/typo');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function testResolveNamesTheExpectedFormForAUserWithoutExports(): void {
|
||||||
|
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||||
|
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||||
|
|
||||||
|
$this->expectException(RepositoryException::class);
|
||||||
|
$this->expectExceptionMessage('bob has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports');
|
||||||
|
$repo->resolveUserExport('bob');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -173,6 +173,20 @@ if [ "$LEGACY" = legacy ] && [ "$DB" != sqlite ]; then
|
|||||||
ok "older backup refused without --skip-missing-tables, missing tables named"
|
ok "older backup refused without --skip-missing-tables, missing tables named"
|
||||||
RESTORE_OPTS+=(--skip-missing-tables)
|
RESTORE_OPTS+=(--skip-missing-tables)
|
||||||
fi
|
fi
|
||||||
|
if [ "$DB" = sqlite ]; then
|
||||||
|
# Another process writing while the restore starts (a cron job that was already running, or
|
||||||
|
# the installer still finishing after it says "installed"): the restore must wait for it, not
|
||||||
|
# stop with "database is locked". Short write transactions on a table of its own, until the
|
||||||
|
# restore is done (/tmp/frt-side-stop; Nextcloud's own commands after it are not under test).
|
||||||
|
docker exec -d -u www-data ngb-frt-restore php -r '
|
||||||
|
$p = new PDO("sqlite:/var/www/html/data/nextcloud.db"); $p->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
||||||
|
$p->exec("CREATE TABLE IF NOT EXISTS frt_side_writer (t REAL)");
|
||||||
|
for ($end = microtime(true) + 300; microtime(true) < $end && !file_exists("/tmp/frt-side-stop");) {
|
||||||
|
$p->beginTransaction(); $p->exec("INSERT INTO frt_side_writer VALUES (" . microtime(true) . ")"); usleep(300000); $p->commit(); usleep(20000);
|
||||||
|
}
|
||||||
|
touch("/tmp/frt-side-done");'
|
||||||
|
sleep 1
|
||||||
|
fi
|
||||||
sftp_logins() { docker logs ngb-frt-sftp 2>&1 | grep -c "Accepted password" || true; }
|
sftp_logins() { docker logs ngb-frt-sftp 2>&1 | grep -c "Accepted password" || true; }
|
||||||
[ "$MODE" = sftp ] && LOGINS_BEFORE=$(sftp_logins)
|
[ "$MODE" = sftp ] && LOGINS_BEFORE=$(sftp_logins)
|
||||||
OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" "${RESTORE_OPTS[@]}" 2>&1) || {
|
OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" "${RESTORE_OPTS[@]}" 2>&1) || {
|
||||||
@@ -184,6 +198,13 @@ printf '%s\n' "$OUT" | grep -q "maintenance mode while the restore runs" || fail
|
|||||||
run_on ngb-frt-restore status 2>&1 | grep -q "maintenance: false" || fail "maintenance mode is still on after the restore"
|
run_on ngb-frt-restore status 2>&1 | grep -q "maintenance: false" || fail "maintenance mode is still on after the restore"
|
||||||
run_on ngb-frt-restore config:app:get ng_backup restore_full_started >/dev/null 2>&1 && fail "the restore left its 'started' marker behind"
|
run_on ngb-frt-restore config:app:get ng_backup restore_full_started >/dev/null 2>&1 && fail "the restore left its 'started' marker behind"
|
||||||
ok "backup:restore:full completed, in maintenance mode and out of it again"
|
ok "backup:restore:full completed, in maintenance mode and out of it again"
|
||||||
|
if [ "$DB" = sqlite ]; then
|
||||||
|
docker exec ngb-frt-restore touch /tmp/frt-side-stop
|
||||||
|
for i in $(seq 1 20); do docker exec ngb-frt-restore test -f /tmp/frt-side-done && break; sleep 0.5; done
|
||||||
|
WRITES=$(docker exec -u www-data ngb-frt-restore php -r 'echo (new PDO("sqlite:/var/www/html/data/nextcloud.db"))->query("SELECT COUNT(*) FROM frt_side_writer")->fetchColumn();')
|
||||||
|
[ "${WRITES:-0}" -gt 3 ] || fail "the other writer did not run alongside the restore ($WRITES writes)"
|
||||||
|
ok "the restore waited for another process writing to SQLite ($WRITES writes alongside)"
|
||||||
|
fi
|
||||||
|
|
||||||
# Everything below must work in a FRESH occ process (a new PHP process per docker exec), since
|
# Everything below must work in a FRESH occ process (a new PHP process per docker exec), since
|
||||||
# earlier bugs here only showed up after config.php's secret was rewritten mid-restore.
|
# earlier bugs here only showed up after config.php's secret was rewritten mid-restore.
|
||||||
|
|||||||
@@ -36,8 +36,13 @@ MANIFEST=$(occ backup:user:backup ci "$U" | sed -n 's/.* to \(.*\)\.$/\1/p')
|
|||||||
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
||||||
ok "backed up $U to $MANIFEST"
|
ok "backed up $U to $MANIFEST"
|
||||||
|
|
||||||
# as-backup: restore into <uid>-bak; the original account must stay untouched.
|
occ backup:user:list ci "$U" | grep -qF "$MANIFEST" || fail "backup:user:list does not show $MANIFEST"
|
||||||
occ backup:user:restore ci "$MANIFEST" --mode=as-backup >/dev/null || fail "restore as-backup"
|
ok "backup:user:list shows the export"
|
||||||
|
|
||||||
|
# as-backup: restore into <uid>-bak; the original account must stay untouched. $U has one export,
|
||||||
|
# so the bare user id names it (#22).
|
||||||
|
OUT=$(occ backup:user:restore ci "$U" --mode=as-backup) || fail "restore as-backup by user id"
|
||||||
|
echo "$OUT" | grep -qF "Using $MANIFEST." || fail "restore by user id did not say which export it used"
|
||||||
occ files:scan "$U-bak" >/dev/null
|
occ files:scan "$U-bak" >/dev/null
|
||||||
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||||
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
||||||
@@ -55,6 +60,11 @@ SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xa
|
|||||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
||||||
ok "replace restores the original content under the same uid, byte-identical"
|
ok "replace restores the original content under the same uid, byte-identical"
|
||||||
|
|
||||||
|
# The replace above added a safety export: a bare user id is now ambiguous and must list both.
|
||||||
|
if OUT=$(occ backup:user:restore ci "$U" --mode=as-backup 2>&1); then fail "restore by an ambiguous user id ran"; fi
|
||||||
|
echo "$OUT" | grep -qF "$U has 2 exports here" && echo "$OUT" | grep -qF "$MANIFEST" || fail "an ambiguous user id did not list the exports"
|
||||||
|
ok "an ambiguous user id lists the exports instead of picking one"
|
||||||
|
|
||||||
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
||||||
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
||||||
# must forget exactly the 2 oldest.
|
# must forget exactly the 2 oldest.
|
||||||
|
|||||||
Reference in New Issue
Block a user