Compare commits
9 Commits
v1.0.0
..
5cde57bc1b
| Author | SHA1 | Date | |
|---|---|---|---|
| 5cde57bc1b | |||
| 3eec2e9554 | |||
| cdf0b70e25 | |||
| 66615c2a88 | |||
| 4847ce293d | |||
| 90dd97977a | |||
| 08534975d7 | |||
| 95e3dfcf6d | |||
| 7f4cae0e42 |
@@ -41,6 +41,20 @@ jobs:
|
||||
- run: composer install --no-interaction --no-progress
|
||||
- run: composer psalm
|
||||
|
||||
frontend:
|
||||
# Builds the admin page, so a broken Vue component fails CI instead of showing up after a release.
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: npm
|
||||
- run: npm ci --no-audit --no-fund
|
||||
- run: npm run build
|
||||
|
||||
integration:
|
||||
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
|
||||
# restore, retention, trash) on a fresh Nextcloud per version and database.
|
||||
|
||||
+34
-5
@@ -7,17 +7,46 @@ the app store accepts a `-beta` suffix, but Nextcloud servers then only offer th
|
||||
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
||||
instead carried by the app description, the git tag, the release, and this file.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- `occ backup:user:list [location] [user]` shows the user exports on a location, with the manifest
|
||||
path `backup:user:restore` expects. Before, that path was only printed once by
|
||||
`backup:user:backup` (#22).
|
||||
|
||||
### Changed
|
||||
- `occ backup:user:restore` also takes a user id instead of the manifest path, when that user has
|
||||
exactly one export on the location. With more, and for a manifest path that is not on the
|
||||
location, the error lists that user's exports; for a user without exports it names the
|
||||
expected form (#22). Before, `backup:user:restore <location> <uid>` only said "is not a
|
||||
recorded user export for this location".
|
||||
|
||||
### Fixed
|
||||
- An SFTP location with an RSA key in OpenSSH format ("BEGIN OPENSSH PRIVATE KEY") stopped
|
||||
working after an upgrade to Nextcloud 35 with "Unable to read key", and such a key could not be
|
||||
added there at all (#24). The "Private key" field on the admin page is a single line, so a
|
||||
pasted key lost its line breaks; Nextcloud 34 still read it, Nextcloud 35 reads keys with a
|
||||
stricter library. The line breaks are now restored before the key is used, so a stored location
|
||||
works again without re-entering it.
|
||||
- A full restore onto SQLite could stop at once with "database is locked" when another process
|
||||
wrote to the database just as the restore began (a background job that was already running).
|
||||
The restore now takes the write lock first and waits for the other process instead. The
|
||||
full-restore test on SQLite runs with a second process writing alongside.
|
||||
|
||||
## [1.0.0] - 2026-10-29
|
||||
|
||||
First stable release: the same code as 1.0.0-rc.6, which had no blocking problems reported in its
|
||||
week out (about 220 downloads). Only the version number and the README changed. The fixes for
|
||||
what was reported (#22, #24) follow in 1.0.1.
|
||||
First stable release. NG Backup makes encrypted, incremental backups of your whole Nextcloud
|
||||
(files, database and configuration) to S3, SFTP, WebDAV, SMB and other storage, and restores them
|
||||
from the web interface or with occ. Disaster recovery has been tested on real installations; see
|
||||
"Tested disaster recoveries" in the README, and read "Known limitations" before you rely on it.
|
||||
|
||||
The code is the same as 1.0.0-rc.6 (app version 0.9.12): updating from it changes nothing in your
|
||||
backups or settings.
|
||||
|
||||
## [1.0.0-rc.6] - 2026-10-24
|
||||
|
||||
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
||||
simulations and a user report. If no problems are reported for about a week, 1.0.0 will be this
|
||||
same code.
|
||||
simulations and a user report.
|
||||
|
||||
### Fixed
|
||||
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
||||
|
||||
@@ -434,6 +434,40 @@ in README "Known limitations":
|
||||
Een restore ruimt `*.ngb-restore-*` in de mappen die hij terugzet op.
|
||||
7. **Eén bestand terugzetten met `backup:restore`**: een pad naar een bestand zet 0 bestanden
|
||||
terug (er komt een `/` achter); nu werkt alleen een map.
|
||||
8. **Uitleg (i) bij de prullenbak van herstelpunten** (besluit 2026-10-25): bij "removed by
|
||||
retention policy" een infoknop met wat het betekent: het bewaarbeleid houdt voor oudere dagen
|
||||
één herstelpunt per dag (het laatste), weken en maanden; de rest gaat naar de prullenbak en
|
||||
wordt pas na de ingestelde wachttijd (`delete_delay_days`, standaard 30) echt verwijderd, en is
|
||||
tot die datum terug te zetten. Aanleiding: de beheerder zag "19-10 13:12 – after rc4 upgrade
|
||||
removed by retention policy" en dacht dat er iets was weggegooid.
|
||||
9. **Tijdzone in `occ backup:status`**: de tabel toont de servertijd (UTC), de webinterface de
|
||||
tijd van de beheerder; dat leest als een verkeerde starttijd. Zet de zone in de kolomkop
|
||||
("started (UTC)") of toon de tijden in de zone van het schema.
|
||||
10. **User-restore op gebruikersnaam** (issue #22, 2026-10-27): `backup:user:restore <locatie>
|
||||
New_User` gaf alleen "is not a recorded user export"; het argument moet het manifestpad
|
||||
zijn, dat alleen `backup:user:backup` eenmalig toonde. Nu: `backup:user:list` toont de
|
||||
exports, een kale uid werkt bij precies één export, anders een lijst om uit te kiezen —
|
||||
**klaar** (2026-10-27).
|
||||
11. **Tabblad "Docs" in de beheerpagina** (besluit 2026-10-27): een derde knop naast "Backup and
|
||||
restore" en "Settings", met links naar de HOWTO (`docs/HOWTO.md`), de DR-gids
|
||||
(`docs/DISASTER-RECOVERY.md`) en de roadmap (README, `#roadmap`) op GitHub, elk met een
|
||||
regel wat erin staat. Nu zijn ze alleen via de README te vinden. Zichtbaar ook zonder
|
||||
sleutel, zodat de gids vindbaar is vóór de eerste installatie-stap. "NG Backup is in beta"
|
||||
gaat uit de feedbackregel — **klaar** (2026-10-27).
|
||||
12. **SFTP-sleutel zonder regeleinden** (issue #24, 2026-10-28): het veld "Private key" op de
|
||||
beheerpagina is één regel, dus een geplakte sleutel verliest zijn regeleinden. NC34
|
||||
(phpseclib 2) las dat nog, NC35 (phpseclib 3) weigert een OpenSSH-sleutel dan met "Unable
|
||||
to read key". De regeleinden worden nu teruggezet vlak voor gebruik — **klaar** (2026-10-28).
|
||||
13. **"database is locked" bij het begin van `backup:restore:full` op SQLite** (CI, 2026-10-26
|
||||
bij #22 en 2026-10-29 op `main` na de merge van 1.0.0): de restore stopt meteen na het
|
||||
aanzetten van de onderhoudsmodus. Niet altijd, dus een tweede proces dat op hetzelfde moment
|
||||
de database gebruikt (cron, een webverzoek of de opstart van de container). Kan ook bij een
|
||||
echte restore naar SQLite gebeuren. Oorzaak (2026-10-29): SQLite (WAL) begint een transactie
|
||||
als lezer; schrijft een ander proces tussen ons eerste lezen en ons eerste schrijven, dan
|
||||
faalt dat schrijven direct (geen wachttijd helpt). In CI was dat de installer, die
|
||||
"installed" zet en daarna nog seconden doorschrijft. Nu neemt de restore eerst het
|
||||
schrijfslot en wacht hij op de ander; de SQLite-test draait met een tweede schrijver
|
||||
ernaast — **klaar** (2026-10-29).
|
||||
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
|
||||
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
|
||||
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
|
||||
@@ -470,6 +504,15 @@ in README "Known limitations":
|
||||
6. **Notitie achteraf bij een restorepunt**, te wijzigen in de lijst. Het label in de snapshot
|
||||
is versleuteld en ligt vast; de notitie apart op de locatie bewaren (bijv. in de catalogus),
|
||||
zodat een nieuwe server na een DR hem ook ziet.
|
||||
8. **Herstelpunt vastpinnen** (besluit 2026-10-25): een herstelpunt dat het bewaarbeleid
|
||||
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
|
||||
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
|
||||
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
|
||||
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
|
||||
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
|
||||
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
|
||||
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
|
||||
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
|
||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||
|
||||
@@ -265,7 +265,11 @@ a 👍 on its issue helps decide what comes first.
|
||||
system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
|
||||
while the NG Backup page is open ([#13](https://github.com/dillardblom/ng_backup/issues/13)).
|
||||
- An optional label for a backup started with "Back up now".
|
||||
- An explanation in the trash of restore points of what "removed by retention policy" means and
|
||||
until when a restore point can be brought back.
|
||||
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
|
||||
- A "Docs" tab on the admin page with links to the how-to, the disaster recovery guide and this
|
||||
roadmap.
|
||||
|
||||
**1.1.0**
|
||||
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
||||
@@ -273,6 +277,8 @@ a 👍 on its issue helps decide what comes first.
|
||||
- A schedule per location, also "every N hours".
|
||||
- Locations backed up one after the other instead of at the same time.
|
||||
- A note on a restore point that you can change afterwards.
|
||||
- Pinning a restore point, so the retention policy never removes it (for example the one from
|
||||
before an upgrade).
|
||||
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
||||
Generator) while a backup runs.
|
||||
- Faster backups: reading and uploading at the same time.
|
||||
@@ -330,4 +336,4 @@ On the backlog; know them before you rely on NG Backup:
|
||||
| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) |
|
||||
| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) |
|
||||
| `backup:key:import` | import a recovery kit on a fresh installation |
|
||||
| `backup:user:backup`, `backup:user:restore` | one user via user_migration (asks before installing it) |
|
||||
| `backup:user:backup`, `backup:user:list`, `backup:user:restore` | one user via user_migration (asks before installing it); list the exports |
|
||||
|
||||
@@ -92,6 +92,7 @@ you need help with restoring a backup after a crash.]]></description>
|
||||
<command>OCA\NgBackup\Command\KeySlotRemove</command>
|
||||
<command>OCA\NgBackup\Command\BackupUser</command>
|
||||
<command>OCA\NgBackup\Command\RestoreUser</command>
|
||||
<command>OCA\NgBackup\Command\ListUserExports</command>
|
||||
<command>OCA\NgBackup\Command\KeyImport</command>
|
||||
<command>OCA\NgBackup\Command\RestoreFull</command>
|
||||
<command>OCA\NgBackup\Command\Verify</command>
|
||||
|
||||
@@ -114,10 +114,16 @@ Needs the `user_migration` app. NG Backup asks before installing it:
|
||||
|
||||
```
|
||||
occ backup:user:backup offsite <uid>
|
||||
occ backup:user:list offsite [<uid>] # the exports and their manifest
|
||||
occ backup:user:restore offsite users/<uid>/<exportId> # replace (default)
|
||||
occ backup:user:restore offsite users/<uid>/<exportId> --mode=as-backup
|
||||
```
|
||||
|
||||
- `backup:user:restore` needs the manifest path that `backup:user:backup` prints at the end
|
||||
(`users/<uid>/<exportId>`); `backup:user:list` shows it again. A user id alone also works when
|
||||
that user has exactly one export on the location. With more (a `replace` restore adds a safety
|
||||
export), the command lists them and you pass one.
|
||||
|
||||
- If `user_migration` is missing, the command asks to install and enable it. For non-interactive
|
||||
use, add `--install-user-migration` to approve that explicitly. The command stops after the
|
||||
installation; run it once more.
|
||||
|
||||
@@ -72,6 +72,10 @@ final class ExternalStorageFactory {
|
||||
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
|
||||
}
|
||||
|
||||
if (is_string($options['private_key'] ?? null)) {
|
||||
$options['private_key'] = self::normalizePrivateKey($options['private_key']);
|
||||
}
|
||||
|
||||
$config = new StorageConfig();
|
||||
$config->setBackend($backend);
|
||||
$config->setAuthMechanism($auth);
|
||||
@@ -86,4 +90,22 @@ final class ExternalStorageFactory {
|
||||
$storage = $backend->wrapStorage($storage);
|
||||
return $auth->wrapStorage($storage);
|
||||
}
|
||||
|
||||
/**
|
||||
* An OpenSSH-format private key with its line breaks restored. Up to Nextcloud 34
|
||||
* (phpseclib 2) a key whose newlines were lost or turned into spaces, e.g. on its way through
|
||||
* a shell or a form, still loaded; Nextcloud 35 (phpseclib 3) refuses it with "Unable to read
|
||||
* key" (#24). The key itself is not changed, so it is fixed when it is used, not when stored.
|
||||
* Other formats (PEM, PKCS#8) still load as they are and are returned unchanged.
|
||||
*/
|
||||
public static function normalizePrivateKey(string $key): string {
|
||||
if (!preg_match('/-----BEGIN OPENSSH PRIVATE KEY-----(.*?)-----END OPENSSH PRIVATE KEY-----/s', $key, $m)) {
|
||||
return $key;
|
||||
}
|
||||
$body = (string)preg_replace('/\s+/', '', $m[1]);
|
||||
if (!preg_match('#^[A-Za-z0-9+/]+=*$#', $body)) {
|
||||
return $key;
|
||||
}
|
||||
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Command;
|
||||
|
||||
use OCA\NgBackup\Service\TargetService;
|
||||
use Symfony\Component\Console\Command\Command;
|
||||
use Symfony\Component\Console\Helper\Table;
|
||||
use Symfony\Component\Console\Input\InputArgument;
|
||||
use Symfony\Component\Console\Input\InputInterface;
|
||||
use Symfony\Component\Console\Output\OutputInterface;
|
||||
|
||||
final class ListUserExports extends Command {
|
||||
public function __construct(
|
||||
private TargetService $targets,
|
||||
) {
|
||||
parent::__construct();
|
||||
}
|
||||
|
||||
protected function configure(): void {
|
||||
$this->setName('backup:user:list')
|
||||
->setDescription('List the user exports (backup:user:backup) on a location')
|
||||
->addArgument('target', InputArgument::OPTIONAL, 'Location name or id (default: all)')
|
||||
->addArgument('user', InputArgument::OPTIONAL, 'Only the exports of this user id')
|
||||
->setHelp(<<<'HELP'
|
||||
The manifest column is what occ backup:user:restore expects. A safety export made by a restore
|
||||
with --mode=replace is listed here as well.
|
||||
|
||||
Example:
|
||||
occ backup:user:list offsite alice
|
||||
HELP);
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$name = $input->getArgument('target');
|
||||
$uid = $input->getArgument('user');
|
||||
$rows = [];
|
||||
try {
|
||||
$targets = $name === null ? $this->targets->list() : [$this->targets->get($name)];
|
||||
foreach ($targets as $t) {
|
||||
foreach ($this->targets->repository($t)->listUserExports($uid) as $path) {
|
||||
$parts = explode('/', $path, 3);
|
||||
$user = $parts[1] ?? '';
|
||||
$exportId = $parts[2] ?? '';
|
||||
$time = \DateTimeImmutable::createFromFormat('Ymd\THis\Z', substr($exportId, 0, 16), new \DateTimeZone('UTC'));
|
||||
$rows[] = [$t->getName(), $user, $time !== false ? $time->format('Y-m-d H:i') : '', $path];
|
||||
}
|
||||
}
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
if ($rows === []) {
|
||||
$output->writeln('No user exports yet (occ backup:user:backup).');
|
||||
return 0;
|
||||
}
|
||||
(new Table($output))->setHeaders(['location', 'user', 'date (UTC)', 'manifest'])->setRows($rows)->render();
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -33,7 +33,7 @@ final class RestoreUser extends Command {
|
||||
$this->setName('backup:user:restore')
|
||||
->setDescription('Restore a user from a user_migration export (see backup:user:backup)')
|
||||
->addArgument('target', InputArgument::REQUIRED, 'Location name or id')
|
||||
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>)')
|
||||
->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users/<uid>/<exportId>), or a user id with exactly one export')
|
||||
->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE)
|
||||
->addOption('force', null, InputOption::VALUE_NONE, 'Replace an existing account without asking (required when not interactive)')
|
||||
->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)')
|
||||
@@ -45,16 +45,31 @@ Never imports over a live account. Modes:
|
||||
compare and transfer by hand; Nextcloud cannot rename a user id, so the backup gets
|
||||
the new name, never the existing account
|
||||
|
||||
occ backup:user:list shows the exports on a location. A user id alone works when that user
|
||||
has exactly one export there; with more, the command lists them and you pass one.
|
||||
|
||||
Example:
|
||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34
|
||||
occ backup:user:restore offsite alice
|
||||
occ backup:user:restore offsite users/alice/20261005T120000Z-ab12cd34 --mode=as-backup
|
||||
HELP);
|
||||
}
|
||||
|
||||
protected function execute(InputInterface $input, OutputInterface $output): int {
|
||||
$mode = (string)$input->getOption('mode');
|
||||
$uid = explode('/', (string)$input->getArgument('manifest'))[1] ?? '';
|
||||
if ($mode === UserRestoreService::MODE_REPLACE && $uid !== '' && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
||||
try {
|
||||
$target = $this->targets->get($input->getArgument('target'));
|
||||
$ref = (string)$input->getArgument('manifest');
|
||||
$manifest = $this->targets->repository($target)->resolveUserExport($ref);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
}
|
||||
if ($manifest !== $ref) {
|
||||
$output->writeln("Using $manifest.");
|
||||
}
|
||||
$uid = explode('/', $manifest)[1];
|
||||
if ($mode === UserRestoreService::MODE_REPLACE && $this->userManager->userExists($uid) && !$input->getOption('force')) {
|
||||
// Replace deletes the live account (after a safety export). Shares to others and data of
|
||||
// apps without a user_migration migrator do not come back, so never do it unasked.
|
||||
if (!$input->isInteractive()) {
|
||||
@@ -69,9 +84,8 @@ HELP);
|
||||
}
|
||||
try {
|
||||
$output->writeln('<comment>Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.</comment>');
|
||||
$target = $this->targets->get($input->getArgument('target'));
|
||||
$this->installer->ensureEnabled($input, $output);
|
||||
$user = $this->userRestore->restoreUser($target, (string)$input->getArgument('manifest'), $mode);
|
||||
$user = $this->userRestore->restoreUser($target, $manifest, $mode);
|
||||
} catch (\Throwable $e) {
|
||||
$output->writeln('<error>' . $e->getMessage() . '</error>');
|
||||
return 1;
|
||||
|
||||
+7
-1
@@ -169,7 +169,7 @@ final class DbDumper {
|
||||
$foreignKeys = [];
|
||||
$this->db->beginTransaction();
|
||||
try {
|
||||
$foreignKeys = $this->suspendForeignKeys();
|
||||
$foreignKeys = $this->suspendForeignKeys(); // first: on SQLite it also takes the write lock
|
||||
foreach ($manifest['tables'] as $table => $info) {
|
||||
if (in_array($table, $report['missing'], true)) {
|
||||
continue;
|
||||
@@ -412,6 +412,12 @@ final class DbDumper {
|
||||
return $checks;
|
||||
}
|
||||
if ($platform instanceof \Doctrine\DBAL\Platforms\SqlitePlatform) {
|
||||
// SQLite (WAL) starts a transaction as a reader. When another process (a cron job, the
|
||||
// installer still finishing) writes between our first read and our first write, that
|
||||
// write fails at once with "database is locked": no waiting helps a reader whose view
|
||||
// is out of date. Take the write lock first, so we wait for the other writer instead
|
||||
// and it waits for us after that.
|
||||
$this->db->executeStatement('DELETE FROM ' . $this->db->quoteIdentifier($this->prefix . 'appconfig') . ' WHERE 0 = 1');
|
||||
$this->db->executeStatement('PRAGMA defer_foreign_keys = ON'); // checked at commit
|
||||
}
|
||||
return [];
|
||||
|
||||
@@ -561,6 +561,31 @@ final class Repository {
|
||||
return isset($this->catalog->userExports[$manifestPath]);
|
||||
}
|
||||
|
||||
/**
|
||||
* The recorded export $ref names: a manifest path (users/<uid>/<exportId>) as is, or a bare
|
||||
* user id (also "users/<uid>") when that user has exactly one export here. Anything else
|
||||
* throws, with the exports to choose from in the message.
|
||||
*/
|
||||
public function resolveUserExport(string $ref): string {
|
||||
if ($this->isRecordedUserExport($ref)) {
|
||||
return $ref;
|
||||
}
|
||||
$rest = trim($ref, '/');
|
||||
if (str_starts_with($rest, 'users/')) {
|
||||
$rest = substr($rest, strlen('users/'));
|
||||
}
|
||||
$uid = explode('/', $rest)[0];
|
||||
$exports = $uid !== '' ? $this->listUserExports($uid) : [];
|
||||
if (!str_contains($rest, '/') && count($exports) === 1) {
|
||||
return $exports[0];
|
||||
}
|
||||
if ($exports === []) {
|
||||
throw new RepositoryException("$ref is not a recorded user export for this location, and $uid has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports");
|
||||
}
|
||||
$which = str_contains($rest, '/') ? "$ref is not a recorded user export for this location" : "$uid has " . count($exports) . ' exports here';
|
||||
throw new RepositoryException("$which; pass one of these (newest first):\n " . implode("\n ", $exports));
|
||||
}
|
||||
|
||||
/** Permanently delete a user export. Its blobs are freed by the next prune() like any other. */
|
||||
public function forgetUserExport(string $manifestPath): void {
|
||||
if (!isset($this->catalog->userExports[$manifestPath])) {
|
||||
|
||||
+41
-2
@@ -7,7 +7,7 @@
|
||||
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
|
||||
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
|
||||
<p class="ngb-feedback">
|
||||
{{ t('ng_backup', 'NG Backup is in beta. We would like to hear how it works for you, also when everything went fine.') }}
|
||||
{{ t('ng_backup', 'We would like to hear how NG Backup works for you, also when everything went fine.') }}
|
||||
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
|
||||
</p>
|
||||
<NcLoadingIcon v-if="!status" :size="32" />
|
||||
@@ -18,9 +18,18 @@
|
||||
<div class="ngb-tabs">
|
||||
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
|
||||
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
|
||||
<NcButton :variant="tab === 'docs' ? 'primary' : 'secondary'" @click="tab = 'docs'">{{ t('ng_backup', 'Docs') }}</NcButton>
|
||||
</div>
|
||||
|
||||
<template v-if="tab === 'settings'">
|
||||
<NcSettingsSection v-if="tab === 'docs'" :name="t('ng_backup', 'Documentation')">
|
||||
<ul class="ngb-docs">
|
||||
<li v-for="doc in docs" :key="doc.url">
|
||||
<a :href="doc.url" target="_blank" rel="noopener noreferrer">{{ doc.name }} ↗</a>
|
||||
<span>{{ doc.description }}</span>
|
||||
</li>
|
||||
</ul>
|
||||
</NcSettingsSection>
|
||||
<template v-else-if="tab === 'settings'">
|
||||
<KeySetup :status="status" @changed="refresh" />
|
||||
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
|
||||
</template>
|
||||
@@ -52,6 +61,25 @@ import RestoreBrowser from './components/RestoreBrowser.vue'
|
||||
// Opens the experience report form in .github/ISSUE_TEMPLATE
|
||||
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
|
||||
|
||||
const repoUrl = 'https://github.com/dillardblom/ng_backup'
|
||||
const docs = [
|
||||
{
|
||||
name: t('ng_backup', 'How-to'),
|
||||
description: t('ng_backup', 'Step by step: from first install to a tested restore.'),
|
||||
url: repoUrl + '/blob/main/docs/HOWTO.md',
|
||||
},
|
||||
{
|
||||
name: t('ng_backup', 'Disaster recovery'),
|
||||
description: t('ng_backup', 'Your server is gone: get Nextcloud back on a new server from a backup location.'),
|
||||
url: repoUrl + '/blob/main/docs/DISASTER-RECOVERY.md',
|
||||
},
|
||||
{
|
||||
name: t('ng_backup', 'Roadmap'),
|
||||
description: t('ng_backup', 'What is planned for the next versions.'),
|
||||
url: repoUrl + '#roadmap',
|
||||
},
|
||||
]
|
||||
|
||||
const status = ref(null)
|
||||
const allRuns = ref(false) // runs older than the last days too
|
||||
const error = ref('')
|
||||
@@ -102,6 +130,17 @@ onBeforeUnmount(() => clearTimeout(timer))
|
||||
color: var(--color-primary-element);
|
||||
text-decoration: underline;
|
||||
}
|
||||
.ngb-docs li {
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.ngb-docs a {
|
||||
color: var(--color-primary-element);
|
||||
text-decoration: underline;
|
||||
margin-right: 8px;
|
||||
}
|
||||
.ngb-docs span {
|
||||
color: var(--color-text-maxcontrast);
|
||||
}
|
||||
.ngb-tabs {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Backend;
|
||||
|
||||
use OCA\NgBackup\Backend\ExternalStorageFactory;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** ExternalStorageFactory::normalizePrivateKey(): keys that phpseclib 2 read, for phpseclib 3 (#24). */
|
||||
class ExternalStorageFactoryTest extends TestCase {
|
||||
/** An OpenSSH-format key laid out as ssh-keygen writes it; the body is random, not a real key. */
|
||||
private static function key(): string {
|
||||
$body = base64_encode(random_bytes(1000));
|
||||
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
|
||||
}
|
||||
|
||||
public function testRestoresLostLineBreaks(): void {
|
||||
$key = self::key();
|
||||
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey($key));
|
||||
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", ' ', $key)), 'newlines as spaces');
|
||||
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", '', $key)), 'no newlines');
|
||||
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", "\r\n", $key)), 'CRLF');
|
||||
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(rtrim($key)), 'no trailing newline');
|
||||
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(preg_replace('/^/m', ' ', $key)), 'indented');
|
||||
}
|
||||
|
||||
public function testLeavesOtherFormatsAlone(): void {
|
||||
$pem = "-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\nDEK-Info: DES-EDE3-CBC,FF44E0A94036766C\n\nAAAA\n-----END RSA PRIVATE KEY-----\n";
|
||||
$this->assertSame($pem, ExternalStorageFactory::normalizePrivateKey($pem));
|
||||
$this->assertSame('not a key', ExternalStorageFactory::normalizePrivateKey('not a key'));
|
||||
$odd = "-----BEGIN OPENSSH PRIVATE KEY-----\\nAAAA\\n-----END OPENSSH PRIVATE KEY-----";
|
||||
$this->assertSame($odd, ExternalStorageFactory::normalizePrivateKey($odd), 'not base64 inside: unchanged');
|
||||
}
|
||||
}
|
||||
@@ -106,4 +106,51 @@ class UserExportTest extends TestCase {
|
||||
$this->expectExceptionMessage('missing on the location');
|
||||
$repo->listUserExports();
|
||||
}
|
||||
|
||||
public function testResolveTakesARecordedPathAsIs(): void {
|
||||
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||
$this->putManifest($repo, $path);
|
||||
|
||||
$this->assertSame($path, $repo->resolveUserExport($path));
|
||||
}
|
||||
|
||||
public function testResolveTakesAUserIdWithOneExport(): void {
|
||||
// Issue #22: "backup:user:restore <location> New_User" was rejected as not a recorded export.
|
||||
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||
$path = 'users/alice/20261005T120000Z-aaaa0001';
|
||||
$this->putManifest($repo, $path);
|
||||
$this->putManifest($repo, 'users/alice2/20261005T120000Z-aaaa0002');
|
||||
|
||||
$this->assertSame($path, $repo->resolveUserExport('alice'));
|
||||
$this->assertSame($path, $repo->resolveUserExport('users/alice/'));
|
||||
}
|
||||
|
||||
public function testResolveListsTheExportsOfAUserWithMore(): void {
|
||||
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||
$this->putManifest($repo, 'users/alice/20261006T120000Z-aaaa0002');
|
||||
|
||||
$this->expectException(RepositoryException::class);
|
||||
$this->expectExceptionMessage("alice has 2 exports here; pass one of these (newest first):\n users/alice/20261006T120000Z-aaaa0002\n users/alice/20261005T120000Z-aaaa0001");
|
||||
$repo->resolveUserExport('alice');
|
||||
}
|
||||
|
||||
public function testResolveListsTheExportsOfTheUserOfAWrongPath(): void {
|
||||
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||
|
||||
$this->expectException(RepositoryException::class);
|
||||
$this->expectExceptionMessage("users/alice/typo is not a recorded user export for this location; pass one of these (newest first):\n users/alice/20261005T120000Z-aaaa0001");
|
||||
$repo->resolveUserExport('users/alice/typo');
|
||||
}
|
||||
|
||||
public function testResolveNamesTheExpectedFormForAUserWithoutExports(): void {
|
||||
$repo = Repository::initWithKey($this->backend, $this->keys, $this->keys->wrap('pw'), $this->anchor);
|
||||
$this->putManifest($repo, 'users/alice/20261005T120000Z-aaaa0001');
|
||||
|
||||
$this->expectException(RepositoryException::class);
|
||||
$this->expectExceptionMessage('bob has no exports here. Expected users/<uid>/<exportId>; occ backup:user:list shows the exports');
|
||||
$repo->resolveUserExport('bob');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -173,6 +173,20 @@ if [ "$LEGACY" = legacy ] && [ "$DB" != sqlite ]; then
|
||||
ok "older backup refused without --skip-missing-tables, missing tables named"
|
||||
RESTORE_OPTS+=(--skip-missing-tables)
|
||||
fi
|
||||
if [ "$DB" = sqlite ]; then
|
||||
# Another process writing while the restore starts (a cron job that was already running, or
|
||||
# the installer still finishing after it says "installed"): the restore must wait for it, not
|
||||
# stop with "database is locked". Short write transactions on a table of its own, until the
|
||||
# restore is done (/tmp/frt-side-stop; Nextcloud's own commands after it are not under test).
|
||||
docker exec -d -u www-data ngb-frt-restore php -r '
|
||||
$p = new PDO("sqlite:/var/www/html/data/nextcloud.db"); $p->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
||||
$p->exec("CREATE TABLE IF NOT EXISTS frt_side_writer (t REAL)");
|
||||
for ($end = microtime(true) + 300; microtime(true) < $end && !file_exists("/tmp/frt-side-stop");) {
|
||||
$p->beginTransaction(); $p->exec("INSERT INTO frt_side_writer VALUES (" . microtime(true) . ")"); usleep(300000); $p->commit(); usleep(20000);
|
||||
}
|
||||
touch("/tmp/frt-side-done");'
|
||||
sleep 1
|
||||
fi
|
||||
sftp_logins() { docker logs ngb-frt-sftp 2>&1 | grep -c "Accepted password" || true; }
|
||||
[ "$MODE" = sftp ] && LOGINS_BEFORE=$(sftp_logins)
|
||||
OUT=$(run_on ngb-frt-restore backup:restore:full offsite "$SNAP" "${RESTORE_OPTS[@]}" 2>&1) || {
|
||||
@@ -184,6 +198,13 @@ printf '%s\n' "$OUT" | grep -q "maintenance mode while the restore runs" || fail
|
||||
run_on ngb-frt-restore status 2>&1 | grep -q "maintenance: false" || fail "maintenance mode is still on after the restore"
|
||||
run_on ngb-frt-restore config:app:get ng_backup restore_full_started >/dev/null 2>&1 && fail "the restore left its 'started' marker behind"
|
||||
ok "backup:restore:full completed, in maintenance mode and out of it again"
|
||||
if [ "$DB" = sqlite ]; then
|
||||
docker exec ngb-frt-restore touch /tmp/frt-side-stop
|
||||
for i in $(seq 1 20); do docker exec ngb-frt-restore test -f /tmp/frt-side-done && break; sleep 0.5; done
|
||||
WRITES=$(docker exec -u www-data ngb-frt-restore php -r 'echo (new PDO("sqlite:/var/www/html/data/nextcloud.db"))->query("SELECT COUNT(*) FROM frt_side_writer")->fetchColumn();')
|
||||
[ "${WRITES:-0}" -gt 3 ] || fail "the other writer did not run alongside the restore ($WRITES writes)"
|
||||
ok "the restore waited for another process writing to SQLite ($WRITES writes alongside)"
|
||||
fi
|
||||
|
||||
# Everything below must work in a FRESH occ process (a new PHP process per docker exec), since
|
||||
# earlier bugs here only showed up after config.php's secret was rewritten mid-restore.
|
||||
|
||||
@@ -36,8 +36,13 @@ MANIFEST=$(occ backup:user:backup ci "$U" | sed -n 's/.* to \(.*\)\.$/\1/p')
|
||||
[ -n "$MANIFEST" ] || fail "backup:user:backup printed no manifest path"
|
||||
ok "backed up $U to $MANIFEST"
|
||||
|
||||
# as-backup: restore into <uid>-bak; the original account must stay untouched.
|
||||
occ backup:user:restore ci "$MANIFEST" --mode=as-backup >/dev/null || fail "restore as-backup"
|
||||
occ backup:user:list ci "$U" | grep -qF "$MANIFEST" || fail "backup:user:list does not show $MANIFEST"
|
||||
ok "backup:user:list shows the export"
|
||||
|
||||
# as-backup: restore into <uid>-bak; the original account must stay untouched. $U has one export,
|
||||
# so the bare user id names it (#22).
|
||||
OUT=$(occ backup:user:restore ci "$U" --mode=as-backup) || fail "restore as-backup by user id"
|
||||
echo "$OUT" | grep -qF "Using $MANIFEST." || fail "restore by user id did not say which export it used"
|
||||
occ files:scan "$U-bak" >/dev/null
|
||||
SUM_BAK=$($X sh -c "cd $DATA/$U-bak/files && find . -type f -print0 | sort -z | xargs -0 md5sum | md5sum")
|
||||
[ "$SUM_BEFORE" = "$SUM_BAK" ] || fail "the <uid>-bak account's files differ from the original"
|
||||
@@ -55,6 +60,11 @@ SUM_AFTER=$($X sh -c "cd $DATA/$U/files && find . -type f -print0 | sort -z | xa
|
||||
[ "$SUM_BEFORE" = "$SUM_AFTER" ] || fail "files differ after a replace restore"
|
||||
ok "replace restores the original content under the same uid, byte-identical"
|
||||
|
||||
# The replace above added a safety export: a bare user id is now ambiguous and must list both.
|
||||
if OUT=$(occ backup:user:restore ci "$U" --mode=as-backup 2>&1); then fail "restore by an ambiguous user id ran"; fi
|
||||
echo "$OUT" | grep -qF "$U has 2 exports here" && echo "$OUT" | grep -qF "$MANIFEST" || fail "an ambiguous user id did not list the exports"
|
||||
ok "an ambiguous user id lists the exports instead of picking one"
|
||||
|
||||
# Retention: $U now has the manual export above plus the automatic safety-export the replace
|
||||
# restore made of it just before deleting it (4 total after these two more), then --user-last=2
|
||||
# must forget exactly the 2 oldest.
|
||||
|
||||
Reference in New Issue
Block a user