diff --git a/PLAN.md b/PLAN.md index d146e9a..6627a34 100644 --- a/PLAN.md +++ b/PLAN.md @@ -350,8 +350,12 @@ minimale volledige restore (`occ backup:restore:full` op een verse installatie) (standaard 3, per installatie, net als de bestaande snapshot-policy), toegepast door `occ backup:prune` (geen prullenbak voor user-exports, in tegenstelling tot snapshots — on-demand, niet het primaire vangnet). -5. Automatische user-export inpluggen in het reguliere schema (of expliciet besluit dat het on-demand blijft) -9. SMB/NFS intern testen op de eigen OMV-instantie — nog open (intern, niet in CI) +5. Automatische user-export in het reguliere schema — **naar backlog** (niet voor deze release; on-demand blijft) +9. SMB-roundtrip (protocol, niet een specifieke server) — **klaar** (2026-10-06): CI-matrix, 2 GB-roundtrip en + user-export via SMB; NFS via de OMV-share getest (`local`-backend op een gemounte share, 2026-10-06) +10. `backup:user:restore` zonder user_migration: vraagt om toestemming om de app te installeren en te activeren (of + `--install-user-migration`) — **klaar** (2026-10-06); waarschuwing over de settings-allowlist — **klaar**; + agenda/contacten bij migratie naar een server zonder die gebruiker — **getest** (2026-10-06, naar `migrated-personal`/`migrated-contacts`) Doorgeschoven naar **v1.3.0** (fase 3, niet meer fase 2): in-place volledige restore (eigen restore-modus i.p.v. onderhoudsmodus) en zelfbediening voor gebruikers (eigen bestanden diff --git a/README.md b/README.md index 4455b8a..d64caf9 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,8 @@ installations and hosted Nextcloud. (Choosing SMB as a storage backend is the on see "Storage backend experiences" below — that requirement comes from files_external, not from ng_backup, and applies to any Nextcloud app that writes to an SMB share.) -**Status: in development (phase 1). Not ready for production use.** See `PLAN.md` for the design. +**Status: pre-release. Not yet tested on production.** See `PLAN.md` for the design and `docs/HOWTO.md` for a +step-by-step guide from setup to a tested restore. License: AGPL-3.0-or-later. @@ -30,7 +31,7 @@ License: AGPL-3.0-or-later. occ backup:key:init --label="Safe" # first passphrase, deletion delay (default 7 days) occ backup:key:slot:add --label="CTO" # optional second and third passphrase occ backup:key:slot:add --label="Head of IT" -occ backup:key:kit --output=/root/ng_backup-kit.json +occ backup:key:kit --output=/var/tmp/ng_backup-kit.json occ backup:key:confirm occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket=... -o key=... -o secret=... occ backup:run @@ -106,7 +107,9 @@ What you should do on the location (strongly recommended): - **Files and folders** of a user: in the web interface or `occ backup:restore`, into a new folder ("Restored "), merged into the original place, or replacing it. - **Raw restore** of any path (for example `config/`) to a local directory: `occ backup:restore ... --to-directory=DIR`. -- **Whole server** (disaster recovery onto a fresh installation): planned for phase 2. +- **Whole server** (disaster recovery onto a fresh installation): `occ backup:restore:full`, see the how-to. +- **One user** (account, settings, files, calendars, contacts via user_migration): `occ backup:user:backup` and + `occ backup:user:restore`, replacing the account or restoring it as `-bak`. See the how-to. ## Storage backend experiences @@ -154,3 +157,7 @@ Contributions welcome via PR — add what you ran into with your own provider. | `backup:run`, `backup:status`, `backup:list` | back up, status, restore points | | `backup:browse`, `backup:restore` | browse and restore | | `backup:retention`, `backup:prune`, `backup:trash`, `backup:trash:restore` | retention, cleanup, trash | +| `backup:verify` | checksum audit of a restore point (`--deep` downloads and decrypts everything) | +| `backup:restore:full` | whole-server restore onto a fresh installation (after `backup:key:import`) | +| `backup:key:import` | import a recovery kit on a fresh installation | +| `backup:user:backup`, `backup:user:restore` | one user via user_migration (asks before installing it) | diff --git a/docs/HOWTO.md b/docs/HOWTO.md new file mode 100644 index 0000000..f90a0e9 --- /dev/null +++ b/docs/HOWTO.md @@ -0,0 +1,149 @@ +# NG Backup how-to + +Step by step: from first install to a tested restore. All commands run as the web user +(`sudo -u www-data php occ ...`, or `docker exec -u www-data php occ ...` in +All-in-One and Docker setups). Replace names in `` with your own. + +## 1. Create the key and the recovery kit + +``` +occ backup:key:init --label="Main" --delete-delay-days=7 +occ backup:key:kit --output=/var/tmp/ng_backup-kit.json +occ backup:key:confirm -u --code= --confirm="Yes, I confirm" +``` + +- The passphrase protects the backup. Keep it somewhere other than this server. +- Optional: extra passphrases for other people with `occ backup:key:slot:add --label="..."`. + Any one passphrase is enough to restore. +- The recovery kit (`/var/tmp/ng_backup-kit.json` above) is also required for disaster recovery. The web + user writes it there; then move it off the server as root, next to the passphrase, and delete the + copy in `/var/tmp`. Without the kit and a passphrase, the backup cannot be read. + +## 2. Add a backup location + +NG Backup uses Nextcloud's External storage backends. Check what your server supports first: + +``` +occ backup:target:backends +``` + +Examples: + +``` +occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket= -o key= -o secret= +occ backup:target:add nas sftp -a sftp::password -o host= -o user= -o password= -o root=home +occ backup:target:add local local -a null::null -o datadir=/mnt/backup +``` + +- **SFTP on a Hetzner Storage Box sub-account:** use `-o root=home`, not `/` (see README). +- **FTP:** use a server that accepts absolute paths (for example `pure-ftpd`). +- **SMB:** requires the native `smbclient` PHP extension. The CLI binary alone is not enough (see README). + `backup:target:add` refuses SMB when the extension is missing. +- **NFS:** not a backend of its own. Mount the export on the server and use the `local` backend + with the mount point as `datadir`. + +Verify the location before relying on it: + +``` +occ backup:target:list +occ backup:status +``` + +## 3. Run the first backup + +``` +occ backup:run offsite -l first +occ backup:list +``` + +Later runs upload only changed data. An unchanged run uploads (almost) nothing; the output shows +the number of uploaded bytes, so check that value when you test incremental behaviour. + +Schedule `occ backup:run` from system cron (or use the Administration settings, if you prefer). + +## 4. Check the backup regularly + +``` +occ backup:verify offsite # shallow: all blocks are present +occ backup:verify offsite --deep # also downloads and decrypts every block +``` + +Run the deep check at least once after setting up a location, and after changing it. + +## 5. Restore files and folders + +In the web interface, or with occ: + +``` +occ backup:browse offsite +occ backup:restore offsite data//files/Docs --mode=new-folder +occ backup:restore offsite data//files/Docs --mode=merge +occ backup:restore offsite data//files/Docs --mode=replace +occ backup:restore offsite config --to-directory=/var/tmp/restore-config +``` + +- `new-folder` (default): puts the files in "Restored " next to the original. Safest choice. +- `merge`: adds missing files, keeps current ones. +- `replace`: overwrites the folder. Replaced files keep their old content as a version, and removed + files go to the trash. +- `--to-directory`: writes raw files to a local directory (for example `config/`). + +## 6. Back up and restore one user + +Needs the `user_migration` app. NG Backup asks before installing it: + +``` +occ backup:user:backup offsite +occ backup:user:restore offsite users// # replace (default) +occ backup:user:restore offsite users// --mode=as-backup +``` + +- If `user_migration` is missing, the command asks to install and enable it. For non-interactive + use, add `--install-user-migration` to approve that explicitly. The command stops after the + installation; run it once more. +- `replace` first makes a safety export of the current account, then recreates the user under the + same id. `as-backup` restores into `-bak` and leaves the original untouched. +- A restore onto a server where the user does not exist works. Calendars and address books are + placed in `migrated-personal` and `migrated-contacts`, not in the original names. +- **Settings:** on restore, `user_migration` imports only the app settings on its own allowlist + (for example calendar view and reminder settings). Other app settings of the user are skipped. + NG Backup reuses `user_migration`'s importer, so this list is not extended. +- Keep the last few exports per user: `occ backup:retention --user-last=3`. + +## 7. Disaster recovery: restore the whole server + +Use a fresh Nextcloud installation of the same major version, with NG Backup installed and +the same location available: + +``` +occ backup:key:import /path/ng_backup-kit.json --passphrase-file=/path/passphrase +occ backup:target:add offsite ... # same details as on the original server +occ backup:list offsite # the snapshots must be listed +occ backup:restore:full offsite +``` + +Afterwards, check that users, files and the database are complete, and run a backup of the restored +server. The restored installation keeps its own app configuration and secret. + +## 8. Retention + +``` +occ backup:retention --last=3 --daily=7 --weekly=4 --monthly=12 +occ backup:prune offsite --dry-run # show first +occ backup:prune offsite # apply the policy; forgotten snapshots go to the trash first +occ backup:trash # list the trash +occ backup:trash:restore offsite +``` + +Snapshots are kept according to the policy, and their data is deleted only after the deletion delay +(set with `backup:key:init --delete-delay-days`). `occ backup:prune offsite --dry-run` shows what would be +removed without changing anything. + +## 9. Checklist before you rely on it + +- [ ] Key created, passphrase stored off-server, recovery kit stored off-server +- [ ] Location added; `backup:status` shows it reachable +- [ ] First backup done and listed in `backup:list` +- [ ] `backup:verify --deep` succeeds +- [ ] One file restore tested (`new-folder`) +- [ ] Disaster recovery tested on a second, fresh installation (or in a test environment) diff --git a/lib/Command/BackupUser.php b/lib/Command/BackupUser.php index a73c8ca..6f3072f 100644 --- a/lib/Command/BackupUser.php +++ b/lib/Command/BackupUser.php @@ -9,10 +9,12 @@ namespace OCA\NgBackup\Command; use OCA\NgBackup\Service\TargetService; use OCA\NgBackup\Service\UserBackupService; +use OCA\NgBackup\UserMigration\UserMigrationInstaller; use OCP\IUserManager; use Symfony\Component\Console\Command\Command; use Symfony\Component\Console\Input\InputArgument; use Symfony\Component\Console\Input\InputInterface; +use Symfony\Component\Console\Input\InputOption; use Symfony\Component\Console\Output\OutputInterface; final class BackupUser extends Command { @@ -20,6 +22,7 @@ final class BackupUser extends Command { private TargetService $targets, private UserBackupService $userBackup, private IUserManager $userManager, + private UserMigrationInstaller $installer, ) { parent::__construct(); } @@ -29,6 +32,7 @@ final class BackupUser extends Command { ->setDescription('Back up one user via user_migration (account, settings, files and everything else a migrator covers)') ->addArgument('target', InputArgument::REQUIRED, 'Location name or id') ->addArgument('user', InputArgument::REQUIRED, 'User id') + ->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)') ->setHelp(<<<'HELP' Streams a user_migration export straight into the encrypted repository: no zip, no copy in the user's own storage. Shares data with the regular backup; this needs the user_migration app @@ -47,6 +51,7 @@ HELP); return 1; } try { + $this->installer->ensureEnabled($input, $output); $target = $this->targets->get($input->getArgument('target')); $manifest = $this->userBackup->backupUser($target, $user); } catch (\Throwable $e) { diff --git a/lib/Command/Prune.php b/lib/Command/Prune.php index 6b5bf3b..eee2a1e 100644 --- a/lib/Command/Prune.php +++ b/lib/Command/Prune.php @@ -48,8 +48,9 @@ final class Prune extends Command { continue; } if ($ru['forgotten'] !== []) { + $affected = count(array_unique(array_map(static fn (string $path): string => explode('/', $path)[1] ?? '', $ru['forgotten']))); $output->writeln(sprintf('%s: %s %d user export(s) for %d user(s)', - $t->getName(), $dry ? 'would forget' : 'forgot', count($ru['forgotten']), count($ru['kept']))); + $t->getName(), $dry ? 'would forget' : 'forgot', count($ru['forgotten']), $affected)); } try { $r = $this->prune->apply($t, $dry); diff --git a/lib/Command/RestoreUser.php b/lib/Command/RestoreUser.php index 198097a..c2f86ad 100644 --- a/lib/Command/RestoreUser.php +++ b/lib/Command/RestoreUser.php @@ -9,6 +9,7 @@ namespace OCA\NgBackup\Command; use OCA\NgBackup\Service\TargetService; use OCA\NgBackup\Service\UserRestoreService; +use OCA\NgBackup\UserMigration\UserMigrationInstaller; use Symfony\Component\Console\Command\Command; use Symfony\Component\Console\Input\InputArgument; use Symfony\Component\Console\Input\InputInterface; @@ -19,6 +20,7 @@ final class RestoreUser extends Command { public function __construct( private TargetService $targets, private UserRestoreService $userRestore, + private UserMigrationInstaller $installer, ) { parent::__construct(); } @@ -29,6 +31,7 @@ final class RestoreUser extends Command { ->addArgument('target', InputArgument::REQUIRED, 'Location name or id') ->addArgument('manifest', InputArgument::REQUIRED, 'Manifest path printed by backup:user:backup (users//)') ->addOption('mode', 'm', InputOption::VALUE_REQUIRED, 'replace or as-backup', UserRestoreService::MODE_REPLACE) + ->addOption('install-user-migration', null, InputOption::VALUE_NONE, 'Install and enable user_migration if it is missing (non-interactive consent)') ->setHelp(<<<'HELP' Never imports over a live account. Modes: replace (default) safety-exports the current account first, then deletes and recreates it @@ -45,6 +48,8 @@ HELP); protected function execute(InputInterface $input, OutputInterface $output): int { try { + $this->installer->ensureEnabled($input, $output); + $output->writeln('Note: on restore, user_migration only imports the app settings on its own allowlist (e.g. calendar view and reminder settings); other app settings of this user are skipped. Reason: NG Backup uses user_migration\'s importer; widening the list would mean building a settings importer of our own.'); $target = $this->targets->get($input->getArgument('target')); $user = $this->userRestore->restoreUser($target, (string)$input->getArgument('manifest'), (string)$input->getOption('mode')); } catch (\Throwable $e) { diff --git a/lib/Command/Retention.php b/lib/Command/Retention.php index 43eff92..bd44742 100644 --- a/lib/Command/Retention.php +++ b/lib/Command/Retention.php @@ -50,8 +50,13 @@ final class Retention extends Command { } } if ($input->getOption('user-last') !== null) { + $userLast = (string)$input->getOption('user-last'); + if (!ctype_digit($userLast)) { + $output->writeln('--user-last must be a whole number'); + return 1; + } try { - $this->prune->setUserExportRetention((int)$input->getOption('user-last')); + $this->prune->setUserExportRetention((int)$userLast); } catch (\InvalidArgumentException $e) { $output->writeln('' . $e->getMessage() . ''); return 1; diff --git a/lib/Service/TargetService.php b/lib/Service/TargetService.php index dfa8735..f705414 100644 --- a/lib/Service/TargetService.php +++ b/lib/Service/TargetService.php @@ -152,7 +152,11 @@ final class TargetService { } public function get(string $nameOrId): Target { - return ctype_digit($nameOrId) ? $this->mapper->find((int)$nameOrId) : $this->mapper->findByName($nameOrId); + try { + return ctype_digit($nameOrId) ? $this->mapper->find((int)$nameOrId) : $this->mapper->findByName($nameOrId); + } catch (DoesNotExistException) { + throw new \InvalidArgumentException("No such location: $nameOrId"); + } } /** Size limit for the data stored on this location (null = none). */ diff --git a/lib/UserMigration/UserMigrationInstaller.php b/lib/UserMigration/UserMigrationInstaller.php new file mode 100644 index 0000000..2c902ab --- /dev/null +++ b/lib/UserMigration/UserMigrationInstaller.php @@ -0,0 +1,80 @@ +appManager->isEnabledForAnyone(self::APP); + } + + private function codeIsPresent(): bool { + try { + $this->appManager->getAppPath(self::APP); + return true; + } catch (AppPathNotFoundException) { + return false; + } + } + + /** Installs (from the app store, if missing) and enables user_migration; throws on failure. */ + public function install(): void { + $args = $this->codeIsPresent() ? ['app:enable', self::APP] : ['app:install', self::APP]; + $occ = dirname($this->appManager->getAppPath(Application::APP_ID), 2) . '/occ'; + $command = implode(' ', array_map('escapeshellarg', [PHP_BINARY, $occ, ...$args])) . ' 2>&1'; + exec($command, $lines, $code); + if ($code !== 0) { + throw new \RuntimeException("occ " . implode(' ', $args) . " failed:\n" . implode("\n", $lines)); + } + } + + /** + * Returns when user_migration is enabled. Otherwise asks the administrator (interactive shell) or + * requires --install-user-migration, and throws when neither gives consent. + * + * @throws \RuntimeException + */ + public function ensureEnabled(InputInterface $input, OutputInterface $output): void { + if ($this->isEnabled()) { + return; + } + $consent = $input->getOption('install-user-migration') === true; + if (!$consent && $input->isInteractive()) { + $consent = (new QuestionHelper())->ask($input, $output, new ConfirmationQuestion( + 'The user_migration app is not enabled. Install and enable it now? [y/N] ', false)) === true; + } + if (!$consent) { + throw new \RuntimeException('The user_migration app is not enabled; per-user backup/restore needs it. Run "occ ' + . 'app:install ' . self::APP . '" and "occ app:enable ' . self::APP . '", or repeat with --install-user-migration'); + } + $output->writeln('Installing and enabling user_migration (approved by the administrator)...'); + $this->install(); + $this->appManager->clearAppsCache(); + if (!$this->isEnabled()) { + throw new \RuntimeException('user_migration was installed; run this command again (check with "occ app:list" if it stays disabled)'); + } + } +}