Compare commits
28 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 965440d1f7 | |||
| 3a0f882a38 | |||
| 632d2a2838 | |||
| 9e9234e2a8 | |||
| 3ac3298313 | |||
| 384b2d2792 | |||
| 8032411789 | |||
| b4f20cb070 | |||
| 343389b98b | |||
| 133f5f9d34 | |||
| cf4bd9db7f | |||
| 67cc350b8c | |||
| fb5b234e8a | |||
| fd12c586b6 | |||
| 6ffe979ea7 | |||
| a03b7354b3 | |||
| be1c628f07 | |||
| a8355df12e | |||
| cd251cffb3 | |||
| 0b4872599d | |||
| b7d09c9784 | |||
| 24686897c3 | |||
| 315e83b847 | |||
| b728db36c7 | |||
| 6ed0a555cf | |||
| 221ed9dc17 | |||
| 285153a968 | |||
| 6c8a78f6bb |
@@ -7,6 +7,35 @@ the app store accepts a `-beta` suffix, but Nextcloud servers then only offer th
|
||||
beta/daily update channel, which would hide the app from most admins. The stability stage is
|
||||
instead carried by the app description, the git tag, the release, and this file.
|
||||
|
||||
## [1.0.0-rc.6] - 2026-10-24
|
||||
|
||||
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
|
||||
simulations and a user report. If no problems are reported for about a week, 1.0.0 will be this
|
||||
same code.
|
||||
|
||||
### Fixed
|
||||
- The daily backup time was taken as UTC (Nextcloud runs PHP in UTC): 03:00 ran at 05:00 in
|
||||
Berlin (#19). The time is now saved with the administrator's time zone and shown with it. A time
|
||||
saved before is converted once to the zone of the server or an administrator, so it shows as it
|
||||
effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer) and nothing moves.
|
||||
- A full restore over SMB stopped after 17 hours with "Decryption failed": a read over a long
|
||||
line handed out wrong bytes (the backup itself was fine, the same files restored identically
|
||||
afterwards). A blob that does not decrypt as first read, of a file or of the database, is now
|
||||
read once more on a request of its own, and only a second failure stops the restore.
|
||||
- A full restore onto MySQL/MariaDB stopped with "Incorrect string value" when it created a table
|
||||
(of an app that is not installed here) whose rows hold emoji or other 4-byte characters: the
|
||||
table got 3-byte utf8. Tables the restore creates now get this installation's character set and
|
||||
collation (utf8mb4). Found in a disaster recovery simulation onto MariaDB.
|
||||
|
||||
### Changed
|
||||
- The admin page uses more of the screen (sections up to 1400 pixels instead of 900), so the
|
||||
list of backup runs fits without scrolling sideways.
|
||||
- README: a roadmap, the tested disaster recoveries, measurements for OVH S3, the Storage Box over
|
||||
SFTP and SMB, how to build the SMB extension when PECL is down, and what works without shell
|
||||
access. The guide lists shell access among what you need.
|
||||
- The full-restore integration test checks a 4-byte character in a table the restore creates, and
|
||||
retries app installs from the app store.
|
||||
|
||||
## [1.0.0-rc.5] - 2026-10-22
|
||||
|
||||
Fifth release candidate for 1.0.0; the app version is 0.9.11. A full restore of 1 TB over SFTP
|
||||
|
||||
@@ -395,11 +395,53 @@ in README "Known limitations":
|
||||
- **1.0.1:**
|
||||
1. (Naar rc4: herstelsnelheid over SFTP.) S3 leest al 16 tegelijk bij index en tree, maar een
|
||||
restore leest de packs nog één voor één: ook vooruitlezen via de S3-client.
|
||||
2. Waarschuwing in de beheerpagina en `backup:status` als achtergrondjobs niet op systeem-cron
|
||||
staan, en "waiting for background jobs" als een lopende backup lang geen stap zette (issue
|
||||
#13). Cron zelf aanzetten kan de app niet: zonder echte cron-regel stopt dan alles.
|
||||
2. Achtergrondjobs zonder systeem-cron (issue #13; besluit 2026-10-19). De PR die dit
|
||||
oplost zegt "Fixes #13" (bevestigd: #13 was de cron; besluit 2026-10-21):
|
||||
a. **Waarschuwen** in de beheerpagina en `backup:status`: modus uit `core backgroundjobs_mode`
|
||||
(`ajax`/`webcron`/`cron`) en de laatste run uit `core lastcron`. Niet op `cron`, of de
|
||||
laatste run te lang geleden: melding met de cron-regel erbij ("Backups only move while
|
||||
someone has Nextcloud open"), en "waiting for background jobs" als een lopende backup lang
|
||||
geen stap zette.
|
||||
b. **De pagina laat de backup lopen** zolang ze openstaat: in `ajax`/`webcron` voert de
|
||||
statusaanroep die de pagina toch al herhaalt ook een korte, begrensde stap uit (zoals de
|
||||
korte stappen die AJAX-cron nu al krijgt). "Back up now" werkt dan ook zonder cron zolang
|
||||
het tabblad openstaat; de melding zegt dat geplande backups echte cron nodig hebben.
|
||||
De melding zegt (zonder werkende systeem-cron): backups hebben systeem-cron nodig, ook de
|
||||
gewone nachtelijke; zo stel je cron in (link naar de Nextcloud-documentatie over
|
||||
achtergrondjobs voor de versie van de server, nu
|
||||
https://docs.nextcloud.com/server/34/admin_manual/configuration_server/background_jobs_configuration.html,
|
||||
versienummer uit de server, niet vast); zolang er geen cron is, loopt een backup die je
|
||||
met de knop start alleen door terwijl de NG Backup-pagina openstaat.
|
||||
c. **Voortgang in de databasefase** (issue #13): nu staat er alleen "running (db)", wat er
|
||||
hetzelfde uitziet als vastzitten. Toon "running (db: 47/183 tables)" zoals de bestandsfase
|
||||
een percentage toont, en "waiting for background jobs (last ran 12 min ago)" als een run
|
||||
een tijd geen stap zette, ook bij een cron die is blijven staan.
|
||||
Niet doen: de cron-modus vanuit de app omzetten (zonder echte cron-regel stopt alles, en
|
||||
`cron.php` zet de modus zelf op `cron`), en geen `exec php occ backup:run &` vanuit de knop
|
||||
(vaak uitgeschakeld, niet op hosted, buiten Nextcloud om lastig te volgen en te stoppen).
|
||||
3. Volledige DR-test met een backup die al de tabelstructuur bevat (gemaakt met rc2 of later),
|
||||
zonder `--skip-missing-tables`: dat pad is op 1 TB nog niet getest.
|
||||
4. **Label bij "Back up now"** (besluit 2026-10-19): een optioneel tekstveld naast de knop; de
|
||||
tekst gaat als label van de snapshot mee (nu kan dat alleen met `occ backup:run -l`; de knop
|
||||
stuurt een leeg label, geplande runs krijgen "scheduled"). Raakt alleen de UI en een
|
||||
bestaande parameter.
|
||||
5. **`--salvage`** voor `backup:restore` en `backup:restore:full` (besluit 2026-10-22): terughalen
|
||||
wat kan, een bestand dat niet te lezen of te ontsleutelen is overslaan en aan het eind
|
||||
opsommen, in plaats van stoppen.
|
||||
6. **Tijdelijke bestanden van een afgebroken restore opruimen** (gevonden in de SMB-DR van
|
||||
rc6, 2026-10-24): de rc5-poging die stopte liet `VID_0009.mp4.ngb-restore-<id>` staan; de
|
||||
geslaagde restore daarna haalde het niet weg, en `files:scan` maakte het zichtbaar in Files.
|
||||
Een restore ruimt `*.ngb-restore-*` in de mappen die hij terugzet op.
|
||||
7. **Eén bestand terugzetten met `backup:restore`**: een pad naar een bestand zet 0 bestanden
|
||||
terug (er komt een `/` achter); nu werkt alleen een map.
|
||||
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
|
||||
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
|
||||
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
|
||||
(MariaDB-bron, 138 GB) nogmaals naar een MariaDB-doel en de SMB-simulatie naar een SQLite-doel;
|
||||
**na 1.0.0** de 1 TB van nc01 (PostgreSQL) naar een MariaDB-doel.
|
||||
- **Polish:** `target:add` toonde bij het eerste indexbestand al "about 642 min left" (S3,
|
||||
4.568 bestanden, daarna binnen een minuut klaar): een schatting pas na ~5% of na meer dan
|
||||
een paar bestanden.
|
||||
- **1.1.0:**
|
||||
1. `backup:key:import` zonder kit: de sleutel-slots van de locatie (`config`) lezen, met de kit
|
||||
als reserve als `config` ontbreekt of niet opent. Vergt een andere volgorde van
|
||||
@@ -412,6 +454,22 @@ in README "Known limitations":
|
||||
3. Snelheid: lezen en uploaden laten overlappen, meerdere bestanden tegelijk lezen, eventueel
|
||||
een eigen worker (`occ background-job:worker`). Gemeten op nc01: 55% wachten op de schijf,
|
||||
33% op de SMB-upload, ~7% CPU. Vergt extra PHP-processen; botst mogelijk met uitgangspunt 1.
|
||||
4. **Schema per locatie** (besluit 2026-10-19): in de instellingen van een locatie "uit",
|
||||
"dagelijks om HH:MM" of "elke N uur (vanaf HH:MM)". Nu is er één `schedule_time` voor alle
|
||||
locaties, één keer per dag. Bij de update wordt de huidige tijd het schema van elke
|
||||
bestaande locatie (voor bestaande gebruikers verandert er niets).
|
||||
5. **Locaties na elkaar laten starten** in plaats van tegelijk: twee runs die dezelfde schijven
|
||||
lezen zijn samen niet sneller (gezien op nc01); wachtende runs zichtbaar als "queued".
|
||||
7. **SMB sneller** (gemeten 2026-10-21): lokaal netwerk zonder vertraging (Samba-container op de
|
||||
laptop, 1,8 GB) back-up 24,9 MB/s, restore 27,6 MiB/s; vanaf Canada naar de Storage Box (130 ms)
|
||||
2,6 MiB/s, omdat SMB per blok op het antwoord wacht. Grotere schrijfblokken, en vooruitlezen
|
||||
bij een restore zoals SFTP (IReadsAhead) of meerdere packs tegelijk. Een NAS met 1 Gbit haalt
|
||||
~110 MB/s: daar is winst.
|
||||
Ook de index: `target:add` over SMB vanaf Canada haalt ~1 indexbestand per seconde (4.927
|
||||
bestanden ~85 min), want SMB krijgt geen `getMany()` met verzoeken tegelijk zoals S3 en SFTP.
|
||||
6. **Notitie achteraf bij een restorepunt**, te wijzigen in de lijst. Het label in de snapshot
|
||||
is versleuteld en ligt vast; de notitie apart op de locatie bewaren (bijv. in de catalogus),
|
||||
zodat een nieuwe server na een DR hem ook ziet.
|
||||
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
|
||||
PostgreSQL 18)** — voor **1.0.0-rc.2**:
|
||||
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
|
||||
|
||||
@@ -146,10 +146,31 @@ What you should do on the location (strongly recommended):
|
||||
- **Files and folders** of a user: in the web interface or `occ backup:restore`, into a new folder
|
||||
("Restored <date>"), merged into the original place, or replacing it.
|
||||
- **Raw restore** of any path (for example `config/`) to a local directory: `occ backup:restore ... --to-directory=DIR`.
|
||||
- **Whole server** (disaster recovery onto a fresh installation): `occ backup:restore:full`, see the how-to.
|
||||
- **Whole server** (disaster recovery onto a fresh installation): `occ backup:restore:full`, see
|
||||
[docs/DISASTER-RECOVERY.md](docs/DISASTER-RECOVERY.md). It runs in maintenance mode, so that
|
||||
nobody and no background job writes into the instance while it is being replaced.
|
||||
- **One user** (account, settings, files, calendars, contacts via user_migration): `occ backup:user:backup` and
|
||||
`occ backup:user:restore`, replacing the account or restoring it as `<uid>-bak`. See the how-to.
|
||||
|
||||
**Without shell access** (a hosted Nextcloud where you cannot run `occ`): files and folders can be
|
||||
restored in the web interface. Restoring a whole user or the whole server needs `occ`, so on such
|
||||
hosting it is something your provider (or whoever has shell access) runs. A restore mode that
|
||||
works without maintenance mode, and so from the web interface, is on the roadmap.
|
||||
|
||||
## Tested disaster recoveries
|
||||
|
||||
Full restores onto a fresh server, following only [the guide](docs/DISASTER-RECOVERY.md):
|
||||
|
||||
| Backup | Location | Database | Result |
|
||||
|---|---|---|---|
|
||||
| Production, about 1 TB, 150,000 files (Nextcloud All-in-One) | Hetzner Storage Box (SFTP), Germany → Canada | PostgreSQL → PostgreSQL | 150,622 files restored in about 25 h (13-14 MiB/s), sample identical |
|
||||
| 138 GB simulation: 20 users, calendars, contacts, mail, shares, team folders with an ACL, versions, trash | OVH S3, Warsaw → Canada | MariaDB → PostgreSQL | all 15,772 files and every database count identical |
|
||||
| The same | OVH S3 | MariaDB → MariaDB | all 15,772 files and every database count identical |
|
||||
| The same | Hetzner Storage Box (SMB), Germany → Canada | MariaDB → SQLite | all 15,772 files and every database count identical (27.6 h, about 1.5 MB/s) |
|
||||
| 51 GB test instance | OVH S3 | PostgreSQL → PostgreSQL | 23,519/23,519 checksums identical |
|
||||
|
||||
The integration tests restore across every pair of PostgreSQL, MariaDB and SQLite.
|
||||
|
||||
## Storage backend experiences
|
||||
|
||||
Notes from actually running a backend, beyond what's in files_external's own documentation.
|
||||
@@ -206,12 +227,64 @@ Contributions welcome via PR — add what you ran into with your own provider.
|
||||
can be read on more connections: `occ config:app:set ng_backup restore_sftp_connections
|
||||
--value=4` (at most 8; the server must allow that many plus one at once for the user).
|
||||
|
||||
- **OVH Object Storage (S3, Warsaw, 2026-10):** works with the `amazons3` backend (path style,
|
||||
region `waw`). A full backup of 1 TB from Germany ran at about 7 MiB/s, from Canada about
|
||||
6 MiB/s with BBR; a check of a 1 TB backup took 3.5 minutes (index files are fetched 16 at a
|
||||
time); a restore of 138 GB from Warsaw to Canada took about 6 hours. Turn on versioning and
|
||||
Object Lock on the bucket (see above); OVH charges no egress.
|
||||
|
||||
- **Hetzner Storage Box over SFTP, full restore:** about 13-14 MiB/s from Germany to Canada
|
||||
(122 ms) with the read-ahead of 1.0.0-rc.4; without it about 2.8 MiB/s. A Storage Box gets
|
||||
slower with several connections reading at once (4-6 MiB/s for 2-4 connections, it reads from
|
||||
disks), so one extra connection is the default. A Storage Box drops an SFTP connection that
|
||||
stayed quiet for a while without telling the client; NG Backup checks and replaces it.
|
||||
|
||||
- **SMB speed:** on a local network (Samba, no delay) a backup ran at about 25 MB/s and a
|
||||
restore at about 28 MiB/s. Over a long line SMB waits for an answer to each request: from
|
||||
Canada to a Storage Box in Germany (130 ms) about 2.6 MiB/s for a backup and 1.6 MiB/s for a
|
||||
restore, and connecting a location with an existing backup reads its index at about one file a
|
||||
second. SMB is meant for a local network; use SFTP or S3 for a location far away. When
|
||||
`pecl install smbclient` fails (pecl.php.net was unreachable for hours on 2026-10-21), the
|
||||
extension builds from its source: `libsmbclient-php` on GitHub, `phpize && ./configure && make
|
||||
&& make install`, then `docker-php-ext-enable smbclient`.
|
||||
|
||||
- **NFS is not a files_external backend at all.** Mount the NFS export on the server yourself
|
||||
(e.g. via `/etc/fstab` or a systemd `.mount` unit) and point ng_backup's `local` backend at the
|
||||
mount point; ng_backup never mounts or unmounts anything on its own, so an NFS mount dropping
|
||||
mid-run behaves like any other local-path failure (the run resumes). See PLAN.md 3.5 for why NFS
|
||||
can't be driven from PHP directly.
|
||||
|
||||
## Roadmap
|
||||
|
||||
What is planned, without dates. Ideas and votes are welcome in the
|
||||
[issues](https://github.com/dillardblom/ng_backup/issues); if something you need is listed here,
|
||||
a 👍 on its issue helps decide what comes first.
|
||||
|
||||
**1.0.1**
|
||||
- A warning on the admin page and in `occ backup:status` when background jobs do not run on a
|
||||
system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on
|
||||
while the NG Backup page is open ([#13](https://github.com/dillardblom/ng_backup/issues/13)).
|
||||
- An optional label for a backup started with "Back up now".
|
||||
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
|
||||
|
||||
**1.1.0**
|
||||
- Restore without the recovery kit: the backup key from the location itself, opened with one of
|
||||
your passphrases.
|
||||
- A schedule per location, also "every N hours".
|
||||
- Locations backed up one after the other instead of at the same time.
|
||||
- A note on a restore point that you can change afterwards.
|
||||
- An option to pause heavy background jobs (such as Memories, Recognize or the Preview
|
||||
Generator) while a backup runs.
|
||||
- Faster backups: reading and uploading at the same time.
|
||||
|
||||
**Later**
|
||||
- A full restore onto the existing installation, without first installing a fresh Nextcloud, in a
|
||||
restore mode of its own instead of maintenance mode, so that it can also run from the web
|
||||
interface (for hosted Nextcloud without shell access).
|
||||
- Users restoring their own files from the Files app.
|
||||
- Primary object storage and External Storage mounts (see Known limitations).
|
||||
- An option not to keep the backup key on the location (the recovery kit is then the only copy).
|
||||
|
||||
## Known limitations
|
||||
|
||||
On the backlog; none of these block the beta or 1.0.0, but know them before you rely on it:
|
||||
@@ -236,8 +309,11 @@ On the backlog; none of these block the beta or 1.0.0, but know them before you
|
||||
- Location passwords passed with `-o password=...` are visible in the process list and shell
|
||||
history while the command runs.
|
||||
- Temporary files of a restore into a user's files end up in that user's trash.
|
||||
- Restoring the database does not order tables by foreign keys; apps with cascading foreign keys
|
||||
could lose rows (Nextcloud core does not use them).
|
||||
- **Restoring a whole user or the whole server needs `occ`** (shell access); see "Without shell
|
||||
access" above.
|
||||
- **`--install-apps` depends on the app store.** Nextcloud downloads the whole app list (about
|
||||
38 MB) and gives up after 120 seconds; on a slow day apps are not installed. The restore then
|
||||
lists them: install them afterwards with `occ app:install`, their data is restored already.
|
||||
- A pack whose index upload failed stays on the location until removed by hand.
|
||||
|
||||
## Commands
|
||||
|
||||
+4
-4
@@ -42,7 +42,7 @@ welcome: use the "Share your experience" link on the admin page.
|
||||
|
||||
Developed and sponsored by [Ensembia](https://www.ensembia.com). We can offer (paid) assistance if
|
||||
you need help with restoring a backup after a crash.]]></description>
|
||||
<version>0.9.11</version>
|
||||
<version>0.9.12</version>
|
||||
<licence>agpl</licence>
|
||||
<author mail="dillard.blom@ensembia.com">Dillard Blom</author>
|
||||
<namespace>NgBackup</namespace>
|
||||
@@ -53,9 +53,9 @@ you need help with restoring a backup after a crash.]]></description>
|
||||
<website>https://github.com/dillardblom/ng_backup</website>
|
||||
<bugs>https://github.com/dillardblom/ng_backup/issues</bugs>
|
||||
<repository type="git">https://github.com/dillardblom/ng_backup</repository>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.5/screenshots/1-backups.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.5/screenshots/2-restore.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.5/screenshots/3-settings.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/1-backups.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/2-restore.png</screenshot>
|
||||
<screenshot>https://raw.githubusercontent.com/dillardblom/ng_backup/v1.0.0-rc.6/screenshots/3-settings.png</screenshot>
|
||||
<dependencies>
|
||||
<php min-version="8.1"/>
|
||||
<lib>sodium</lib>
|
||||
|
||||
@@ -10,6 +10,8 @@ Nextcloud back from an NG Backup location. Do them in this order.
|
||||
- [ ] A new server with the **same Nextcloud major version** as the old one (for example 34),
|
||||
a database (PostgreSQL, MySQL/MariaDB or SQLite; it may differ from the old one) and room for
|
||||
the whole backup plus a margin.
|
||||
- [ ] **Shell access** to the new server, to run `occ` (on a hosted Nextcloud without it, your
|
||||
provider runs these steps).
|
||||
- [ ] A list of the apps that are **not in the app store** (your own builds, apps copied in by
|
||||
hand). Keep their packages somewhere safe beforehand.
|
||||
|
||||
|
||||
@@ -32,6 +32,7 @@ final class BackupJob extends TimedJob {
|
||||
private RunMapper $runs,
|
||||
private IAppConfig $appConfig,
|
||||
private LoggerInterface $logger,
|
||||
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||
) {
|
||||
parent::__construct($time);
|
||||
// Well below the usual 5-minute cron interval: the last run is stamped a moment after cron
|
||||
@@ -101,14 +102,10 @@ final class BackupJob extends TimedJob {
|
||||
$this->appConfig->setValueString(Application::APP_ID, 'last_prune_week', $due);
|
||||
}
|
||||
|
||||
/** Daily schedule: appconfig ng_backup/schedule_time "HH:MM" (server time); empty = no schedule. */
|
||||
/** Daily schedule: "HH:MM" in the schedule's time zone (see Schedule); empty = no schedule. */
|
||||
private function startScheduled(): void {
|
||||
$at = $this->appConfig->getValueString(Application::APP_ID, 'schedule_time', '');
|
||||
if (!preg_match('/^([01]\d|2[0-3]):[0-5]\d$/', $at)) {
|
||||
return;
|
||||
}
|
||||
$todayAt = (int)strtotime(date('Y-m-d') . ' ' . $at);
|
||||
if (time() < $todayAt) {
|
||||
$todayAt = $this->schedule->todayAt();
|
||||
if ($todayAt === null || time() < $todayAt) {
|
||||
return;
|
||||
}
|
||||
foreach ($this->targets->list() as $t) {
|
||||
|
||||
@@ -47,6 +47,7 @@ final class AdminApiController extends Controller {
|
||||
private \OCA\NgBackup\Service\AlertService $alerts,
|
||||
private \OCA\NgBackup\Service\KeySlotService $slots,
|
||||
private \OCA\NgBackup\Service\PruneService $prune,
|
||||
private \OCA\NgBackup\Service\Schedule $schedule,
|
||||
) {
|
||||
parent::__construct(Application::APP_ID, $request);
|
||||
}
|
||||
@@ -72,7 +73,8 @@ final class AdminApiController extends Controller {
|
||||
'runs' => $runs,
|
||||
'olderRuns' => $allRuns ? 0 : $this->runs->countAll() - count($shown),
|
||||
'recentDays' => \OCA\NgBackup\Command\Status::RECENT_DAYS,
|
||||
'schedule' => $this->appConfig->getValueString(Application::APP_ID, 'schedule_time', ''),
|
||||
'schedule' => $this->schedule->get()['time'],
|
||||
'scheduleTimezone' => $this->schedule->get()['timezone'],
|
||||
]);
|
||||
}
|
||||
|
||||
@@ -240,12 +242,14 @@ final class AdminApiController extends Controller {
|
||||
return new JSONResponse($this->runJson($run));
|
||||
}
|
||||
|
||||
public function setSchedule(string $time): JSONResponse {
|
||||
if ($time !== '' && !preg_match('/^([01]\d|2[0-3]):[0-5]\d$/', $time)) {
|
||||
return new JSONResponse(['error' => 'Use HH:MM or empty'], Http::STATUS_BAD_REQUEST);
|
||||
/** @param string $timezone the administrator's zone (the browser's); empty: keep the current one */
|
||||
public function setSchedule(string $time, string $timezone = ''): JSONResponse {
|
||||
try {
|
||||
$this->schedule->set($time, $timezone !== '' ? $timezone : $this->schedule->get()['timezone']);
|
||||
} catch (\InvalidArgumentException $e) {
|
||||
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
|
||||
}
|
||||
$this->appConfig->setValueString(Application::APP_ID, 'schedule_time', $time);
|
||||
return new JSONResponse(['schedule' => $time]);
|
||||
return new JSONResponse(['schedule' => $time, 'scheduleTimezone' => $this->schedule->get()['timezone']]);
|
||||
}
|
||||
|
||||
public function snapshots(int $targetId): JSONResponse {
|
||||
|
||||
+26
-3
@@ -159,7 +159,7 @@ final class DbDumper {
|
||||
// everywhere); tables it does not know are left out, as asked.
|
||||
foreach ($missing as $table) {
|
||||
if (isset($manifest['tables'][$table]['schema'])) {
|
||||
$this->db->createSchemaManager()->createTable(self::tableFromStructure($table, $manifest['tables'][$table]['schema']));
|
||||
$this->db->createSchemaManager()->createTable(self::tableFromStructure($table, $manifest['tables'][$table]['schema'], $this->tableOptions()));
|
||||
$report['created'][] = $table;
|
||||
} else {
|
||||
$report['missing'][] = $table;
|
||||
@@ -319,8 +319,31 @@ final class DbDumper {
|
||||
'foreignKeys' => $foreignKeys];
|
||||
}
|
||||
|
||||
private static function tableFromStructure(string $name, array $structure): \Doctrine\DBAL\Schema\Table {
|
||||
$table = new \Doctrine\DBAL\Schema\Table($name);
|
||||
/**
|
||||
* Table options for a table the restore creates: those of this installation's own tables.
|
||||
* On MySQL/MariaDB that is the character set and collation, as Nextcloud's tables have them
|
||||
* (utf8mb4, so emoji and other 4-byte characters fit); DBAL alone would use 3-byte utf8.
|
||||
*
|
||||
* @return array<string, mixed>
|
||||
* @psalm-suppress UndefinedClass the DBAL classes come with Nextcloud, not with this app
|
||||
*/
|
||||
private function tableOptions(): array {
|
||||
$options = $this->db->createSchemaManager()->createSchemaConfig()->getDefaultTableOptions();
|
||||
if ($this->db->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\AbstractMySQLPlatform) {
|
||||
$collation = $this->db->fetchOne('SELECT TABLE_COLLATION FROM information_schema.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = ?',
|
||||
[$this->prefix . 'appconfig']);
|
||||
if (is_string($collation) && $collation !== '') {
|
||||
$options['collation'] = $collation;
|
||||
$options['charset'] = explode('_', $collation, 2)[0];
|
||||
unset($options['collate']);
|
||||
}
|
||||
}
|
||||
return $options;
|
||||
}
|
||||
|
||||
/** @param array<string, mixed> $options table options (see tableOptions()) */
|
||||
private static function tableFromStructure(string $name, array $structure, array $options = []): \Doctrine\DBAL\Schema\Table {
|
||||
$table = new \Doctrine\DBAL\Schema\Table($name, [], [], [], [], $options);
|
||||
foreach ($structure['columns'] as $c) {
|
||||
$table->addColumn($c['name'], $c['type'], array_filter([
|
||||
'length' => $c['length'], 'precision' => $c['precision'], 'scale' => $c['scale'],
|
||||
|
||||
@@ -845,7 +845,15 @@ final class Repository {
|
||||
try {
|
||||
foreach ($this->backend->getRanges($ranges) as $i => $data) {
|
||||
foreach ($parts[$i] as [$id, $at, $length]) {
|
||||
yield $this->decodeBlob($id, substr($data, $at, $length));
|
||||
try {
|
||||
$blob = $this->decodeBlob($id, substr($data, $at, $length));
|
||||
} catch (\Exception) {
|
||||
// a read that went wrong on the way (see loadBlob()): this blob on its own
|
||||
$blob = $this->loadBlob($id);
|
||||
// its pack stream is closed before the next range is read on the same connection
|
||||
$this->packReader->close();
|
||||
}
|
||||
yield $blob;
|
||||
$done++;
|
||||
}
|
||||
}
|
||||
@@ -895,8 +903,46 @@ final class Repository {
|
||||
} catch (BackendException) {
|
||||
throw $e;
|
||||
}
|
||||
$pack = $moved;
|
||||
}
|
||||
return $this->decodeBlob($id, $encrypted);
|
||||
try {
|
||||
return $this->decodeBlob($id, $encrypted);
|
||||
} catch (\Exception $damaged) {
|
||||
// Bytes that do not decrypt: damaged on the location, or a read that went wrong on the
|
||||
// way (a stream that lost its place: an SMB restore stopped after 17 hours like this).
|
||||
// Read exactly this blob once more, on a request of its own; only a second failure
|
||||
// means the data itself is damaged.
|
||||
$this->packReader->close();
|
||||
try {
|
||||
$again = $this->backend->getRange('packs/' . substr($pack, 0, 2) . '/' . $pack, $offset, $length);
|
||||
} catch (\Exception) {
|
||||
throw $damaged;
|
||||
}
|
||||
$data = $this->decodeBlob($id, $again);
|
||||
if (function_exists('OCP\Log\logger')) {
|
||||
\OCP\Log\logger('ng_backup')->warning('NG Backup: blob ' . $id . ' did not decrypt as first read from pack ' . $pack . ' at ' . $offset . '; read again, it is fine. First read: ' . self::howItDiffered($encrypted, $again));
|
||||
}
|
||||
return $data;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* What a read that went wrong looked like next to the right bytes, for the log: where it starts
|
||||
* to differ, and whether the rest is zeros or the right bytes from another place (a stream that
|
||||
* lost its place).
|
||||
*/
|
||||
private static function howItDiffered(string $wrong, string $right): string {
|
||||
$at = strspn($wrong ^ $right, "\0");
|
||||
$text = strlen($wrong) . ' bytes, differ from byte ' . $at . ' of ' . strlen($right);
|
||||
$tail = substr($wrong, $at, 64);
|
||||
if ($tail === '') {
|
||||
return $text . ' (too short)';
|
||||
}
|
||||
if (trim($tail, "\0") === '') {
|
||||
return $text . ' (zeros)';
|
||||
}
|
||||
$found = strpos($right, $tail);
|
||||
return $text . ($found !== false ? ' (the right bytes from ' . ($found - $at) . ' bytes further on)' : ' (other bytes)');
|
||||
}
|
||||
|
||||
/** Decrypts and checks a blob read from its pack; throws when it is corrupted or tampered with. */
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Service;
|
||||
|
||||
use OCA\NgBackup\AppInfo\Application;
|
||||
use OCP\IAppConfig;
|
||||
use OCP\IConfig;
|
||||
use OCP\IGroupManager;
|
||||
|
||||
/**
|
||||
* The daily backup time: "HH:MM" in a time zone. Nextcloud runs PHP in UTC, so a time without a
|
||||
* zone meant UTC (issue #19: 03:00 ran at 05:00 in Berlin). A time saved before there was a zone
|
||||
* is converted once to the zone of the server or of an administrator, at today's offset: it then
|
||||
* shows as it effectively ran (02:30 UTC becomes 04:30 Europe/Berlin in summer).
|
||||
*/
|
||||
final class Schedule {
|
||||
public const TIME_KEY = 'schedule_time';
|
||||
public const ZONE_KEY = 'schedule_timezone';
|
||||
|
||||
public function __construct(
|
||||
private IAppConfig $appConfig,
|
||||
private IConfig $config,
|
||||
private IGroupManager $groups,
|
||||
) {
|
||||
}
|
||||
|
||||
/** @return array{time: string, timezone: non-empty-string} time "" = no schedule */
|
||||
public function get(): array {
|
||||
$time = $this->appConfig->getValueString(Application::APP_ID, self::TIME_KEY, '');
|
||||
$zone = $this->appConfig->getValueString(Application::APP_ID, self::ZONE_KEY, '');
|
||||
if ($zone === '' || !self::validZone($zone)) {
|
||||
$zone = $this->guessZone();
|
||||
if ($time !== '' && self::validTime($time)) {
|
||||
$time = self::convert($time, 'UTC', $zone);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
|
||||
}
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
|
||||
}
|
||||
return ['time' => $time, 'timezone' => $zone];
|
||||
}
|
||||
|
||||
/** @throws \InvalidArgumentException for a malformed time or an unknown zone */
|
||||
public function set(string $time, string $zone): void {
|
||||
if ($time !== '' && !self::validTime($time)) {
|
||||
throw new \InvalidArgumentException('Use HH:MM or empty');
|
||||
}
|
||||
if (!self::validZone($zone)) {
|
||||
throw new \InvalidArgumentException('Unknown time zone: ' . $zone);
|
||||
}
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::ZONE_KEY, $zone);
|
||||
$this->appConfig->setValueString(Application::APP_ID, self::TIME_KEY, $time);
|
||||
}
|
||||
|
||||
/** Today's scheduled moment as a Unix time, in the schedule's zone; null without a schedule. */
|
||||
public function todayAt(?int $now = null): ?int {
|
||||
['time' => $time, 'timezone' => $zone] = $this->get();
|
||||
if ($time === '' || !self::validTime($time)) {
|
||||
return null;
|
||||
}
|
||||
$tz = new \DateTimeZone($zone);
|
||||
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone($tz)->format('Y-m-d');
|
||||
return (new \DateTimeImmutable($day . ' ' . $time, $tz))->getTimestamp();
|
||||
}
|
||||
|
||||
/**
|
||||
* "HH:MM" in zone $from as "HH:MM" in zone $to, at today's offsets.
|
||||
*
|
||||
* @param non-empty-string $from
|
||||
* @param non-empty-string $to
|
||||
*/
|
||||
public static function convert(string $time, string $from, string $to, ?int $now = null): string {
|
||||
$day = (new \DateTimeImmutable('@' . ($now ?? time())))->setTimezone(new \DateTimeZone($from))->format('Y-m-d');
|
||||
return (new \DateTimeImmutable($day . ' ' . $time, new \DateTimeZone($from)))->setTimezone(new \DateTimeZone($to))->format('H:i');
|
||||
}
|
||||
|
||||
public static function validTime(string $time): bool {
|
||||
return preg_match('/^([01]\d|2[0-3]):[0-5]\d$/', $time) === 1;
|
||||
}
|
||||
|
||||
/** @psalm-assert-if-true non-empty-string $zone */
|
||||
public static function validZone(string $zone): bool {
|
||||
return in_array($zone, \DateTimeZone::listIdentifiers(), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* The server's default_timezone when set, else the first administrator's, else UTC. A server
|
||||
* zone of UTC is passed over on purpose: it is mostly what an image or installer left there,
|
||||
* while the administrator who set the time read it as their own (#19).
|
||||
*
|
||||
* @return non-empty-string
|
||||
*/
|
||||
private function guessZone(): string {
|
||||
$zone = $this->config->getSystemValueString('default_timezone', '');
|
||||
if ($zone !== '' && $zone !== 'UTC' && self::validZone($zone)) {
|
||||
return $zone;
|
||||
}
|
||||
foreach ($this->groups->get('admin')?->getUsers() ?? [] as $user) {
|
||||
$zone = $this->config->getUserValue($user->getUID(), 'core', 'timezone', '');
|
||||
if ($zone !== '' && self::validZone($zone)) {
|
||||
return $zone;
|
||||
}
|
||||
}
|
||||
return 'UTC';
|
||||
}
|
||||
}
|
||||
@@ -89,8 +89,10 @@ onBeforeUnmount(() => clearTimeout(timer))
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
/* NcSettingsSection sets its own width (at most 900px): the runs table needs more */
|
||||
.ng-backup :deep(.settings-section) {
|
||||
max-width: 900px;
|
||||
width: min(1400px, 100% - var(--default-grid-baseline) * 7 * 2);
|
||||
max-width: none;
|
||||
}
|
||||
.ngb-feedback {
|
||||
color: var(--color-text-maxcontrast);
|
||||
|
||||
+1
-1
@@ -33,7 +33,7 @@ export default {
|
||||
testTarget: (id) => call('post', `/targets/${id}/test`),
|
||||
setTargetLimit: (id, maxGb) => call('put', `/targets/${id}/limit`, { maxGb }),
|
||||
startBackup: (targetId, label) => call('post', '/backups', { targetId, label }),
|
||||
setSchedule: (time) => call('put', '/schedule', { time }),
|
||||
setSchedule: (time, timezone) => call('put', '/schedule', { time, timezone }),
|
||||
snapshots: (targetId) => call('get', `/targets/${targetId}/snapshots`),
|
||||
browse: (targetId, snapshotId, path) => call('get', `/targets/${targetId}/snapshots/${snapshotId}/browse`, { path }),
|
||||
startRestore: (targetId, snapshotId, path, mode) => call('post', '/restores', { targetId, snapshotId, path, mode }),
|
||||
|
||||
@@ -6,8 +6,8 @@
|
||||
<NcSettingsSection :name="t('ng_backup', 'Backups')"
|
||||
:description="t('ng_backup', 'Backups run in the background through Nextcloud\'s cron. Unchanged files are not read again; only new data is uploaded.')">
|
||||
<div class="ngb-row">
|
||||
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, empty = off)')" />
|
||||
<NcButton :disabled="schedule === status.schedule" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||
<NcTextField v-model="schedule" class="ngb-time" :label="t('ng_backup', 'Daily at (HH:MM, {zone}, empty = off)', { zone: browserZone })" />
|
||||
<NcButton :disabled="schedule === status.schedule && browserZone === status.scheduleTimezone" @click="saveSchedule">{{ t('ng_backup', 'Save') }}</NcButton>
|
||||
</div>
|
||||
<div class="ngb-row">
|
||||
<NcButton v-for="target in status.targets" :key="target.id" :disabled="isRunning(target.id)" @click="start(target)">
|
||||
@@ -79,15 +79,19 @@ import NcNoteCard from '@nextcloud/vue/components/NcNoteCard'
|
||||
import NcSettingsSection from '@nextcloud/vue/components/NcSettingsSection'
|
||||
import NcTextField from '@nextcloud/vue/components/NcTextField'
|
||||
import api from '../api.js'
|
||||
import { formatDateTime, formatDate, formatSize } from '../format.js'
|
||||
import { formatDateTime, formatDate, formatSize, toZone } from '../format.js'
|
||||
import { confirmPassword } from '@nextcloud/password-confirmation'
|
||||
import { reactive } from 'vue'
|
||||
|
||||
const props = defineProps({ status: { type: Object, required: true }, allRuns: { type: Boolean, default: false } })
|
||||
const emit = defineEmits(['changed', 'show-all-runs'])
|
||||
const schedule = ref(props.status.schedule)
|
||||
// The time is saved in the administrator's own zone (the browser's); a schedule saved in another
|
||||
// zone is shown in this one, so it reads as the moment it runs here.
|
||||
const browserZone = Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'
|
||||
const shown = (s) => (s && props.status.scheduleTimezone && props.status.scheduleTimezone !== browserZone) ? toZone(s, props.status.scheduleTimezone, browserZone) : s
|
||||
const schedule = ref(shown(props.status.schedule))
|
||||
const error = ref('')
|
||||
watch(() => props.status.schedule, (s) => { schedule.value = s })
|
||||
watch(() => [props.status.schedule, props.status.scheduleTimezone], ([s]) => { schedule.value = shown(s) })
|
||||
|
||||
const trash = reactive({})
|
||||
async function loadTrash() {
|
||||
@@ -131,7 +135,7 @@ async function start(target) {
|
||||
async function saveSchedule() {
|
||||
error.value = ''
|
||||
try {
|
||||
await api.setSchedule(schedule.value.trim())
|
||||
await api.setSchedule(schedule.value.trim(), browserZone)
|
||||
emit('changed')
|
||||
} catch (e) {
|
||||
error.value = e.message
|
||||
|
||||
@@ -26,3 +26,30 @@ export function formatSize(bytes) {
|
||||
}
|
||||
return i === 0 ? `${b} B` : `${b.toFixed(1)} ${units[i]}`
|
||||
}
|
||||
|
||||
/**
|
||||
* A time of day "HH:MM" in zone `from`, as "HH:MM" in zone `to`, at today's offsets.
|
||||
*
|
||||
* @param {string} hhmm the time
|
||||
* @param {string} from IANA zone it is in
|
||||
* @param {string} to IANA zone to show it in
|
||||
* @param {Date} now the day to convert on (today)
|
||||
* @return {string}
|
||||
*/
|
||||
export function toZone(hhmm, from, to, now = new Date()) {
|
||||
const [h, m] = hhmm.split(':').map(Number)
|
||||
const parts = (zone, date) => Object.fromEntries(new Intl.DateTimeFormat('en-GB', {
|
||||
timeZone: zone, hourCycle: 'h23', year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit',
|
||||
}).formatToParts(date).map((p) => [p.type, p.value]))
|
||||
const offsetAt = (at) => {
|
||||
const seen = parts(from, new Date(at))
|
||||
return Date.UTC(+seen.year, +seen.month - 1, +seen.day, +seen.hour, +seen.minute) - at
|
||||
}
|
||||
const today = parts(from, now)
|
||||
const wall = Date.UTC(+today.year, +today.month - 1, +today.day, h, m)
|
||||
// the offset at the wall time itself: on a day the clocks change it differs from the one at
|
||||
// midnight, so it is looked up a second time at the moment found first
|
||||
const at = wall - offsetAt(wall - offsetAt(wall))
|
||||
const there = parts(to, new Date(at))
|
||||
return `${there.hour}:${there.minute}`
|
||||
}
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Repository;
|
||||
|
||||
use OCA\NgBackup\Backend\IBackend;
|
||||
use OCA\NgBackup\Backend\LocalBackend;
|
||||
use OCA\NgBackup\Crypto\KeyRing;
|
||||
use OCA\NgBackup\Repository\Repository;
|
||||
use OCA\NgBackup\Tests\Unit\TempDirTrait;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/**
|
||||
* A pack stream that hands out wrong bytes once (an SMB restore stopped after 17 hours with
|
||||
* "Decryption failed"): the blob is read once more on a request of its own; only data that is
|
||||
* damaged on the location stops the restore.
|
||||
*/
|
||||
class DamagedReadTest extends TestCase {
|
||||
use TempDirTrait;
|
||||
|
||||
private string $root;
|
||||
private KeyRing $keys;
|
||||
private string $src;
|
||||
private string $snapshot;
|
||||
|
||||
protected function setUp(): void {
|
||||
$this->root = $this->tempDir();
|
||||
$this->keys = KeyRing::generate();
|
||||
$this->src = $this->tempDir();
|
||||
for ($i = 0; $i < 6; $i++) {
|
||||
file_put_contents("$this->src/f$i", random_bytes(3 * 1048576));
|
||||
}
|
||||
$repo = Repository::initWithKey(new LocalBackend($this->root), $this->keys, $this->keys->wrap('pw'));
|
||||
$this->snapshot = $repo->backupDirectory($this->src)['snapshot'];
|
||||
}
|
||||
|
||||
public function testABlobThatCameInDamagedOnceIsReadAgain(): void {
|
||||
$backend = new GarblingBackend(new LocalBackend($this->root), streams: 1);
|
||||
$target = $this->tempDir();
|
||||
|
||||
$this->assertSame(6, Repository::openWithKey($backend, $this->keys)->restore($this->snapshot, $target));
|
||||
|
||||
$this->assertSame(self::hashTree($this->src), self::hashTree($target));
|
||||
$this->assertSame(0, $backend->streams, 'the damaged read happened');
|
||||
}
|
||||
|
||||
public function testATableDumpThatCameInDamagedOnceIsReadAgain(): void {
|
||||
$repo = Repository::openWithKey(new LocalBackend($this->root), $this->keys);
|
||||
$stats = [];
|
||||
$writer = $repo->blobWriter($stats);
|
||||
$lines = [];
|
||||
for ($i = 0; $i < 20000; $i++) {
|
||||
$lines[] = json_encode(['i' => $i, 'pad' => str_repeat('x', $i % 400)]);
|
||||
}
|
||||
$writer->write(implode("\n", $lines) . "\n");
|
||||
$blobs = $writer->finish();
|
||||
$repo->flushPacks();
|
||||
$backend = new GarblingBackend(new LocalBackend($this->root), ranges: 1);
|
||||
|
||||
$this->assertSame($lines, iterator_to_array(Repository::openWithKey($backend, $this->keys)->readLines($blobs), false));
|
||||
$this->assertSame(0, $backend->ranges, 'the damaged read happened');
|
||||
}
|
||||
|
||||
public function testDataDamagedOnTheLocationStillStopsTheRestore(): void {
|
||||
$backend = new GarblingBackend(new LocalBackend($this->root), streams: 1000, ranges: 1000);
|
||||
|
||||
$this->expectException(\Exception::class);
|
||||
Repository::openWithKey($backend, $this->keys)->restore($this->snapshot, $this->tempDir());
|
||||
}
|
||||
}
|
||||
|
||||
/** Pack reads that flip a byte: the first $streams pack streams and $ranges range reads. */
|
||||
final class GarblingBackend implements IBackend {
|
||||
use \OCA\NgBackup\Backend\GetsOneByOne;
|
||||
|
||||
public function __construct(
|
||||
private IBackend $inner,
|
||||
public int $streams = 0,
|
||||
public int $ranges = 0,
|
||||
) {
|
||||
}
|
||||
|
||||
public function put(string $path, $stream): void {
|
||||
$this->inner->put($path, $stream);
|
||||
}
|
||||
|
||||
public function get(string $path) {
|
||||
$fh = $this->inner->get($path);
|
||||
if (!str_starts_with($path, 'packs/') || $this->streams <= 0) {
|
||||
return $fh;
|
||||
}
|
||||
$this->streams--;
|
||||
$data = stream_get_contents($fh);
|
||||
fclose($fh);
|
||||
for ($k = 100; $k < strlen($data); $k += 65536) {
|
||||
$data[$k] = chr(ord($data[$k]) ^ 0x55);
|
||||
}
|
||||
$out = fopen('php://temp', 'w+b');
|
||||
fwrite($out, $data);
|
||||
rewind($out);
|
||||
return $out;
|
||||
}
|
||||
|
||||
public function getRange(string $path, int $offset, int $length): string {
|
||||
$data = $this->inner->getRange($path, $offset, $length);
|
||||
if (str_starts_with($path, 'packs/') && $this->ranges > 0) {
|
||||
$this->ranges--;
|
||||
$data[0] = chr(ord($data[0]) ^ 0x55);
|
||||
}
|
||||
return $data;
|
||||
}
|
||||
|
||||
public function exists(string $path): bool {
|
||||
return $this->inner->exists($path);
|
||||
}
|
||||
|
||||
public function move(string $from, string $to): void {
|
||||
$this->inner->move($from, $to);
|
||||
}
|
||||
|
||||
public function list(string $prefix): array {
|
||||
return $this->inner->list($prefix);
|
||||
}
|
||||
|
||||
public function listFiles(string $dir): array {
|
||||
return $this->inner->listFiles($dir);
|
||||
}
|
||||
|
||||
public function delete(string $path): void {
|
||||
$this->inner->delete($path);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace OCA\NgBackup\Tests\Unit\Service;
|
||||
|
||||
use OCA\NgBackup\Service\Schedule;
|
||||
use OCP\IAppConfig;
|
||||
use OCP\IConfig;
|
||||
use OCP\IGroup;
|
||||
use OCP\IGroupManager;
|
||||
use OCP\IUser;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
/** The daily time in a time zone (issue #19), and the conversion of a time saved without one. */
|
||||
class ScheduleTest extends TestCase {
|
||||
private array $values = [];
|
||||
|
||||
private function schedule(string $serverZone = '', string $adminZone = ''): Schedule {
|
||||
$app = $this->createMock(IAppConfig::class);
|
||||
$app->method('getValueString')->willReturnCallback(fn (string $a, string $k, string $d = '') => $this->values[$k] ?? $d);
|
||||
$app->method('setValueString')->willReturnCallback(function (string $a, string $k, string $v): bool {
|
||||
$this->values[$k] = $v;
|
||||
return true;
|
||||
});
|
||||
$config = $this->createMock(IConfig::class);
|
||||
$config->method('getSystemValueString')->willReturn($serverZone);
|
||||
$config->method('getUserValue')->willReturn($adminZone);
|
||||
$user = $this->createMock(IUser::class);
|
||||
$user->method('getUID')->willReturn('admin');
|
||||
$group = $this->createMock(IGroup::class);
|
||||
$group->method('getUsers')->willReturn([$user]);
|
||||
$groups = $this->createMock(IGroupManager::class);
|
||||
$groups->method('get')->willReturn($group);
|
||||
return new Schedule($app, $config, $groups);
|
||||
}
|
||||
|
||||
public function testATimeSavedWithoutAZoneShowsAsItRanInTheAdministratorsZone(): void {
|
||||
$this->values = [Schedule::TIME_KEY => '02:30'];
|
||||
|
||||
$s = $this->schedule(adminZone: 'Europe/Berlin')->get();
|
||||
|
||||
// it ran at 02:30 UTC: in summer that is 04:30 in Berlin, in winter 03:30
|
||||
$summer = Schedule::convert('02:30', 'UTC', 'Europe/Berlin', strtotime('2026-07-01 12:00 UTC'));
|
||||
$this->assertSame('04:30', $summer);
|
||||
$this->assertSame('Europe/Berlin', $s['timezone']);
|
||||
$this->assertSame(Schedule::convert('02:30', 'UTC', 'Europe/Berlin'), $s['time']);
|
||||
}
|
||||
|
||||
public function testTheServersDefaultZoneComesFirst(): void {
|
||||
$this->values = [Schedule::TIME_KEY => '01:00'];
|
||||
$this->assertSame('America/Toronto', $this->schedule('America/Toronto', 'Europe/Berlin')->get()['timezone']);
|
||||
}
|
||||
|
||||
public function testWithoutAnyZoneItStaysUtc(): void {
|
||||
$this->values = [Schedule::TIME_KEY => '02:30'];
|
||||
$this->assertSame(['time' => '02:30', 'timezone' => 'UTC'], $this->schedule()->get());
|
||||
}
|
||||
|
||||
public function testTodayAtIsTheMomentInTheSchedulesZone(): void {
|
||||
$this->values = [Schedule::TIME_KEY => '03:00', Schedule::ZONE_KEY => 'Europe/Berlin'];
|
||||
$at = $this->schedule()->todayAt(strtotime('2026-10-22 12:00 UTC'));
|
||||
$this->assertSame(strtotime('2026-10-22 01:00 UTC'), $at, '03:00 in Berlin (summer time) is 01:00 UTC');
|
||||
}
|
||||
|
||||
public function testAnUnknownZoneOrABadTimeIsRefused(): void {
|
||||
$s = $this->schedule();
|
||||
$this->expectException(\InvalidArgumentException::class);
|
||||
$s->set('03:00', 'Mars/Olympus');
|
||||
}
|
||||
}
|
||||
@@ -47,6 +47,12 @@ if [ "$LEGACY" = legacy ]; then
|
||||
fi
|
||||
|
||||
run_on() { docker exec -u www-data "$1" php occ "${@:2}"; }
|
||||
# The app store sends the whole app list (~38 MB) and Nextcloud gives up after 120 s: on a slow day
|
||||
# an install fails for reasons outside this test. Up to three tries.
|
||||
install_app() { # container app
|
||||
for i in 1 2 3; do run_on "$1" app:install "$2" >/dev/null 2>&1 && return 0; sleep 20; done
|
||||
return 1
|
||||
}
|
||||
|
||||
# Database settings for an instance: SQLite inside the container, or its own PostgreSQL container
|
||||
db_env() {
|
||||
@@ -109,7 +115,7 @@ if [ "$DB" != sqlite ]; then
|
||||
# while the restore loads tables in name order, accounts before mailboxes.
|
||||
docker run -d --name ngb-frt-imap --network "$NET" -e MAILNAME=mail.domain.tld -e MAIL_ADDRESS=user@domain.tld \
|
||||
-e MAIL_PASS=mypassword ghcr.io/christophwurst/docker-imap-devel:latest >/dev/null
|
||||
run_on ngb-frt-origin app:install mail >/dev/null || fail "install mail on the origin"
|
||||
install_app ngb-frt-origin mail || fail "install mail on the origin (app store unreachable?)"
|
||||
run_on ngb-frt-origin config:system:set app.mail.verify-tls-peer --type=boolean --value=false >/dev/null
|
||||
sleep 10
|
||||
run_on ngb-frt-origin mail:account:create frtest Test user@domain.tld ngb-frt-imap 993 ssl user@domain.tld mypassword \
|
||||
@@ -118,7 +124,9 @@ if [ "$DB" != sqlite ]; then
|
||||
[ "$(sql origin 'select count(*) from oc_mail_accounts where drafts_mailbox_id is not null')" = 1 ] \
|
||||
|| fail "the mail account does not point at its mailboxes"
|
||||
# Deck installed and then switched off: its tables stay in the database and in the backup
|
||||
run_on ngb-frt-origin app:install deck >/dev/null || fail "install deck on the origin"
|
||||
install_app ngb-frt-origin deck || fail "install deck on the origin (app store unreachable?)"
|
||||
# a 4-byte character (emoji) in a table the restore has to create: utf8mb4 on MySQL/MariaDB
|
||||
sql origin "insert into oc_deck_boards (title, owner, color, archived, deleted_at, last_modified) values ('Plan 📌', 'frtest', '0087C5', false, 0, 0)" >/dev/null
|
||||
run_on ngb-frt-origin app:disable deck >/dev/null
|
||||
ok "origin has a synced Mail account and the tables of a disabled app"
|
||||
fi
|
||||
@@ -202,6 +210,7 @@ if [ "$ORIGIN_DBTYPE" != sqlite ] && [ "$DB" != sqlite ] && [ "$LEGACY" = legacy
|
||||
ok "older backup: tables of the disabled app left out, Mail restored"
|
||||
elif [ "$ORIGIN_DBTYPE" != sqlite ] && [ "$DB" != sqlite ]; then
|
||||
[ "$(sql restore "select count(*) from information_schema.tables where table_name = 'oc_deck_boards'")" = 1 ] || fail "the tables of the disabled app were not created"
|
||||
[ "$(sql restore "select title from oc_deck_boards")" = "Plan 📌" ] || fail "a 4-byte character did not survive in a created table"
|
||||
[ "$(sql restore "select count(*) from oc_mail_accounts where drafts_mailbox_id is not null")" = 1 ] || fail "the mail account was not restored"
|
||||
[ "$(sql restore "select count(*) from information_schema.table_constraints where constraint_type = 'FOREIGN KEY' and table_name like 'oc_mail_%'")" -gt 0 ] \
|
||||
|| fail "the foreign keys of Mail are missing after the restore"
|
||||
|
||||
Reference in New Issue
Block a user