Compare commits

..

3 Commits

Author SHA1 Message Date
dillard 4847ce293d SFTP: restore the line breaks of an OpenSSH private key before use (#24)
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
Nextcloud 35 reads keys with phpseclib 3, which refuses an OpenSSH-format
key whose line breaks were lost ("Unable to read key"); phpseclib 2 on
Nextcloud 34 read it. The key is fixed when used, so stored locations work
again without re-entering it.
2026-10-28 01:41:58 +01:00
dillard 90dd97977a CI: build the admin page on every push 2026-10-27 11:34:53 +01:00
dillard 08534975d7 Admin page: Docs tab with links to the how-to, disaster recovery guide and roadmap; drop "in beta" from the feedback line
PLAN: 1.0.1 item 11 (Docs tab, done); 1.1.0 item 9: remove the feedback request in 1.1.0 or 1.2.0.
2026-10-27 11:23:34 +01:00
7 changed files with 132 additions and 2 deletions
+14
View File
@@ -41,6 +41,20 @@ jobs:
- run: composer install --no-interaction --no-progress
- run: composer psalm
frontend:
# Builds the admin page, so a broken Vue component fails CI instead of showing up after a release.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions/setup-node@v4
with:
node-version: '22'
cache: npm
- run: npm ci --no-audit --no-fund
- run: npm run build
integration:
# Full round trip (key, kit, location, backup, incremental, restore modes, database dump and
# restore, retention, trash) on a fresh Nextcloud per version and database.
+7
View File
@@ -21,6 +21,13 @@ instead carried by the app description, the git tag, the release, and this file.
expected form (#22). Before, `backup:user:restore <location> <uid>` only said "is not a
recorded user export for this location".
### Fixed
- An SFTP location with an RSA key in OpenSSH format ("BEGIN OPENSSH PRIVATE KEY") whose line
breaks were lost, e.g. turned into spaces on the way in, stopped working after an upgrade to
Nextcloud 35 with "Unable to read key": Nextcloud 35 reads keys with a stricter library. The
line breaks are now restored before the key is used, so the stored location works again without
re-entering it (#24).
## [1.0.0-rc.6] - 2026-10-24
Sixth release candidate for 1.0.0; the app version is 0.9.12. Fixes from two disaster recovery
+11
View File
@@ -448,6 +448,12 @@ in README "Known limitations":
zijn, dat alleen `backup:user:backup` eenmalig toonde. Nu: `backup:user:list` toont de
exports, een kale uid werkt bij precies één export, anders een lijst om uit te kiezen —
**klaar** (2026-10-27).
11. **Tabblad "Docs" in de beheerpagina** (besluit 2026-10-27): een derde knop naast "Backup and
restore" en "Settings", met links naar de HOWTO (`docs/HOWTO.md`), de DR-gids
(`docs/DISASTER-RECOVERY.md`) en de roadmap (README, `#roadmap`) op GitHub, elk met een
regel wat erin staat. Nu zijn ze alleen via de README te vinden. Zichtbaar ook zonder
sleutel, zodat de gids vindbaar is vóór de eerste installatie-stap. "NG Backup is in beta"
gaat uit de feedbackregel — **klaar** (2026-10-27).
- **DR-tests over databasegrenzen** (besluit 2026-10-20; productie nc01 draait PostgreSQL): na
de MariaDB→PostgreSQL-fout (tellers, rc5) elke overstap in de integratietest (`to-pgsql`,
`to-mysql`, `to-sqlite`, ook in CI). Volle DR-tests met echte data: vóór 1.0.0 de S3-simulatie
@@ -488,6 +494,11 @@ in README "Known limitations":
nooit opruimt (bijv. "vóór de upgrade naar NC 35"), in de lijst en met
`occ backup:pin <snapshot>` / `--unpin`. Op de locatie bewaren (catalogus), zodat de
opschoning het ziet, ook na een DR; vastgepinde punten tellen niet mee in last/daily/weekly.
9. **Feedbackvraag weghalen** in 1.1.0 of 1.2.0 (besluit 2026-10-27): de regel "We would like
to hear how NG Backup works for you ... Share your experience" in de beheerpagina blijft tot
dan staan (de vermelding "in beta" is er in 1.0.1 al af). Zijn er tegen die
tijd geen noemenswaardige meldingen, dan werkt het: mensen melden eerder wat niet werkt dan
wat gewoon werkt. Het formulier (`experience.yml`) mag blijven voor wie het zelf opzoekt.
- **Bevindingen DR-test op PVE02 (2026-10-17, snapshot van nc01, 1 TB, AIO → gewone NC 34 met
PostgreSQL 18)** — voor **1.0.0-rc.2**:
1. Tabellen van apps die bij de backup **niet aanstonden** (restanten: Deck, ONLYOFFICE,
+2
View File
@@ -268,6 +268,8 @@ a 👍 on its issue helps decide what comes first.
- An explanation in the trash of restore points of what "removed by retention policy" means and
until when a restore point can be brought back.
- Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.
- A "Docs" tab on the admin page with links to the how-to, the disaster recovery guide and this
roadmap.
**1.1.0**
- Restore without the recovery kit: the backup key from the location itself, opened with one of
+22
View File
@@ -72,6 +72,10 @@ final class ExternalStorageFactory {
throw new BackendException("Auth mechanism $authId does not fit backend $backendId");
}
if (is_string($options['private_key'] ?? null)) {
$options['private_key'] = self::normalizePrivateKey($options['private_key']);
}
$config = new StorageConfig();
$config->setBackend($backend);
$config->setAuthMechanism($auth);
@@ -86,4 +90,22 @@ final class ExternalStorageFactory {
$storage = $backend->wrapStorage($storage);
return $auth->wrapStorage($storage);
}
/**
* An OpenSSH-format private key with its line breaks restored. Up to Nextcloud 34
* (phpseclib 2) a key whose newlines were lost or turned into spaces, e.g. on its way through
* a shell or a form, still loaded; Nextcloud 35 (phpseclib 3) refuses it with "Unable to read
* key" (#24). The key itself is not changed, so it is fixed when it is used, not when stored.
* Other formats (PEM, PKCS#8) still load as they are and are returned unchanged.
*/
public static function normalizePrivateKey(string $key): string {
if (!preg_match('/-----BEGIN OPENSSH PRIVATE KEY-----(.*?)-----END OPENSSH PRIVATE KEY-----/s', $key, $m)) {
return $key;
}
$body = (string)preg_replace('/\s+/', '', $m[1]);
if (!preg_match('#^[A-Za-z0-9+/]+=*$#', $body)) {
return $key;
}
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
}
}
+41 -2
View File
@@ -7,7 +7,7 @@
<NcSettingsSection :name="t('ng_backup', 'NG Backup')"
:description="t('ng_backup', 'Encrypted, incremental backups of this Nextcloud (database, configuration and data). Backup locations are never mounted and are not visible in Files.')">
<p class="ngb-feedback">
{{ t('ng_backup', 'NG Backup is in beta. We would like to hear how it works for you, also when everything went fine.') }}
{{ t('ng_backup', 'We would like to hear how NG Backup works for you, also when everything went fine.') }}
<a :href="feedbackUrl" target="_blank" rel="noopener noreferrer">{{ t('ng_backup', 'Share your experience ↗') }}</a>
</p>
<NcLoadingIcon v-if="!status" :size="32" />
@@ -18,9 +18,18 @@
<div class="ngb-tabs">
<NcButton :variant="tab === 'backup' ? 'primary' : 'secondary'" @click="tab = 'backup'">{{ t('ng_backup', 'Backup and restore') }}</NcButton>
<NcButton :variant="tab === 'settings' ? 'primary' : 'secondary'" @click="tab = 'settings'">{{ t('ng_backup', 'Settings') }}</NcButton>
<NcButton :variant="tab === 'docs' ? 'primary' : 'secondary'" @click="tab = 'docs'">{{ t('ng_backup', 'Docs') }}</NcButton>
</div>
<template v-if="tab === 'settings'">
<NcSettingsSection v-if="tab === 'docs'" :name="t('ng_backup', 'Documentation')">
<ul class="ngb-docs">
<li v-for="doc in docs" :key="doc.url">
<a :href="doc.url" target="_blank" rel="noopener noreferrer">{{ doc.name }} ↗</a>
<span>{{ doc.description }}</span>
</li>
</ul>
</NcSettingsSection>
<template v-else-if="tab === 'settings'">
<KeySetup :status="status" @changed="refresh" />
<Locations v-if="status.key.everConfirmed" :status="status" @changed="refresh" />
</template>
@@ -52,6 +61,25 @@ import RestoreBrowser from './components/RestoreBrowser.vue'
// Opens the experience report form in .github/ISSUE_TEMPLATE
const feedbackUrl = 'https://github.com/dillardblom/ng_backup/issues/new?template=experience.yml'
const repoUrl = 'https://github.com/dillardblom/ng_backup'
const docs = [
{
name: t('ng_backup', 'How-to'),
description: t('ng_backup', 'Step by step: from first install to a tested restore.'),
url: repoUrl + '/blob/main/docs/HOWTO.md',
},
{
name: t('ng_backup', 'Disaster recovery'),
description: t('ng_backup', 'Your server is gone: get Nextcloud back on a new server from a backup location.'),
url: repoUrl + '/blob/main/docs/DISASTER-RECOVERY.md',
},
{
name: t('ng_backup', 'Roadmap'),
description: t('ng_backup', 'What is planned for the next versions.'),
url: repoUrl + '#roadmap',
},
]
const status = ref(null)
const allRuns = ref(false) // runs older than the last days too
const error = ref('')
@@ -102,6 +130,17 @@ onBeforeUnmount(() => clearTimeout(timer))
color: var(--color-primary-element);
text-decoration: underline;
}
.ngb-docs li {
margin-bottom: 8px;
}
.ngb-docs a {
color: var(--color-primary-element);
text-decoration: underline;
margin-right: 8px;
}
.ngb-docs span {
color: var(--color-text-maxcontrast);
}
.ngb-tabs {
display: flex;
gap: 8px;
@@ -0,0 +1,35 @@
<?php
declare(strict_types=1);
namespace OCA\NgBackup\Tests\Unit\Backend;
use OCA\NgBackup\Backend\ExternalStorageFactory;
use PHPUnit\Framework\TestCase;
/** ExternalStorageFactory::normalizePrivateKey(): keys that phpseclib 2 read, for phpseclib 3 (#24). */
class ExternalStorageFactoryTest extends TestCase {
/** An OpenSSH-format key laid out as ssh-keygen writes it; the body is random, not a real key. */
private static function key(): string {
$body = base64_encode(random_bytes(1000));
return "-----BEGIN OPENSSH PRIVATE KEY-----\n" . chunk_split($body, 70, "\n") . "-----END OPENSSH PRIVATE KEY-----\n";
}
public function testRestoresLostLineBreaks(): void {
$key = self::key();
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey($key));
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", ' ', $key)), 'newlines as spaces');
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", '', $key)), 'no newlines');
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(str_replace("\n", "\r\n", $key)), 'CRLF');
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(rtrim($key)), 'no trailing newline');
$this->assertSame($key, ExternalStorageFactory::normalizePrivateKey(preg_replace('/^/m', ' ', $key)), 'indented');
}
public function testLeavesOtherFormatsAlone(): void {
$pem = "-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\nDEK-Info: DES-EDE3-CBC,FF44E0A94036766C\n\nAAAA\n-----END RSA PRIVATE KEY-----\n";
$this->assertSame($pem, ExternalStorageFactory::normalizePrivateKey($pem));
$this->assertSame('not a key', ExternalStorageFactory::normalizePrivateKey('not a key'));
$odd = "-----BEGIN OPENSSH PRIVATE KEY-----\\nAAAA\\n-----END OPENSSH PRIVATE KEY-----";
$this->assertSame($odd, ExternalStorageFactory::normalizePrivateKey($odd), 'not base64 inside: unchanged');
}
}