Full restore over SFTP: read packs ahead on four connections at once

A restore read one pack at a time over one SFTP connection, which over a long
line is bound by the round trip and by what one TCP connection gets: about
2-3 MiB/s from a Hetzner Storage Box to Canada, with PHP mostly waiting.

Repository::restore() now first lists the ranges of packs the restore needs,
in order (ReadAheadPlan), and the location reads them ahead into unlinked
files in a spool folder in the target (IReadsAhead). Over SFTP,
SftpRangeFetcher does that on four connections of its own, 16 requests in
flight each, taking the answers of whichever connection has one. A range that
fails, a damaged copy, or a blob that moved since the plan is read the usual
way, which reports what is wrong. Other locations restore as before.

On a test line with 120 ms RTT and 0.2% loss, a 1.6 GiB restore went from
5.1 to 17.8 MiB/s, all files identical.
This commit is contained in:
2026-10-19 01:40:49 +02:00
parent b789d17636
commit ccbbf01c54
12 changed files with 1228 additions and 5 deletions
+11 -1
View File
@@ -8,9 +8,10 @@ declare(strict_types=1);
namespace OCA\NgBackup\Repository;
use OCA\NgBackup\Backend\IBackend;
use OCA\NgBackup\Backend\IReadsAhead;
/** The location as Repository sees it: closes the PackReader's open stream before every other request. */
final class PackReaderGuard implements IBackend {
final class PackReaderGuard implements IBackend, IReadsAhead {
public function __construct(
private IBackend $inner,
private PackReader $reader,
@@ -43,6 +44,15 @@ final class PackReaderGuard implements IBackend {
return $this->inner->getRanges($ranges);
}
public function canReadAhead(): bool {
return $this->inner instanceof IReadsAhead && $this->inner->canReadAhead();
}
/** Over connections of the location's own: the open stream may stay open. */
public function readAhead(array $ranges, string $spool): ?\Generator {
return $this->inner instanceof IReadsAhead ? $this->inner->readAhead($ranges, $spool) : null;
}
public function exists(string $path): bool {
$this->reader->close();
return $this->inner->exists($path);