Full restore over SFTP: read packs ahead on four connections at once
A restore read one pack at a time over one SFTP connection, which over a long line is bound by the round trip and by what one TCP connection gets: about 2-3 MiB/s from a Hetzner Storage Box to Canada, with PHP mostly waiting. Repository::restore() now first lists the ranges of packs the restore needs, in order (ReadAheadPlan), and the location reads them ahead into unlinked files in a spool folder in the target (IReadsAhead). Over SFTP, SftpRangeFetcher does that on four connections of its own, 16 requests in flight each, taking the answers of whichever connection has one. A range that fails, a damaged copy, or a blob that moved since the plan is read the usual way, which reports what is wrong. Other locations restore as before. On a test line with 120 ms RTT and 0.2% loss, a 1.6 GiB restore went from 5.1 to 17.8 MiB/s, all files identical.
This commit is contained in:
@@ -0,0 +1,40 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
namespace OCA\NgBackup\Backend;
|
||||
|
||||
/**
|
||||
* One range that SftpRangeFetcher reads on one connection.
|
||||
*
|
||||
* @internal
|
||||
*/
|
||||
final class SftpRangeJob {
|
||||
public string $handle = '';
|
||||
/** Offset of the next READ to send. */
|
||||
public int $next;
|
||||
/** @var list<array{int, int}> offset and length of parts to ask for again (after a short answer) */
|
||||
public array $redo = [];
|
||||
/** @var list<array{int, int, int, int}> request id, kind, offset and length, in the order sent */
|
||||
public array $queue = [];
|
||||
/** Bytes received. */
|
||||
public int $got = 0;
|
||||
public bool $failed = false;
|
||||
|
||||
/** @var resource the spool file that holds the range */
|
||||
public $fh;
|
||||
|
||||
/** @param resource $fh */
|
||||
public function __construct(
|
||||
public int $i,
|
||||
$fh,
|
||||
public int $start,
|
||||
public int $end,
|
||||
) {
|
||||
$this->fh = $fh;
|
||||
$this->next = $start;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user