Full restore over SFTP: read packs ahead on four connections at once

A restore read one pack at a time over one SFTP connection, which over a long
line is bound by the round trip and by what one TCP connection gets: about
2-3 MiB/s from a Hetzner Storage Box to Canada, with PHP mostly waiting.

Repository::restore() now first lists the ranges of packs the restore needs,
in order (ReadAheadPlan), and the location reads them ahead into unlinked
files in a spool folder in the target (IReadsAhead). Over SFTP,
SftpRangeFetcher does that on four connections of its own, 16 requests in
flight each, taking the answers of whichever connection has one. A range that
fails, a damaged copy, or a blob that moved since the plan is read the usual
way, which reports what is wrong. Other locations restore as before.

On a test line with 120 ms RTT and 0.2% loss, a 1.6 GiB restore went from
5.1 to 17.8 MiB/s, all files identical.

lib/Backend/StorageBackend.php

@@ -14,7 +14,7 @@ use OCP\Files\Storage\IWriteStreamStorage;
  * IBackend on top of a Nextcloud storage object (from files_external or core), below a base
  * directory on that storage.
  */
-final class StorageBackend implements IBackend {
+final class StorageBackend implements IBackend, IReadsAhead {
 	/** Object stores write whole objects atomically; other storages get a temp name + rename. */
 	private bool $atomicPut;
 	/** SFTP: read through SftpReadStream, which keeps several requests in flight. */
@@ -243,6 +243,51 @@ final class StorageBackend implements IBackend {
 		}
 	}
 
+	/**
+	 * Over SFTP: SftpRangeFetcher, on CONNECTIONS new connections of its own (built with the
+	 * reconnect function), so the storage's connection stays free for other requests. Elsewhere
+	 * not (yet): null.
+	 */
+	public function canReadAhead(): bool {
+		return $this->sftpReads && $this->reconnect !== null;
+	}
+
+	public function readAhead(array $ranges, string $spool): ?\Generator {
+		$reconnect = $this->reconnect;
+		if (!$this->sftpReads || $reconnect === null || $ranges === []) {
+			return null;
+		}
+		$storages = [];
+		$connections = [];
+		$root = '';
+		for ($n = 0; $n < SftpRangeFetcher::CONNECTIONS; $n++) {
+			try {
+				$storage = $reconnect();
+				$sftp = SftpReadStream::connection($storage);
+				if ($sftp === null || !is_callable([$storage, 'getRoot'])) {
+					break;
+				}
+				$root = (string)$storage->getRoot();
+			} catch (\Throwable) {
+				break; // the server may allow fewer connections: go on with those that work
+			}
+			$storages[] = $storage;
+			$connections[] = $sftp;
+		}
+		if ($connections === []) {
+			return null;
+		}
+		$abs = array_map(fn (array $r): array => [$this->abs($r[0]), $r[1], $r[2]], $ranges);
+		return (static function () use ($storages, $connections, $root, $abs, $spool): \Generator {
+			// the storages hold their connections: kept until the fetcher is done
+			try {
+				yield from SftpRangeFetcher::fetch($connections, $root, $abs, $spool);
+			} finally {
+				unset($storages);
+			}
+		})();
+	}
+
 	public function getRange(string $path, int $offset, int $length): string {
 		return $this->call(fn () => $this->readRange($path, $offset, $length));
 	}