Fast S3 verify and snapshot commit; shorter runs list; durations with days

On S3, index files and the tree segments read back when a snapshot is
written are fetched 16 at a time through the storage's S3 client
(IBackend::getRanges(), getMany()). Blobs that follow each other in a pack
are read as one range on every location. The snapshot is written in a step
of its own and the run shows "commit" meanwhile. On a 1 TB production
repository the check went from about 4 hours to 3.5 minutes.

backup:status and the web interface list the runs of the last 7 days,
running ones and at least the newest 5; --all / "Show older runs" list
all. Durations of a day or more show the days.
This commit is contained in:
2026-10-14 17:12:41 +02:00
parent 9a3bb99d12
commit ca003c88ba
20 changed files with 424 additions and 30 deletions
+11
View File
@@ -35,6 +35,17 @@ interface IBackend {
/** @return string bytes [$offset, $offset + $length) of the object (used to fetch single blobs from a pack) */
public function getRange(string $path, int $offset, int $length): string;
/**
* Many byte ranges, in the order given (blobs spread over many packs, such as the tree
* segments of a backup). A backend where every request costs a round trip can fetch them
* several at a time.
*
* @param list<array{0:string, 1:int, 2:int}> $ranges [path, offset, length] each
* @return \Generator<int, string> contents by position in $ranges
* @throws BackendException for a range that cannot be read
*/
public function getRanges(array $ranges): \Generator;
public function exists(string $path): bool;
/**