dillard
ca003c88ba
On S3, index files and the tree segments read back when a snapshot is written are fetched 16 at a time through the storage's S3 client (IBackend::getRanges(), getMany()). Blobs that follow each other in a pack are read as one range on every location. The snapshot is written in a step of its own and the run shows "commit" meanwhile. On a 1 TB production repository the check went from about 4 hours to 3.5 minutes. backup:status and the web interface list the runs of the last 7 days, running ones and at least the newest 5; --all / "Show older runs" list all. Durations of a day or more show the days.
73 lines
2.7 KiB
PHP
73 lines
2.7 KiB
PHP
<?php
|
|
|
|
declare(strict_types=1);
|
|
|
|
// SPDX-FileCopyrightText: 2026 Dillard Blom
|
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
|
|
|
namespace OCA\NgBackup\Backend;
|
|
|
|
/**
|
|
* Minimal object store used by the repository. Paths are relative, '/'-separated.
|
|
* Implementations: a local directory (spike, tests), and adapters over files_external storages.
|
|
*/
|
|
interface IBackend {
|
|
/**
|
|
* Store an object; the stream is read to EOF. Takes ownership of the stream: it is always
|
|
* closed afterwards (some storages, like files_external's S3, close it themselves).
|
|
* Must not leave a partial object behind on failure.
|
|
*/
|
|
public function put(string $path, $stream): void;
|
|
|
|
/** @return resource readable stream of the whole object */
|
|
public function get(string $path);
|
|
|
|
/**
|
|
* Many small objects whole, in the order given (the repository's index files). A backend
|
|
* where every request costs a round trip can fetch them several at a time.
|
|
*
|
|
* @param list<string> $paths
|
|
* @return \Generator<string, string> contents by path
|
|
* @throws BackendException for an object that cannot be read
|
|
*/
|
|
public function getMany(array $paths): \Generator;
|
|
|
|
/** @return string bytes [$offset, $offset + $length) of the object (used to fetch single blobs from a pack) */
|
|
public function getRange(string $path, int $offset, int $length): string;
|
|
|
|
/**
|
|
* Many byte ranges, in the order given (blobs spread over many packs, such as the tree
|
|
* segments of a backup). A backend where every request costs a round trip can fetch them
|
|
* several at a time.
|
|
*
|
|
* @param list<array{0:string, 1:int, 2:int}> $ranges [path, offset, length] each
|
|
* @return \Generator<int, string> contents by position in $ranges
|
|
* @throws BackendException for a range that cannot be read
|
|
*/
|
|
public function getRanges(array $ranges): \Generator;
|
|
|
|
public function exists(string $path): bool;
|
|
|
|
/**
|
|
* Move an object within the same location (used for the trash). Backends must use a native
|
|
* rename, not a read+write copy: some storages (e.g. SFTP) cannot reliably read and write on
|
|
* the same connection at once, since the read side is a lazily-pulled stream.
|
|
*/
|
|
public function move(string $from, string $to): void;
|
|
|
|
/** @return list<string> object paths below $prefix (recursive) */
|
|
public function list(string $prefix): array;
|
|
|
|
/**
|
|
* Object paths directly in $dir, for a folder that only holds objects (index/, snapshots/,
|
|
* catalog/, trash/info/). One listing, without checking the type of every entry: on a remote
|
|
* storage list() costs a round trip per entry, which for index/ (one file per pack) grows
|
|
* with the size of the repository.
|
|
*
|
|
* @return list<string>
|
|
*/
|
|
public function listFiles(string $dir): array;
|
|
|
|
public function delete(string $path): void;
|
|
}
|