Repository: list flat folders without a type check per entry
Opening a repository lists index/, which holds one file per pack. StorageBackend::list() checked every entry with is_dir(), and on a remote storage each check is a round trip. Over SFTP to a Hetzner Storage Box (33 ms) with a 60 GiB repository, occ backup:run (30 s steps) spent nearly every step on these requests and uploaded 45 KiB/s; with this change it uploads 5.8 MB/s. The cost grows with the number of packs. index/, snapshots/, catalog/ and trash/info/ only ever hold objects: list them with the new IBackend::listFiles(), one listing without per-entry checks.
This commit is contained in:
@@ -92,8 +92,8 @@ final class Repository {
|
||||
private function loadCatalog(?ICatalogAnchor $anchor): void {
|
||||
$this->catalog = new Catalog($this->backend, $this->cipher, $this->repositoryId, $anchor);
|
||||
$this->catalog->load(
|
||||
fn () => array_map('basename', $this->backend->list('snapshots')),
|
||||
fn () => array_map('basename', $this->backend->list('trash/info')),
|
||||
fn () => array_map('basename', $this->backend->listFiles('snapshots')),
|
||||
fn () => array_map('basename', $this->backend->listFiles('trash/info')),
|
||||
fn () => $this->backend->list('users'));
|
||||
}
|
||||
|
||||
@@ -643,7 +643,7 @@ final class Repository {
|
||||
* @return list<string>
|
||||
*/
|
||||
public function listSnapshots(): array {
|
||||
$present = array_flip(array_map('basename', $this->backend->list('snapshots')));
|
||||
$present = array_flip(array_map('basename', $this->backend->listFiles('snapshots')));
|
||||
$listed = array_keys($this->catalog->snapshots);
|
||||
foreach ($listed as $id) {
|
||||
if (!isset($present[$id])) {
|
||||
|
||||
Reference in New Issue
Block a user