Repository: list flat folders without a type check per entry
Opening a repository lists index/, which holds one file per pack. StorageBackend::list() checked every entry with is_dir(), and on a remote storage each check is a round trip. Over SFTP to a Hetzner Storage Box (33 ms) with a 60 GiB repository, occ backup:run (30 s steps) spent nearly every step on these requests and uploaded 45 KiB/s; with this change it uploads 5.8 MB/s. The cost grows with the number of packs. index/, snapshots/, catalog/ and trash/info/ only ever hold objects: list them with the new IBackend::listFiles(), one listing without per-entry checks.
This commit is contained in:
@@ -92,7 +92,7 @@ final class BlobIndex {
|
||||
$cached = $verify ? [] : ($this->cache?->all($this->repositoryId) ?? []);
|
||||
$present = [];
|
||||
$this->downloaded = 0;
|
||||
foreach ($this->backend->list('index') as $path) {
|
||||
foreach ($this->backend->listFiles('index') as $path) {
|
||||
$packId = basename($path);
|
||||
$present[$packId] = true;
|
||||
if (isset($cached[$packId])) {
|
||||
|
||||
Reference in New Issue
Block a user