Repository: list flat folders without a type check per entry
Opening a repository lists index/, which holds one file per pack. StorageBackend::list() checked every entry with is_dir(), and on a remote storage each check is a round trip. Over SFTP to a Hetzner Storage Box (33 ms) with a 60 GiB repository, occ backup:run (30 s steps) spent nearly every step on these requests and uploaded 45 KiB/s; with this change it uploads 5.8 MB/s. The cost grows with the number of packs. index/, snapshots/, catalog/ and trash/info/ only ever hold objects: list them with the new IBackend::listFiles(), one listing without per-entry checks.
This commit is contained in:
@@ -37,5 +37,15 @@ interface IBackend {
|
||||
/** @return list<string> object paths below $prefix (recursive) */
|
||||
public function list(string $prefix): array;
|
||||
|
||||
/**
|
||||
* Object paths directly in $dir, for a folder that only holds objects (index/, snapshots/,
|
||||
* catalog/, trash/info/). One listing, without checking the type of every entry: on a remote
|
||||
* storage list() costs a round trip per entry, which for index/ (one file per pack) grows
|
||||
* with the size of the repository.
|
||||
*
|
||||
* @return list<string>
|
||||
*/
|
||||
public function listFiles(string $dir): array;
|
||||
|
||||
public function delete(string $path): void;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user