SFTP: replace a dropped connection; write the index cache in one transaction

A Hetzner Storage Box drops an SFTP connection that carried a lot of data
and then stayed quiet for a few minutes; files_external keeps using the dead
client, on which every file looks missing. StorageBackend now checks the
connection after a minute without requests, and replaces a connection that
is dead after a failed request (once per request; put, move and delete are
not repeated). SftpReadStream closes a connection it left in an unknown state.

The verify cache refresh took more than 6 minutes on PostgreSQL for 27,547
packs, one autocommitted insert per pack: putMany()/replaceAll() write in one
transaction. After a verify the cache holds the index files as they are, so a
blob in two packs stays cached for both.
This commit is contained in:
2026-10-12 01:58:54 +02:00
parent d659322e60
commit 9c06ab2841
9 changed files with 442 additions and 33 deletions
+14
View File
@@ -15,6 +15,20 @@ interface IIndexCache {
/** @param list<array{id:string, offset:int, length:int, raw:int, flags:int}> $entries */
public function put(string $repositoryId, string $packId, array $entries): void;
/**
* put() for many packs at once (one database transaction).
*
* @param array<string, list<array{id:string, offset:int, length:int, raw:int, flags:int}>> $packs packId => entries
*/
public function putMany(string $repositoryId, array $packs): void;
/**
* Replace everything cached for the repository with $packs (one database transaction).
*
* @param array<string, list<array{id:string, offset:int, length:int, raw:int, flags:int}>> $packs packId => entries
*/
public function replaceAll(string $repositoryId, array $packs): void;
/** @param list<string> $packIds */
public function remove(string $repositoryId, array $packIds): void;
}