SFTP: replace a dropped connection; write the index cache in one transaction
A Hetzner Storage Box drops an SFTP connection that carried a lot of data and then stayed quiet for a few minutes; files_external keeps using the dead client, on which every file looks missing. StorageBackend now checks the connection after a minute without requests, and replaces a connection that is dead after a failed request (once per request; put, move and delete are not repeated). SftpReadStream closes a connection it left in an unknown state. The verify cache refresh took more than 6 minutes on PostgreSQL for 27,547 packs, one autocommitted insert per pack: putMany()/replaceAll() write in one transaction. After a verify the cache holds the index files as they are, so a blob in two packs stays cached for both.
This commit is contained in:
@@ -15,6 +15,20 @@ interface IIndexCache {
|
||||
/** @param list<array{id:string, offset:int, length:int, raw:int, flags:int}> $entries */
|
||||
public function put(string $repositoryId, string $packId, array $entries): void;
|
||||
|
||||
/**
|
||||
* put() for many packs at once (one database transaction).
|
||||
*
|
||||
* @param array<string, list<array{id:string, offset:int, length:int, raw:int, flags:int}>> $packs packId => entries
|
||||
*/
|
||||
public function putMany(string $repositoryId, array $packs): void;
|
||||
|
||||
/**
|
||||
* Replace everything cached for the repository with $packs (one database transaction).
|
||||
*
|
||||
* @param array<string, list<array{id:string, offset:int, length:int, raw:int, flags:int}>> $packs packId => entries
|
||||
*/
|
||||
public function replaceAll(string $repositoryId, array $packs): void;
|
||||
|
||||
/** @param list<string> $packIds */
|
||||
public function remove(string $repositoryId, array $packIds): void;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user