Over SMB, read index files in fewer round trips
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
CI / unit (8.2) (push) Has been cancelled
CI / unit (8.3) (push) Has been cancelled
CI / unit (8.4) (push) Has been cancelled
CI / psalm (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / integration (mariadb, 32) (push) Has been cancelled
CI / integration (mariadb, 33) (push) Has been cancelled
CI / integration (mariadb, 34) (push) Has been cancelled
CI / integration (mariadb, 35) (push) Has been cancelled
CI / integration (pgsql, 32) (push) Has been cancelled
CI / integration (pgsql, 33) (push) Has been cancelled
CI / integration (pgsql, 34) (push) Has been cancelled
CI / integration (pgsql, 35) (push) Has been cancelled
CI / integration (sqlite, 32) (push) Has been cancelled
CI / integration (sqlite, 33) (push) Has been cancelled
CI / integration (sqlite, 34) (push) Has been cancelled
CI / integration (sqlite, 35) (push) Has been cancelled
CI / integration-targets (push) Has been cancelled
CI / integration-user (32) (push) Has been cancelled
CI / integration-user (33) (push) Has been cancelled
CI / integration-user (34) (push) Has been cancelled
CI / integration-user (35) (push) Has been cancelled
CI / integration-full-restore (push) Has been cancelled
files_external's fopen() first stats the file and icewind's stream reads 1 MiB at a time. Open, read and close straight through the native extension's state instead: from Canada to a Hetzner Storage Box (130 ms) 0.66 instead of 1.05 s per index file, about 5 instead of 8 hours for the 27,000 index files of a 1 TB repository in backup:target:add. Anything unexpected falls back to the usual way.
This commit is contained in:
@@ -34,6 +34,10 @@ shown under 1.0.0. The git tag of a release candidate goes after the date.
|
||||
- Full restores from S3 read the packs ahead, eight ranges at a time, as restores over SFTP do
|
||||
since 1.0.0-rc.4. Over SMB the packs are read in 8 MiB reads instead of 1 MiB: over a line with
|
||||
a 130 ms round trip, reading was four times faster in a test (18 instead of 4.5 MiB/s).
|
||||
- Over SMB the index files, which a new server reads in `backup:target:add`, take fewer round
|
||||
trips: from Canada to a Hetzner Storage Box (130 ms),
|
||||
0.66 instead of 1.05 s per file, so about 5 instead of 8 hours for the 27,000 index files of a
|
||||
1 TB repository.
|
||||
- `occ backup:target:add` only estimates the time left for reading the index once enough of it
|
||||
is in. The first file also carries the time to connect, which made it say "about 642 min left"
|
||||
for something done within a minute.
|
||||
|
||||
Reference in New Issue
Block a user