Fast S3 verify and snapshot commit; shorter runs list; durations with days
On S3, index files and the tree segments read back when a snapshot is written are fetched 16 at a time through the storage's S3 client (IBackend::getRanges(), getMany()). Blobs that follow each other in a pack are read as one range on every location. The snapshot is written in a step of its own and the run shows "commit" meanwhile. On a 1 TB production repository the check went from about 4 hours to 3.5 minutes. backup:status and the web interface list the runs of the last 7 days, running ones and at least the newest 5; --all / "Show older runs" list all. Durations of a day or more show the days.
This commit is contained in:
@@ -225,6 +225,9 @@ final class BackupService {
|
||||
} else {
|
||||
$state['files']['estimate'] ??= $this->estimateBytes($run, $state['files']);
|
||||
$state['files'] = BackupRun::step($repo, $state['files'], $deadline);
|
||||
if (in_array($state['files']['phase'], ['finish', 'commit'], true)) {
|
||||
$run->setPhase('commit'); // all files read: writing and recording the snapshot
|
||||
}
|
||||
if ($state['files']['phase'] === 'commit') {
|
||||
// Catalog write under the catalog lock, from a repository opened inside it: a per-user
|
||||
// export or restore may have written the catalog since this step opened $repo.
|
||||
|
||||
Reference in New Issue
Block a user