Pre-beta review: data safety in storage, catalog and backup runs

Storage and repository:
- Listing skipped every name containing ".part-", directories too, so the
  exports of a user like "john.part-time" were invisible to prune on S3,
  SFTP, FTP and SMB and their data was deleted. Only real temporary uploads
  ("<name>.part-<8 hex>") are skipped now.
- prune refuses to run while a recorded per-user export's manifest is
  missing, instead of freeing that export's data.
- forget() and untrash() copy first, switch the catalog, and delete the
  originals last, so a crash can no longer leave the catalog pointing at a
  missing snapshot or database dump.
- Writes are checked against the expected size, so a short upload is an
  error instead of a truncated pack; raw restores check rename() and touch().
- A catalog write refuses to overwrite a generation another process just
  wrote, and the catalog lock lasts 300 s, so opening a large repository
  inside it no longer lets the lease run out.

Backup runs and scheduling:
- A file that keeps changing is read again at most three times; then the
  last read is kept and reported, instead of restarting forever and
  blocking cleanup.
- Unreadable files and folders are recorded, logged, shown by occ
  backup:run, stored in the snapshot and notified to admins, instead of
  being skipped silently.
- After a failed scheduled backup, at most three attempts a day, an hour
  apart, instead of a new run and notification every five minutes.
- The job no longer spins on a run whose lease another process holds, and
  the weekly cleanup catches up after a Sunday spent backing up.
- Notifications for the per-user export retention events.

lib/Backend/LocalBackend.php

@@ -33,11 +33,16 @@ final class LocalBackend implements IBackend {
 			if ($out === false) {
 				throw new BackendException('Cannot write ' . $path);
 			}
+			$expected = StorageBackend::remaining($stream);
 			if (stream_copy_to_stream($stream, $out) === false || !fflush($out)) {
 				throw new BackendException('Write failed for ' . $path);
 			}
 			fclose($out);
 			$out = false;
+			clearstatcache(true, $tmp);
+			if ($expected !== null && filesize($tmp) !== $expected) {
+				throw new BackendException('Incomplete write for ' . $path);
+			}
 			if (!rename($tmp, $target)) {
 				throw new BackendException('Cannot finalise ' . $path);
 			}
@@ -94,7 +99,7 @@ final class LocalBackend implements IBackend {
 		$result = [];
 		$it = new \RecursiveIteratorIterator(new \RecursiveDirectoryIterator($base, \FilesystemIterator::SKIP_DOTS));
 		foreach ($it as $file) {
-			if ($file->isFile() && !str_contains($file->getFilename(), '.part-')) {
+			if ($file->isFile() && !StorageBackend::isTemporary($file->getFilename())) {
 				$result[] = ltrim(substr($file->getPathname(), strlen($this->root)), '/');
 			}
 		}