How we fixed a snapshot download timeout without touching customer data

A customer reported that a Miniflux snapshot would not download. Their smaller Uptime Kuma snapshot worked, but the Miniflux download sat long enough to time out. Before changing anything, we treated this as a data-safety question first and a delivery problem second.

What we checked

The affected archive was a completed snapshot stored in our off-site snapshot storage. We streamed that exact object through the production download path into a discard buffer and verified that the complete archive arrived. The archive was intact, its recorded size matched, and no customer data had been lost.

That ruled out the most worrying explanations: a partial snapshot, an expired object, or a storage failure. The problem was not the backup itself. It was how long the download path waited before it started responding.

The failure mode

Before this change, an app host looked up the snapshot in remote storage before it sent download headers to the panel and browser. That lookup is normally quick, but a cold or busy storage target can take long enough for a browser to give up. During that period the browser had received no response at all. In this case, the HTTP/2 stream was closed before the archive began flowing.

The smaller Uptime Kuma snapshot completed without exposing the timing problem. The Miniflux snapshot was not unusually large, but it was large enough to make the first-byte delay visible.

The fix

Caliber Node already records a snapshot's exact size when creation finishes. We now pass that recorded size into the download request, so the app host can send the response headers immediately rather than waiting for a remote directory listing. It then streams the snapshot archive directly from storage to the customer.

We also kept the older remote-listing behavior as a fallback for older panel and agent combinations. That makes the change safe to roll out gradually instead of making downloads depend on every host updating at exactly the same moment.

What changed in production

• The panel now provides the known snapshot size to the app host.
• The app host flushes download headers before remote storage begins serving the archive.
• The snapshot agent was refreshed on our Amsterdam, Detroit, and Los Angeles app hosts.
• We verified the panel, public portal, and all three agents were healthy after the deployment.

This change does not alter snapshots, containers, app volumes, or restore behavior. It only removes an unnecessary delay from the start of a download.

What we learned

A successful storage check is not the same thing as a good download experience. For streamed files, time to first byte matters just as much as the final byte count. We will continue treating snapshot delivery as part of the product, not a background implementation detail.