Technology · PRACTICAL GUIDE
curl resume corrupted the file? Bind every range to a strong ETag
A byte offset is not a representation identity. Use Range plus If-Range, append only an exact 206 response, restart cleanly on 200, and verify before atomic promotion.
A resumable download is safe only when every appended byte belongs to the same HTTP representation. The local length used by curl -C - is an offset, not a content identity.
Bind the partial file to the final URL and a strong ETag, request the remainder with Range plus If-Range, and let 206 versus 200 choose append versus fresh replacement. Then verify the finished object before an atomic rename.
1. Treat a local length as an offset, not an identity
curl -C - inspects the destination length and asks the server for the remaining bytes. That proves where the local file stops; it does not prove that the server still exposes the same representation. A retry loop can therefore finish cleanly after appending bytes from a replacement object.
Use a dedicated .part path. Record the final URL after redirects, the exact local length, a strong ETag, the expected total length when available and any publisher-supplied digest in a sidecar tied to that one output. Never share one ETag file across multiple URLs.
2. Require a strong validator before resuming
Send Range: bytes=N- and If-Range with the stored strong ETag. RFC 9110 forbids weak entity tags in If-Range. If there is no strong validator, fail closed: restart from byte zero in a fresh file or obtain an authoritative digest and a stronger distribution contract.
curl --etag-compare sends If-None-Match, which is useful for cache-style change checks but is not the same protocol condition as If-Range. A wrapper can send an explicit If-Range header and capture response headers while keeping validator state isolated per URL.
3. Decide the write mode from the response
Append only when the response is 206 Partial Content, Content-Range starts exactly at N, the total length is plausible and the validator still represents the stored object. Write the range response to a temporary tail first; validate its headers before joining it to the partial.
When If-Range is false, the server ignores Range and returns the full representation with 200 OK. That body must replace a fresh temporary file from byte zero, never append to the existing part. Treat 416 as a condition to compare the local length, reported total and validator; it is not automatic proof that the local file is complete.
4. Retry transport failures, not every failure
Retry only classified transient transport failures with a cap and backoff. Authentication failures, redirect changes, missing validators, unexpected status codes, malformed Content-Range and digest mismatches require a stop or a clean restart, not another blind append.
In a bounded curl 8.5.0 fixture, an unvalidated range after the server changed versions returned 206 and produced a mixed file. Sending the old strong ETag in If-Range produced 200 and a safe full replacement. With the unchanged ETag, the server produced 206 from the requested offset and the joined bytes matched the original digest.
5. Verify before atomic promotion
After the body is complete, compare the publisher digest when available. Otherwise verify at least the expected total length plus the same strong validator, while acknowledging that size alone is not content integrity. Validate the archive or application-specific format if it has its own checker.
Only after acceptance should the .part file be fsynced as required and atomically renamed over the destination on the same filesystem. Keep the sidecar and logs until the promoted file is accepted by its consumer. Delete or quarantine mixed and unverifiable parts instead of teaching the next retry to trust them.
