A response saved without decompression is stored as its compressed bytes

A response saved without decompression is stored as its compressed bytes

reads as

`curl -H 'Accept-Encoding: gzip' -o data.json URL` reports 200, exits zero, and data.json is the expected size. Conclusion drawn: the document was fetched.

actually

curl decompresses only when it negotiated the encoding itself. --compressed 'Request[s] a compressed response using one of the algorithms curl supports, and automatically decompress[es] the content'; a hand-written Accept-Encoding header asks for gzip without arranging for it to be undone. The file on disk begins 1f 8b and is a gzip member, not JSON.

blind because

Status, exit code and transferred byte count are identical to a successful plain fetch. %{size_download} counts wire bytes, so it agrees under both hypotheses: 67 bytes either way.

the check

Ask what the file is rather than how big it is: `file -b data.json`. Observed on curl 8.5.0 against a local gzip-encoding server: with a hand-set header the file was 'gzip compressed data' and `grep -c alpha data.json` found no match and exited 1; with --compressed the same URL produced 'JSON text data' and the same grep printed 1.

cost of missing

A search over the artifact returns nothing and the absence is read as a fact about the content. Tools that treat the body as opaque, such as archivers, uploaders and checksums, propagate the encoded bytes without ever failing.

mitigation

curl's manual warns that saved response headers are not modified, so a stored header still claims the content is compressed after curl has decompressed it. The header is not a reliable record either way.

generalises to

Every transport-level transformation the receiver must undo: content-encoding, transfer-encoding, base64 envelopes, client-side decryption.

source

curl.se