NS-075
documentedbody-truncated-to-a-lying-length
A Content-Length shorter than the body truncates the response with no error anywhere
- reads as
- `curl -s -o data.json -w '%{http_code}' URL` prints 200, curl exits zero, and the bytes written match the Content-Length the server declared. Conclusion drawn: the document was retrieved intact.
- actually
- RFC 9112 makes the declared length authoritative: if a valid Content-Length header field is present without Transfer-Encoding, its decimal value defines the expected message body length in octets. The client reads that many and stops, whatever else is on the connection. The commonest cause is a handler that computes the length in characters and writes the body in UTF-8, so the response is cut short by exactly the number of extra bytes the non-ASCII characters cost. RFC 9112 anticipates the remainder: a user agent MAY discard the remaining data or attempt to determine if that data belongs as part of the prior message body, which might be the case if the prior message's Content-Length value is incorrect.
- blind because
- Every length-based check agrees with itself. The header says 67, the file on disk is 67 bytes, and `%{size_download}` is 67. Unlike a connection that dies mid-body, nothing here is incomplete from the transport's point of view, so no exit code, no warning and no retry is produced.
- the check
- Validate the body on its own terms rather than on the sender's: `curl -s URL | python3 -c 'import sys, json; json.load(sys.stdin)'`. Observed against a local handler serving a 73-byte UTF-8 JSON document under `Content-Length: 67`, the length of the same text in characters: curl reported code=200 size_download=67 and exited 0; the saved file ended `"ok":` and json.load raised `JSONDecodeError: Expecting value: line 1 column 62`. The identical handler taking its length from the encoded bytes returned 73 and parsed. A separate server declaring 16 against a 131-byte body gave curl, Python's http.client and Node all the same silent 16-byte prefix with status 200.
- cost of missing
- A record set is short by a few entries, a token is cut mid-string, a page loses its closing markup. Each looks like a data problem upstream, and re-requesting reproduces it exactly, which reads as confirmation rather than as a framing bug.
- mitigation
- Check completeness at the semantic layer for anything whose length matters: parse it, verify its terminator, or compare a digest the origin computed over the same bytes.
- generalises to
- Every length or checksum supplied by the same party that produced the payload, and every count taken in one unit and spent in another.
- source
- rfc-editor.org