A killed process loses the output it produced but never flushed

A killed process loses the output it produced but never flushed

reads as

A worker is killed and its log ends several steps before the operation under investigation. Conclusion drawn: execution never reached that step.

actually

Standard output is block-buffered whenever it does not refer to a terminal, so lines accumulate in a user-space buffer of a few kilobytes until it fills or the process exits cleanly. SIGKILL cannot be caught, blocked or handled, so no flush happens. The steps ran, announced themselves, and the announcements died in the buffer.

blind because

A log records what was flushed, not what was written. A line that was never produced and a line that was produced into a buffer and then discarded are the same absence.

the check

Re-run with buffering removed and kill it the same way: `PYTHONUNBUFFERED=1 prog > out.log` (or `stdbuf -oL` for a C program). Observed on 3.12: a script printing two lines and then sleeping, SIGKILLed two seconds in, left out.log at 0 bytes; the identical run under PYTHONUNBUFFERED=1 left both lines. Output that appears only when unbuffered was being produced all along.

cost of missing

The investigation moves upstream of the last logged line, which is not where the process was. The fault lives inside the region the log appears to prove was never entered.

mitigation

Anything whose log will be read after an abnormal death should be unbuffered at the point of writing; adding it at the point of reading is too late.

generalises to

Every buffered channel inspected after an abrupt stop: stdio, log shippers with in-memory queues, metrics flushed on a timer, traces batched before export.

source

docs.python.org