set -e aborts a script at a validation step that concerns something else
reads as
The install script ran and the config file is in place. Conclusion drawn: the change is active.
actually
A validation step covering the whole configuration failed on an unrelated block that needed an environment variable the script did not load. Under set -e the script exited before the reload.
blind because
The steps before the failure completed and left visible artifacts. Partial success looks like success when only the artifacts are inspected.
the check
Ask the running service what it loaded, not the filesystem what it holds. For Caddy: the admin API's live config.
cost of missing
The config is correct on disk and absent from the process, an inconsistency that survives inspection of either side alone.
generalises to
Any pipeline where a global check gates a local change.
A teardown script destroys the environment it is executing inside
reads as
Several long-running sessions vanish at once with no error output. Conclusion drawn: the tool crashed, or the machine failed.
actually
A rebuild script ran `kill-session` against the multiplexer session it was itself running in. It killed its own parent, taking four unrelated sessions with it. There is no crash and no error because the script did exactly what it was told.
blind because
A process that is killed cannot report that it was killed, and cannot report why. The absence of an error reads as an unexplained crash rather than a successful destructive command.
the check
Before any teardown, compare the target against the environment you occupy: for tmux, test whether $TMUX is set and whether its session name equals the target. Refuse if they match.
cost of missing
Work in progress across every session in the environment, lost with no diagnostic trail.
generalises to
Any tool that can destroy a container, session, service, or host that it might itself be running inside.
A privilege prompt with nowhere to appear hangs instead of failing
reads as
A deploy step produces no output and does not return. Conclusion drawn: the operation is slow, or the network is stalling.
actually
The command needed a password. There is no terminal to prompt on, so it waits indefinitely. No error, no exit code, no timeout.
blind because
Exit codes only exist for processes that exit. An instrument that reads return status has nothing at all to read, and silence resembles work in progress.
the check
Ask whether credentials are needed before running the real command: sudo -n true returns non-zero immediately when a password would be required.
cost of missing
An agent waits on a command that will never return, and a task that needed a human is reported as in progress.
mitigation
Wrap anything that might prompt in a timeout, so a hang converts into a failure you can observe.
generalises to
Every interactive prompt reached from a non-interactive context: credentials, confirmations, pagers, editors.
A pipeline returns the status of its last command, not its failing one
reads as
`npm test | tee build.log` exits zero and the log file is written. Conclusion drawn: the tests passed.
actually
A shell reports the exit status of the last command in a pipeline. The test runner exited 1; tee wrote the log and exited 0, and 0 is what the pipeline returns. `set -e` does not intervene, because the pipeline as a whole succeeded.
blind because
One number is produced for a chain of processes. The failing member's status is overwritten by its successor's, and the overwrite leaves no trace in the value the caller reads.
the check
Read the whole vector rather than the summary: `false | true; echo "${PIPESTATUS[@]}"` prints `1 0` where `$?` prints `0`. Or set `pipefail` first: `set -o pipefail; false | true` exits 1 where the same pipeline without it exits 0.
cost of missing
Every failure inside a command piped into tee, grep, jq, head or a formatter is recorded as success. A build stays green across a broken test run, and the log written alongside it is treated as proof.
mitigation
`set -euo pipefail` at the top of any script whose exit status will be believed by something else.
generalises to
Any composition that collapses several results into one and keeps the last rather than the worst.
curl exits zero after successfully downloading an error page
reads as
`curl -s -o data.json URL` exits 0 and data.json exists with content in it. Conclusion drawn: the fetch succeeded.
actually
The server answered 404 or 500. curl's task — transferring what the server chose to send — completed without fault, so the exit status is 0 and an HTML error page is now sitting in data.json under the name of the expected document.
blind because
The exit code describes the transfer, not the response. A transferred error page is a completed transfer, indistinguishable at that layer from a transferred payload.
the check
Ask for the status separately, or make curl care about it: `curl -s -o data.json -w '%{http_code}\n' URL`, or add `--fail`, which converts HTTP >= 400 into exit code 22. Observed on a 404: plain curl exits 0, `--fail` exits 22.
cost of missing
A downstream step parses an HTML error page as the config, dataset or credential file it expected. The failure surfaces at the parser, far from the request that caused it.
generalises to
Every client whose success criterion is that the protocol completed, rather than that the answer was the one asked for.
A test that does not match the discovery pattern is neither run nor reported
reads as
pytest exits 0 with a green summary after a new test is added. Conclusion drawn: the new test passes.
actually
Collection matches `test_*.py` or `*_test.py` files, and `test`-prefixed functions or methods inside `Test`-prefixed classes. A file named `tests_auth.py`, or a function named `check_expiry`, is never collected. The green result belongs entirely to the other tests. The exit code that signals an empty run, 5, applies only when nothing at all was collected, so any other test in the suite conceals the omission.
blind because
An uncollected test produces no pass line and no fail line. The summary counts what ran; it has no term for what was skipped by never being seen.
the check
`pytest --collect-only -q | grep expiry` — prints the node id if the test was collected, prints nothing if it was not. The same command distinguishes the two cases before any test is executed.
cost of missing
The behaviour the test was written to protect is unprotected, and the suite's green status is subsequently cited as evidence that it is protected.
generalises to
Any convention-driven runner where registration is implicit and non-registration is silent: test discovery, plugin loaders, autoloaded fixtures, route decorators.
A bare mock answers to method names the real object no longer has
reads as
The suite is green after a collaborator's method is renamed. Conclusion drawn: nothing depended on the old name.
actually
`Mock()` manufactures an attribute on first access and returns another Mock, which is callable and truthy. Code calling `client.charge_card(...)` against the double passes although the real class now exposes only `charge`. The test exercises an interface that no longer exists, and will keep passing however far the real object drifts.
blind because
The assertion is satisfied by the double's auto-created child. Green is a true statement about the mock, and the exit code cannot say which object the statement was about.
the check
Derive the double from the real class: `create_autospec(Client)` or `Mock(spec=Client)` raises AttributeError on exactly the call a bare `Mock()` accepted. Observed on 3.12: `Mock().exsits()` returns a truthy Mock; `create_autospec(Real).exsits()` raises AttributeError.
cost of missing
A rename is shipped with a fully green suite whose coverage of the renamed path is zero. The regression appears in production, in code the tests appeared to cover.
generalises to
Every test double whose surface is invented rather than derived from the thing it replaces.
Outside strict mode MySQL stores an adjusted value and calls the statement successful
reads as
The INSERT returns `Query OK, 1 row affected` and the client exits 0. Conclusion drawn: the row was stored as supplied.
actually
With strict mode absent from sql_mode, MySQL 'inserts adjusted values for invalid or missing values and produces warnings'. A string longer than the column is truncated to fit; `'abc'` into an integer column becomes 0. The statement is not aborted and the affected-row count is the same as for a clean insert.
blind because
Warnings are a separate channel that must be asked for. Neither the return status nor the row count changes when a value is adjusted, so the two outcomes are identical to anything reading the result of the statement.
the check
`SHOW WARNINGS` (or `SHOW COUNT(*) WARNINGS`) immediately after the statement, in the same session: it returns rows such as `Data truncated for column ...` only when a value was adjusted, and nothing when it was not.
cost of missing
Truncated identifiers and coerced numbers are indistinguishable from real data once written, and the originals are gone. Corruption is discovered by a later join that finds nothing.
mitigation
Assert the mode rather than assume it: `SELECT @@SESSION.sql_mode` should contain STRICT_TRANS_TABLES before any load is trusted.
generalises to
Any writer that repairs input rather than rejecting it: lenient parsers, schema-on-read stores, spreadsheet imports.