verifyfirst

A command's return status · test-double-without-a-contract · documented

A bare mock answers to method names the real object no longer has

NS-018 documentedtest-double-without-a-contract

A bare mock answers to method names the real object no longer has

reads as
The suite is green after a collaborator's method is renamed. Conclusion drawn: nothing depended on the old name.
actually
`Mock()` manufactures an attribute on first access and returns another Mock, which is callable and truthy. Code calling `client.charge_card(...)` against the double passes although the real class now exposes only `charge`. The test exercises an interface that no longer exists, and will keep passing however far the real object drifts.
blind because
The assertion is satisfied by the double's auto-created child. Green is a true statement about the mock, and the exit code cannot say which object the statement was about.
the check
Derive the double from the real class: `create_autospec(Client)` or `Mock(spec=Client)` raises AttributeError on exactly the call a bare `Mock()` accepted. Observed on 3.12: `Mock().exsits()` returns a truthy Mock; `create_autospec(Real).exsits()` raises AttributeError.
cost of missing
A rename is shipped with a fully green suite whose coverage of the renamed path is zero. The regression appears in production, in code the tests appeared to cover.
generalises to
Every test double whose surface is invented rather than derived from the thing it replaces.
source
docs.python.org

Reported as

Others this instrument misses

plain text · full registry