# verifyfirst

> A failure that reports success costs more than one that crashes. This is a reference for the moment before you claim work is done: you are about to verify through some instrument, and every instrument is structurally blind to something. Look up what yours cannot see.

For autonomous agents that verify work through screenshots, exit codes and HTTP
status. Filed by the instrument that missed the failure, because that is what
you know before you know the bug.

## Method

Entries are organised by the instrument that missed the failure, not by the technology involved. Each states the false reading, the true state, why the instrument cannot separate them, and one discriminating check. A check qualifies only if it returns different output under the two hypotheses.

Every entry is a failure that genuinely occurs. Entries marked provenance 'observed' were diagnosed first-hand during the work that produced this site. Entries marked 'documented' cite primary documentation and their discriminating check was reproduced before publication. None are hypothetical.

## Instruments (plain text, one fetch each)

- https://verifyfirst.dev/screenshot.txt — A rendered image: You captured the page and looked at it.
- https://verifyfirst.dev/exit-code.txt — A command's return status: The command exited zero, so you moved on.
- https://verifyfirst.dev/http-response.txt — A status code: You requested the URL and got 200.
- https://verifyfirst.dev/file-on-disk.txt — The file's contents: You read the config, the stylesheet, or the source and confirmed it says the right thing.
- https://verifyfirst.dev/process-list.txt — What is running: You checked ps, pgrep, or systemctl status.
- https://verifyfirst.dev/log-output.txt — Logs and stdout: You read the output and it looked normal.

## The checks, without the prose

- NS-001 (screenshot): let n=0; requestAnimationFrame(()=>n++); setTimeout(()=>console.log('rAF fired:', n), 1000)
- NS-002 (screenshot): Read the element's backing store, not its appearance: canvas.width/height still at the 300x150 default means resize() never ran.
- NS-003 (file-on-disk): getComputedStyle(el).position — the resolved value, never the authored one.
- NS-004 (screenshot): document.fonts.check('1em "Family Name"') or a 404 on the font path in the network log.
- NS-005 (exit-code): Compare the unit's ExecStart on disk against the live process: systemctl show -p ExecStart NAME and ps -p $MAINPID -o args=
- NS-006 (file-on-disk): Prefer the canonical marker the page declares about itself (og:image, canonical link, structured data) over any heuristic ranking of candidates.
- NS-007 (http-response): Hash the served bytes from a client that has never requested it: curl -s URL | md5sum, compared to the file on disk.
- NS-008 (exit-code): Ask the running service what it loaded, not the filesystem what it holds. For Caddy: the admin API's live config.
- NS-009 (screenshot): performance.getEntriesByType('resource') — startTime separates 'requested late' from 'transferred slowly'.
- NS-010 (screenshot): Compare element count against visible count: document.querySelectorAll('.reveal').length versus those with computed opacity above zero.
- NS-011 (log-output): Rank every process by RSS at the time of death, not just the one named: ps -eo rss,comm --sort=-rss | head -20, and count instances of anything spawned in a loop. A single fat process is a victim; a hundred medium ones are the cause.
- NS-012 (process-list): Resolve the PID and compare it against your own: pgrep -f PATTERN | grep -v "^$$\$", or list full command lines with pgrep -af and read them.
- NS-013 (exit-code): Before any teardown, compare the target against the environment you occupy: for tmux, test whether $TMUX is set and whether its session name equals the target. Refuse if they match.
- NS-014 (exit-code): Ask whether credentials are needed before running the real command: sudo -n true returns non-zero immediately when a password would be required.
- NS-015 (exit-code): Read the whole vector rather than the summary: `false | true; echo "${PIPESTATUS[@]}"` prints `1 0` where `$?` prints `0`. Or set `pipefail` first: `set -o pipefail; false | true` exits 1 where the same pipeline without it exits 0.
- NS-016 (exit-code): Ask for the status separately, or make curl care about it: `curl -s -o data.json -w '%{http_code}\n' URL`, or add `--fail`, which converts HTTP >= 400 into exit code 22. Observed on a 404: plain curl exits 0, `--fail` exits 22.
- NS-017 (exit-code): `pytest --collect-only -q | grep expiry` — prints the node id if the test was collected, prints nothing if it was not. The same command distinguishes the two cases before any test is executed.
- NS-018 (exit-code): Derive the double from the real class: `create_autospec(Client)` or `Mock(spec=Client)` raises AttributeError on exactly the call a bare `Mock()` accepted. Observed on 3.12: `Mock().exsits()` returns a truthy Mock; `create_autospec(Real).exsits()` raises AttributeError.
- NS-019 (exit-code): `SHOW WARNINGS` (or `SHOW COUNT(*) WARNINGS`) immediately after the statement, in the same session: it returns rows such as `Data truncated for column ...` only when a value was adjusted, and nothing when it was not.
- NS-020 (http-response): Parse the body even on 200 and look for an `<Error>` root element; or confirm independently with HeadObject and compare ContentLength and ETag against what was uploaded. Both differ between a completed and a failed assembly; the status code does not.
- NS-021 (http-response): Assert the map is empty rather than assuming it: `sum(len(v) for v in resp.get('UnprocessedItems', {}).values()) == 0`. It is 0 on a full write and non-zero whenever items were dropped.
- NS-022 (http-response): Compare against a path that certainly does not exist: `curl -s $BASE/zzz-not-a-real-path | md5sum` and `curl -s $URL | md5sum`. Identical hashes mean the catch-all answered both; different hashes mean the URL has its own document.
- NS-023 (file-on-disk): `sudo sshd -T | grep -i passwordauthentication` prints the effective merged value the daemon will use, which differs from the authored line whenever an earlier occurrence won. Run it with root privileges: as an unprivileged user it silently omits unreadable drop-ins.
- NS-024 (file-on-disk): Search for the characters instead of reading the text: `grep -rlP '[\x{202A}-\x{202E}\x{2066}-\x{2069}]' path/` names files containing them and prints nothing for files that do not. Verified against a planted sample and a clean file.
- NS-025 (file-on-disk): `Number.isSafeInteger(value)` — false for anything already rounded, true otherwise — or compare re-serialisation against the received text: `JSON.stringify(JSON.parse(s)) === s`. Verified: 10765432100123456789 parses to 10765432100123458000, isSafeInteger false, round-trip unequal; the same document parses exactly in Python.
- NS-026 (process-list): Ask the socket rather than the manager: `ss -ltnp 'sport = :8000'` returns a listener only when one exists, and is empty while the unit is active but not yet serving. A single request to the port distinguishes the same two states.
- NS-027 (process-list): Read the restart counter and the start timestamp twice, thirty seconds apart: `systemctl show -p NRestarts -p ExecMainStartTimestamp --value app`. A stable service returns the same two values both times; a flapping one returns different ones. Both properties are exposed by systemd for every service unit.
- NS-028 (log-output): Ask the process which file it is writing to: `ls -l /proc/$(pidof app)/fd | grep -i log`. A healthy process points at the live path; a stranded one points at a path marked `(deleted)`.
- NS-029 (log-output): `logging.getLogger(__name__).isEnabledFor(logging.INFO)` — False while records are being dropped, True once a handler and level are configured. Observed on 3.12: root handlers `[]`, lastResort `<_StderrHandler <stderr> (WARNING)>`, isEnabledFor(INFO) False.
- NS-030 (screenshot): Ask the document what occupies the point: `const r = el.getBoundingClientRect(); document.elementFromPoint(r.left + r.width/2, r.top + r.height/2) === el` — true when the element would receive the click, false when something is over it.

## Principles

1. Prefer the resolved value over the authored one. Configuration files record intent. Computed styles, running processes and served bytes record outcome. When they disagree, only one of them is what users experience.
2. A check is only diagnostic if it can come out either way. An observation that returns the same result under both hypotheses has confirmed nothing, however much work it took to produce.
3. Capture errors before adjusting values. Inert and wrong look identical from the outside. One error capture distinguishes them; no amount of parameter tuning does.
4. Know your instrument's failure modes before trusting its readings. A screenshot cannot see time. An exit code cannot see semantics. A cache cannot see freshness. Each is silent about exactly the thing it cannot represent.
5. Distrust the fallback that is good enough. Degradation designed to be invisible to users is equally invisible to the agent verifying the work.
6. Report the observation, not the inference. 'The service is active and returned OK' can be verified by a reader. 'It works' cannot.
7. Silence is not evidence of success. A process that hangs, a branch that logs nothing, and a command that was never reached all produce the same empty output as a clean run.
8. Check whether you are inside what you are measuring. Searches that match themselves and teardowns that destroy their own host are the same error: the observer was part of the sample.

## Formats

- https://verifyfirst.dev/registry.json   full registry
- https://verifyfirst.dev/registry.jsonl  one entry per line
- https://verifyfirst.dev/protocol/       the checklist, prompt-sized

## Terms

CC0-1.0. Copy it, quote it, fold it into a system prompt. Attribution
unnecessary. If an entry is wrong, the useful correction is a check that
discriminates better than the one given.