verifyfirst

What is running

You checked ps, pgrep, or systemctl status.

What it captures

That a process with a matching name exists and has a state.

Known failures · 3

NS-012 observedobserver-in-the-sample

reads as: pgrep -f chromium returns a match after cleanup. Conclusion drawn: an instance is still running.
actually: The pattern appears in the command line of the pgrep invocation, so the search finds itself. Nothing is running.
blind because: The instrument is a process, and it is inside the set it is measuring. The output format gives no indication which row is the observer.
the check: Resolve the PID and compare it against your own: pgrep -f PATTERN | grep -v "^$$\$", or list full command lines with pgrep -af and read them.
cost of missing: Cleanup loops that never terminate, or a kill aimed at the shell performing the kill.
generalises to: Any measurement taken from inside the population being measured.

NS-026 documentedacknowledgement-mistaken-for-readiness

reads as: `systemctl start app` returns and `systemctl is-active app` says active. Conclusion drawn: the service is up and accepting connections.
actually: For Type=simple the service manager considers the unit started immediately after the main service process has been forked off — after fork() and before the new process has called execve() to invoke the actual service binary. A unit whose binary is missing, whose port is already taken, or which needs thirty seconds to warm up, is 'active' throughout.
blind because: The process list reports existence and state. A process that will fail in a moment exists now, and readiness is simply not a quantity the manager measures for this type.
the check: Ask the socket rather than the manager: `ss -ltnp 'sport = :8000'` returns a listener only when one exists, and is empty while the unit is active but not yet serving. A single request to the port distinguishes the same two states.
cost of missing: Dependent units and deploy scripts proceed against a service that is not listening. The ordering guarantee that was assumed was never offered.
mitigation: Type=notify with sd_notify(READY=1) makes activeness mean readiness; Type=exec at least waits for execve() to succeed.
generalises to: Every start-up API that acknowledges the request rather than the readiness.
source: freedesktop.org

NS-027 documentedsnapshot-of-a-cycle

reads as: `systemctl status app` shows active (running) with a PID. Conclusion drawn: the service is healthy.
actually: With Restart=always the unit crashes, waits RestartSec, and starts again. Sampled during a run it is active (running) with a fresh PID; sampled during the pause it is activating (auto-restart). Nothing in one sample says the PID is four seconds old and that fifty predecessors are gone.
blind because: The process list is a snapshot. A rapidly replaced process and a stable one are identical in any single frame; only the identity of the PID across frames separates them.
the check: Read the restart counter and the start timestamp twice, thirty seconds apart: `systemctl show -p NRestarts -p ExecMainStartTimestamp --value app`. A stable service returns the same two values both times; a flapping one returns different ones. Both properties are exposed by systemd for every service unit.
cost of missing: A deploy is signed off on a service that drops every request arriving inside its restart window, until the start rate limit is reached and it stays down for good.
generalises to: Every supervised process where the supervisor's diligence in restarting is read as the process's success in running.
source: freedesktop.org