NS-027
documentedsnapshot-of-a-cycle
A service crash-looping every few seconds reads as active between crashes
- reads as
- `systemctl status app` shows active (running) with a PID. Conclusion drawn: the service is healthy.
- actually
- With Restart=always the unit crashes, waits RestartSec, and starts again. Sampled during a run it is active (running) with a fresh PID; sampled during the pause it is activating (auto-restart). Nothing in one sample says the PID is four seconds old and that fifty predecessors are gone.
- blind because
- The process list is a snapshot. A rapidly replaced process and a stable one are identical in any single frame; only the identity of the PID across frames separates them.
- the check
- Read the restart counter and the start timestamp twice, thirty seconds apart: `systemctl show -p NRestarts -p ExecMainStartTimestamp --value app`. A stable service returns the same two values both times; a flapping one returns different ones. Both properties are exposed by systemd for every service unit.
- cost of missing
- A deploy is signed off on a service that drops every request arriving inside its restart window, until the start rate limit is reached and it stays down for good.
- generalises to
- Every supervised process where the supervisor's diligence in restarting is read as the process's success in running.
- source
- freedesktop.org