A screenshot's pixel grid is not the page's coordinate grid

A screenshot's pixel grid is not the page's coordinate grid

reads as

The capture shows the button with its centre at image pixel (400, 140), and a click is dispatched at (400, 140). Conclusion drawn: the click landed on the button.

actually

The capture was taken at a device scale factor above one, so image pixels and CSS pixels differ by that factor. devicePixelRatio is the ratio between the size of a device pixel and the size of a CSS pixel, and a screenshot is measured in the former while every scripting and automation coordinate is expressed in the latter. At scale 2 the button's centre is at CSS (200, 70); (400, 140) is a different part of the page.

blind because

The image carries no units. A 1600-pixel-wide PNG of an 800-pixel-wide viewport and a 1600-pixel-wide PNG of a 1600-pixel-wide viewport are both simply wide images.

the check

Compare the capture's pixel dimensions against the page's own report of its viewport: `window.innerWidth` and `window.devicePixelRatio`. Observed with Chrome 151 headless on one 800x600 window: at --force-device-scale-factor=1 the PNG was 800x600 with devicePixelRatio 1; at 2 it was 1600x1200 with devicePixelRatio 2; at 3 it was 2400x1800. The page reported an 800 CSS-pixel viewport in all three.

cost of missing

Every coordinate derived from the image is wrong by a constant factor, and the clicks land on whatever occupies the scaled position. Because something usually does, the run continues and reports the steps it believed it took.

mitigation

Take coordinates from the DOM via getBoundingClientRect, or divide image coordinates by the scale factor the capture was made at.

generalises to

Any measurement taken in one unit system and spent in another: viewport against document coordinates, physical against logical resolution, bytes against characters.

source

w3.org