Problems finding correct last visual change

[soulgalore]

Hi Pat,
we have a problem with finding the correct last visual change when we test our mobile version of the site, running Chrome in emulated mode (or setting the viewport to mobile phone size). The last visual change is always a lot higher but visually I can't see any change.

Here's two example videos:
https://www.dropbox.com/s/i3ch0ic9jnit28j/en.m.wikipedia.org-wiki-Facebook.mp4?dl=0
https://www.dropbox.com/s/vfya08i7t46d6gv/en.m.mediawiki.org-wiki-Download.mp4?dl=0

If I run latest master:
../visualmetrics.py -i en.m.mediawiki.org-wiki-Download.mp4 --orange --force --viewport --dir a

I get the Last Visual Change 4817 but it looks like it should be something like 1100 when I look at the video and the screenshots.

If I increase the 10% difference in individual pixels to a little higher (14%) then it looks better but is that the way to go? I got a feeling the problem for us is higher when we have more plain text on the screenshots.

Best
Peter

[pmeenan]

I'll have to look closer at the videos and run it through some testing to see what the diffs are. Sometimes it seems that with text content Chrome will turn on antialiasing or something like that when onload fires and there are slight differences around all of the text. A higher percentage worries me because it can miss some larger visible changes.

[soulgalore]

Chrome will turn on antialiasing or something like that when onload fires

Ah I see, that match my experience that something happens really late. Yep I see your point with the percentage.

[soulgalore]

Thanks @pmeenan ! I've installed another font set on the server where we run the test and it seems to work now. I'll keep the issue open during the weekend when I continue to test.

[pmeenan]

Which fonts did you install? The wptagent ubuntu installer installs "ttf-mscorefonts-installer fonts-noto*" but if there are others that are commonly installed on desktops or mobile devices that should be there I'm happy to expand the default set as well.

[soulgalore]

I've installed fonts-noto* and that seems to help I but will look to see if I can narrow it down a bit since it adds almost 300mb to the container.

[pmeenan]

FWIW, the dockerfile for wptagent should also include fonts-noto*

[soulgalore]

I was wrong. Locally on my machine the last visual change looked right, but when I pushed it to the server and let it test out different URLs, the last visual change again started to be really high. It seems to be where onLoad happens as you said with anti alias, do you know if there's a way to turn it off? I've only found the switch to turn it on that I guess was needed in older Chrome.