testing what is possible vs what is likely

[aaronbell]

Out of interest, I note the first word in the Lao diffenator test data is super tall with lots of diacritics (https://github.com/googlefonts/fontheight/blob/main/static-lang-word-lists/data/diffenator/Lao.txt)

While this is quite a test of the possible height of the font if the GPOS is fully active, does it represent a reasonable test of language support for Lao?

If one is trying to use this library to gauge where font metrics should be set this seems quite extreme, no?

[RickyDaMa]

Hey Aaron! It's definitely a question worth asking - maybe in future fontheight could report percentiles or something, so you can get an idea of how extreme of an outlier a word is compared to the rest of the list.

The other angle to approach this from is the word lists themselves. There is no such thing as a perfect word list, but maybe you'd want some with the extreme outliers culled, or some really small lists with only the tallest words included.

Both are valid approaches imo and I'm open to the discussion.

For your case with Lao specifically, if you can confirm it's not a real word then I'd be happy to remove it from the word list entirely. Otherwise, I do think it has a purpose in being included, and that's where the visual report (fontheight --html) can help show visually and allow you to make the judgement call for what metrics to set for your font. At no point do we say that your metrics should be set to the most extreme values reported by fontheight - that's left as a designer decision.

[aaronbell]

The magic of having a clipping vs a line height metric is hopefully you never have to choose whether to support an outlier, or to allow it to get clipped.

I guess I was wondering where these lists came from? As I mentioned with the Lao example, the first entry in that document has 10 combining diacritic marks! That appears to me more likely to be a GPOS test string than something that could ever exist in written language, so it makes me question the validity of the lists as a whole for the intended purpose.

[RickyDaMa]

The diffenator wordlists are derived from Wikipedia dumps is my understanding, and quite heavily processed down to a manageable size. You can see the scripts involved over on the diffenator2 repo.

In private repositories I work on we've also noticed diffenator word lists generally containing less 'real world' strings, and so for clipping we now rely on the AOSP & LibreOffice word lists to produce metrics, using autobase (which reflects this word list preference) to produce feature code, and then reviewing a font after it's been compiled in with fontheight --html.