The Unicode character œ does not render in LibRocket

[MjnMixael]

Discovered while localizing BtA, the font used for BtA in SCPUI (SquarishSans) includes U+0153 and displays it correctly in external font viewers. Other non-ASCII characters (e.g. é, æ) render correctly in the same UI context. However, the character œ consistently fails to render when included in strings. This is getting way outside my knowledge of fonts and utf-8, but I think œ is outside the normal Latin-1 range which suggests that UI strings may be treated as single-byte characters at some point in the libRocket text pipeline causing multi-byte UTF-8 sequences to be corrupted.

[chromi]

One way to verify this would be to try strings written entirely in non-Latin alphabets. Squarish Sans includes the Greek alphabet for both the small-caps and standard variants, so βετα Ακωιλε would be a good test.

[MjnMixael]

One way to verify this would be to try strings written entirely in non-Latin alphabets. Squarish Sans includes the Greek alphabet for both the small-caps and standard variants, so βετα Ακωιλε would be a good test.

Confirmed. None of those characters are displayed in librocket-rendered elements.

This is especially problematic because FSO's internal string rendering does support these characters correctly. So this issue is limited to the librocket library or the interface between librocket and FSO.

[chromi]

Interestingly, I don't think the character code is merely being truncated at the high end. If that were the case, most if not all of the characters tried here would map cleanly onto printable Latin-1 characters. For example, U+0053 is a capital S, which would be "wrong" if you wanted U+0153, but at least visible. Misinterpreting a UTF-8 encoding also seems unlikely, because Latin-1 characters are being displayed correctly, and these are already 2-byte encodings (from U+00A1 upwards) in UTF-8.

Instead, I suspect something is filtering out high-coded characters (beyond U+00FF) entirely. It may be that librocket or one of its dependencies has a glyph cache that is limited to the first 256 codepoints. Higher codepoints thus always map to an empty cache entry and return a null glyph.