The UTF8 and UTF16 classes in cpp::encoding mostly duplicates the encoding which also occurs in String.cpp. I would have updated string to also use those dedicated encoding classes but doing some benchmarking of converting 5 paragraphs of lorem ipsum with a few emojis in it from UTF16 to UTF8 showed it to be twice as slow.
So there probably needs to be an optimisation pass to close the distance before unifying the two.
The UTF8 and UTF16 classes in
cpp::encodingmostly duplicates the encoding which also occurs inString.cpp. I would have updated string to also use those dedicated encoding classes but doing some benchmarking of converting 5 paragraphs of lorem ipsum with a few emojis in it from UTF16 to UTF8 showed it to be twice as slow.So there probably needs to be an optimisation pass to close the distance before unifying the two.