Unicode support in Scilla strings

[anton-trunov]

Scilla does not support Unicode at the moment.

This contract

scilla_version 0

contract HelloWorld ()

transition HelloWorld ()
  e = {_eventname: "你好世界"};
  event e
end

does not pass the checker:

$ scilla-checker -gaslimit 8000 -libdir "$(opam var scilla:lib)/stdlib" helloWorld.scilla

helloWorld.scilla:6:33: error: Syntax error: Invalid String literal 你好世界

Gas remaining: 8000

It would be nice to either document this explicitly in the Scilla docs or add Unicode support.

[jubnzv]

For now, the lexer don't parse unicode characters (symbols started with \u) so this is the expected behavior.

If we need to add it, we should probably tweak our read_string function the same way as in OCaml lexer: https://github.com/ocaml/ocaml/blob/af13e29e0499d479035b064d9d70d9614e76a4fe/lex/lexer.mll#L240-L249.

[jjcnn]

The blast radius for this fix could become quite big.

We definitely need to test any solution against both the state database, the crypto primitives and the PolyNetwork library, to make sure that nothing breaks. We also need to test the integration with the blockchain itself, to make sure that unicode strings in messages are passed correctly between contracts, and that API calls to Scilla work as intended.

[txalkan]

Thank you guys! How long could this take?

[jjcnn]

Thank you guys! How long could this take?

It definitely won't be ready in time for v0.13 given the amount of time it would take to test it properly.

I don't think we have started planning for v0.14 yet, but we'll definitely keep it in mind for that release. No idea what the release date is for v0.14, though.

[jjcnn]

Tentatively marking this for v0.14, since it would be a useful feature to have.