Add support for some F# syntax features

[Thorium]

Add support for the following F# syntax features (all tests passing):

• open type declarations
• FSI directives (#time, #I, #help, #quit)
• XML doc comments (/// as distinct xml_doc node)
• Type test pattern in atomic patterns (:? Type)
• Quotation splicing (%%) prefix operator
• fixed expressions
• Range expressions in computation expressions
• Preprocessor boolean conditions (&&, ||, !, parens, true/false)
• Extern/P/Invoke declarations
• SRTP trait call expressions (^a : (static member ...) )
• Triple-quoted string interpolation ($""" {expr} """)
• Operator precedence for && and || (split into 3 levels)
• module rec (recursive modules)
• and! in computation expressions
• struct tuple type annotations (struct (int * int))
• Add optional type_argument_constraints to _function_or_value_defn_body for SRTP 'when' constraints on return type annotations
• Add optional 'struct' to anon_record_type for struct anonymous record types
• Add optional 'then' clause to additional_constr_defn for secondary constructor initialization expressions
• Expand fsharp_signature parser: named_module, namespace (global/rec), module_defn, type_definition, exception_definition, import_decl, module_abbrev, compiler_directive_decl, preproc_if support
• Fix indentation bug in type extension with ($) identifier test
• Add quotation expression support (<@ @>, <@@ @@>) with external tokens
• Add multi-dollar triple-quoted string interpolation ($$"""...""", $$$"""...""")
• Add module ... = begin...end with begin as external token
• Add exception named fields (of field1: type * field2: type)
• Add multiline type provider support via _multiline_generic_type
• Add signature parser named parameters (curried_spec)
• Fix scanner serialize/deserialize bugs (clamped count, bounds check, off-by-one)
• Update highlights, injections, indents queries

Scanner changes:

• Add FORMAT_TRIPLE_QUOTE_CONTENT external token that stops at unescaped { for interpolation support

Grammar changes:

• fsharp/grammar.js: new rules (trait_call_expression, extern_binding, extern_param, and_bang, struct_type, _preproc_expression, xml_doc), extended existing rules (import_decl, prefixed_expression, module_defn, named_module, infix_expression, format_triple_quoted_string)
• fsharp_signature/grammar.js: added conflict for operator precedence
• common/scanner.h: FORMAT_TRIPLE_QUOTE_CONTENT with {{ escape handling

[Thorium]

After this #76 and #156 is fixed.

[Thorium]

Ok after the next commit, all open issues should be also fixed.
I'm trying to get this repo out of "wip" status.

After this there are a few known "issues" still, but I don't see them so important:

• No native query support (but works as any CE)
• No unit of measures (they are just generics?)
• Signature files support is improved but not complete (who cares?)
• CE code is still partially in comments...
• Anything else?

[Thorium]

Test in test\highlight\type_definitions.fsx failed:
That's a very interesting test. It says < should be marked as an operator, but it is marked as a function. Which would be wrong if it's like function as operator f < 4 as of (<) f 4 vs f<int>. However, I argue the full call ResizeArray<int>() is actually a function call, the constructor of the ResizeArray and thus as part of that < in this context is a part of a generic function call, not a separate operator. I think it's justified to change the test? So, if you search for "give me all the operators", the resize-array-constructor-call-generic-type-argument shouldn't come as part of that.

[Thorium]

I took a random compiling Fantomas linted project:
12 files out of 216 *.fs files failed (24 errors)
So there is still some work to do.

[Nsidorenco]

Really nice you're picking this up! I fixed the workflow so the CI now tests the parser against the FSharp.Core testsuite again - that should give a pretty good indication of the state of the parser.

[Nsidorenco]

If you do not return DEDENT/PREPROC_END tokens during error recovery you get a much worse parse tree during typing since it will I many cases be able to identify a partial parse tree

Effectively, if you use tree-sitter for syntax highlighting and write something like

match x with

It will fail to highlight anything since it lacks the DEDENT token to identify this is a partially correct match-statement

[Nsidorenco]

We should generally be wary of the size of the parser. It went from ~30mb to ~50mb here. 30mb was already rather large. An increase in parser size generally comes from an increased ambiguity within the grammar and is probably one of those things where mimicking the language spec won't necessarily lead to a performant tree-sitter parser