Add modern ABAP 7.40+ and CDS keywords

[mlaggner]

Updated the ABAP lexer to support more modern keywords and CDS functions/annotations (taken from https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/index.htm?file=abenabap_reference.htm)

Also fixed a few false positives

[jneen]

I do appreciate the additional specs, but would you mind adding examples to the visual spec?

[mlaggner]

I have create more visual examples and fixed two more false positives

[jneen]

Thank you!

I've found the ABAP 7.58 documentation but I'm having trouble finding where in there these keywords are listed - is there a specific sub-page they're on?

Ideally I'd like to have these keywords generated from the documentation, similar to Gherkin, VimL, etc.

[jneen]

Like, there's this https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/index.htm?file=abenabap_shortref.htm, but the keywords file here has far more than are listed in that document

[mlaggner]

This is not that easy, because basically ABAP is a mix from different aspects:

1. the core language. You found the main definition at https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/index.htm?file=abenabap_shortref.htm - but there are only the main keywords are listed and not the parameters itself. Just look at the keyword CLASS - this has many sub-keywords (https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/index.htm?file=abapclass_shortref.htm) like INHERITING, ABSTRACT, FINAL - which are not listed in the link you found.

2. ABAP has some built in functions which have arrived shortly (this means in the recent 10-20 years 😆 ): https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-us/abenbuilt_in_functions_overview.htm - those are also part of the core, but have some special syntax and are handled by ABAP in a different way

3. ABAP itself is tied to databases - and when SAP introduced core data services, they invented one more jargon (derived from SQL if I remember correctly): https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/abencds_language_elements.htm

I tried to pick the keywords and its parameters from 1. and 3. and put this into self.keywords (https://github.com/rouge-ruby/rouge/pull/2182/changes#diff-1070116206e71448ce008a76b65a82108b509cc46579dc8fd41df0aecb8e518aR16) and the built in functions from 2. into https://github.com/rouge-ruby/rouge/pull/2182/changes#diff-1070116206e71448ce008a76b65a82108b509cc46579dc8fd41df0aecb8e518aR149 (because these keywords are handled differently and may be detected as method calls otherwise)

I also found https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-us/abenabap_words.htm - this obviously also mixes 1. and 3. but there are some further keywords which are only SQL functions (and they are not in the docs from 3.), so I am unsure if I should use this as my source of truth for 1. and 3.

So for now, I am not sure what is 100% correct... But looking at the PR I saw some false positives. I will update the PR as soon as I have access to a SAP system

[jneen]

Cool, this helps a lot. My goal is to have this autogenerated from publicly available documentation, even if it's from multiple sources, or is slightly complex to look up. I'd rather run some dubious regexes on the documentation or the source than rely on systems like this to generate keywords themselves.