SmaCC: Returning annotation from a production

[joelreymont]

Is it possible to return expr instead of {{DestExpr}} in the below production?

That is, I would like to return {{}} while dropping the left and right bracket tokens.

JumpDest
	: Ident {{}}
	| <lbracket> SemExpr 'expr' <rbracket> {{DestExpr}}
	| Integer {{}}
	| Constant <lbracket> Ident 'id' <rbracket> {{DestConstant}}
	| Label {{}}
	;

[joelreymont]

@j-brant Please correct my understanding...

There's no returning just expo with SmaCC. All the tokens are returned, explicitly (expo) or implicitly (<lbracket>) annotated aka named.

That is, each token in a production becomes a field.

The only control I can exercise here is to annotated the tokens and name the AST node (DestExpr).

[j-brant]

If DestExpr node is a superclass or the same class as return by SemExpr, then you can remove the 'expr' from the expression and it will figure out that you want to add the <lbracket> and <rbracket> to the DestExpr class. Also, I'm not following what you mean by "return {{}}". {{}} is a shortcut to create a node named based on the production's name. In the example above, {{}} is the same as {{JumpDest}}. Finally, I think you want to name the productions in Ident {{}}, Integer {{}}, and Label {{}}. These are just creating nodes without any values in the nodes.

[joelreymont]

I actually don't care about the brackets and similar tokens. Ideally, I would want to keep just the explicit annotations as instance variables. This is not possible at the moment, correct?

[joelreymont]

Why return tokens like brackets, comma, etc. from a production?

[j-brant]

You can return anything from a production. For example, you can return Foo from the production Production: Foo Bar Baz; by doing this: Production: Foo Bar Baz {'1'};. Basically, every production has some code that is run when the production is matched. You can specify exactly what the code is by using { } or [ ] code blocks. You can also have SmaCC define nodes by using {{ }} and also defining a %root. If you want some token included in a node, then you must give it a variable name in the production.

[joelreymont]

I'm asking about the opposite, I suppose.

Can I easily exclude tokens from being returned, e.g. the brackets, braces, commas, semicolons, etc.?

Also, are there examples of using the code blocks to specify what a production returns?

[j-brant]

If you don't name the token, then they are excluded by default from being AST node. The code blocks are explained somewhat here: https://books.pharo.org/booklet-Smacc/

[joelreymont]

I thought that tokens would be excluded but...

EndianDef
	: <define><endian> <assign> Endian 'value' <semi> {{}}
	;

results in

Shouldn't defineToken, endianToken, assignToken and semiToken not be there?

P.S. I'm not using %annotate_tokens.

[j-brant]

Can you post your grammar? It works for me:

%root Program;
%prefix Foo;
%suffix Node;
<define>
	: define
	;
<endian>
	: endian
	;
<assign>
	: \=
	;
<value>
	: [a-zA-Z]+
	;
<whitespace>
	: \s+
	;
<semi>
	: \;
	;
EndianDef
	: <define><endian> <assign> Endian 'value' <semi> {{}}
	;
Endian
	: <value>
	;

This generates a EndianDef class that only includes a value variable.

[joelreymont]

I don't want to post the grammar but can send it to you on Discord, privately.

[joelreymont]

@j-brant I added you as a collaborator to my grammar repo.

Please let me know if you can see the extra tokens in each AST node.

[joelreymont]

I can confirm that your example does not include any extra tokens.

I even tried removing %root and %suffix, testing and putting %root back.

It still works.

[joelreymont]

By the way, I tried removing the repo cloning it after restarting GT.

I still get the extra token instance variables.

[joelreymont]

@j-brant The test file for the grammar is in the .slaspec file at the top of the repo.

[joelreymont]

I started with a fresh copy of GT and cloned my repo.

I still get the extra tokens as instance variables.

[j-brant]

Your grammar has %annotate_tokens;

[joelreymont]

I feel like an idiot now for assuming that the Grammar view was showing all the directives.

Thank you John!