Generated parser for Rascal language itself has spurious ambiguity around nested conditionals

[jurgenvinju]

rascal>parse(#start[Module], |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|)
|TODO:///|: Ambiguity(
  |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>),
  "Type",
  "IndexOutOfBounds")

where the old parser, from util::Reflective, does this:

rascal> t = parseModuleWithSpaces(|file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|);
start[Module]: (start[Module]) `@license{
  Copyright (c) 2009-2015 CWI
  All rights reserved. This program and the accompanying materials
  are made available under the terms of the Eclipse Public License v1.0
  which accompanies this distribution, and is available at
  http://www.eclipse.org/legal/epl-v10.html
}
@contributor{Jurgen J. Vinju - Jurgen.Vinju@cwi.nl - CWI}
@contributor{Tijs van der Storm - Tijs.van.der.Storm@cwi.nl}
@contributor{Paul Klint - Paul.Klint@cwi.nl - CWI}
@contributor{Arnold Lankamp - Arnold.Lankamp@cwi.nl}
@contributor{Vadim Zaytsev - vadim@grammarware.net - SWAT, CWI}

@synopsis{Library functions for strings.}
@description{
The following library functions are defined for strings:
(((TOC)))
}
module String

When we focus on the the ambiguity:

rascal>a = firstAmbiguity(#start[Module], |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|);
Tree: amb({appl(
      prod(
        label(
          "user",
          sort("Type")),
        [label(
            "user",
            sort("UserType"))],
        {}),

it becomes clear that Nonterminal is not filtered from Type -> Symbol as it should be:

rascal>diagnose(a)
list[Message]: [
  info(
    "Ambiguity cluster with 2 alternatives",
    |dunno:///|),
  info(
    "Production unique to the one alternative: QualifiedName = \\default: {Name \"::\"}+ names !\>\> \"::\" ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the one alternative: Type = user: UserType user ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the one alternative: Name = (  [A-Z _ a-z] !\<\< [A-Z _ a-z]   [0-9 A-Z _ a-z]* !\>\> [0-9 A-Z _ a-z]  ) \\ RascalKeywords ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the one alternative: {Name \"::\"}+;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the one alternative: (  [A-Z _ a-z] !\<\< [A-Z _ a-z]   [0-9 A-Z _ a-z]* !\>\> [0-9 A-Z _ a-z]  );",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the one alternative: UserType = name: QualifiedName name ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the other alternative: Nonterminal = (  [A-Z] !\<\< [A-Z]   [0-9 A-Z _ a-z]* !\>\> [0-9 A-Z _ a-z]  ) \\ RascalKeywords ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the other alternative: (  [A-Z] !\<\< [A-Z]   [0-9 A-Z _ a-z]* !\>\> [0-9 A-Z _ a-z]  );",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the other alternative: Sym = nonterminal: Nonterminal nonterminal !\>\> \"[\" ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Production unique to the other alternative: Type = symbol: Sym!labeled!parametrized!nonterminal!parameter symbol ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "The alternatives have different productions at the top, one has \n  Type = user: UserType user \nwhile the other has\n  Type = symbol: Sym!labeled!parametrized!nonterminal!parameter symbol ",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "The alternatives have different lexicals/literals/layout",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Unique lexical to the one: Name = (  [A-Z _ a-z] !\<\< [A-Z _ a-z]   [0-9 A-Z _ a-z]* !\>\> [0-9 A-Z _ a-z]  ) \\ RascalKeywords ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  info(
    "Unique lexical to the other: Nonterminal = (  [A-Z] !\<\< [A-Z]   [0-9 A-Z _ a-z]* !\>\> [0-9 A-Z _ a-z]  ) \\ RascalKeywords ;",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  error(
    "IndexOutOfBounds is overlapping with IndexOutOfBounds, add follow restrictions or a symbol table!",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  error(
    "IndexOutOfBounds is overlapping with IndexOutOfBounds, add follow/precede restrictions!",
    |file:///Users/jurgenv/git/rascal/src/org/rascalmpl/library/String.rsc|(1758,16,<65,44>,<65,60>)),
  error(
    "IndexOutOfBounds is overlapping with IndexOutOfBounds, add follow/precede rest...

The conditional Sym!labeled!parametrized!nonterminal!parameter symbol ; has not worked.

Not that this is a nested conditional for which recently fixes have been applied.

[DavyLandman]

on which version of rascal are you running this? Like which REPL?

So does this mean we have a regression on the parser generator?

[jurgenvinju]

Yes we seem to have a regression on the parser generator or the grammar definition code. I'm running the REPL on the rascal-formatter PR.

The flattener seems to work correctly:

rascal>for (/c:conditional(_,_) := #start[Module].definitions[sort("Type")]) iprintln(c);
conditional(
  sort("Sym"),
  {
    except("labeled"),
    except("parametrized"),
    except("nonterminal"),
    except("parameter")
  })

[jurgenvinju]

This happened only on the #2346 PR.