It would be pretty helpful for error reporting (and possibly map generation) if we attached the tokens to the trees.
Potential problems
- Attached token objects should contain as much information as possible (e.g., raw source text, start/end positions, etc.). We may run into problems here because of memory (we might be able to intern some stuff for repeated identifiers, etc. to alleviate this problem). We should decide how positions should be resolved (offset, source file+offset, source file+line number+column).
- Attached tokens should be immutable & independent of parsing architecture (this may be a problem if we keep a reference to JSLexer for lazy position resolution)
- Attached tokens should be optional to provide to allow synthetic trees and easier adoption
It would be pretty helpful for error reporting (and possibly map generation) if we attached the tokens to the trees.
Potential problems