How does it work?

Everything that one needs to know in order to understand the code of the project.

Architecture of the system

Vocabulary:

• A concrete language is the program as written in a file, for instance System.out.println("Hello") is Java concrete language for printing out to the screen.
• An abstract language is the computer's representation of the program, as an Abstract Syntax Tree, held in memory.
• An input language is what the GOOL system can take as input.
• The source language is what the GOOL system will take as input.
• An output language is what the GOOL system can take as output.
• The target language is what the GOOL system will take as output.
• A primitive is any piece of a language.
• Recognition is when a specific input primitive (e.g. Java Lists) is associated a specific GOOL primitive (e.g. GOOL Lists).
• Generation is when a specific GOOL primitive (e.g. GOOL Lists) is associated a specific output primitive (e.g. C# Lists).
• Passing on is when a specific primitive is associated a generic primitive.
• The core of a language are the control flow (if,...), basic types (int,...), basic operations (=, new...) of a language.
• The libraries of a language are the available classes.
• The system libraries of a language are those available in the standard distribution.
• The default libraries of a language are those system libraries which are imported by default.
• The user libraries of a language are those made available by the user.

Words of explanation:

• If the input language is concrete, then it will be made abstract in the easiest way available: the input's language native parser.
• The abstract input language then gets translated into the common abstract language GOOL.
• During this process, some specific input language primitives are recognized (e.g. Java Lists), meaning that they are represented by specific GOOL primitives (e.g. GOOL Lists), i.e. GOOL is aware of their specific nature. The others are just passed on: they are represented in a generic GOOL primitives (e.g. some GOOL Class X).
• The GOOL abstract language then gets translated into output concrete language.
• During this process, some specific GOOL primitives (e.g. GOOL Lists) provoke code generation, meaning that they get implemented as specific output language primitives (e.g. C# Lists), i.e. the output language will for sure handle those. The others are just passed on: they are implemented as generic output language primitives (e.g. some C# Class X). This generic, syntactic translation may still be useful (e.g. if the user later provides a C# implementation of Class X). Still, a comment is generated which warns whether a primitive was not recognized or did not provoke code generation.
• The generated concrete code can be compiled and run.

Who does what?

We explain which package does what. The order is chronological with respect to the above Figure. Within each package, each file comes with an explanation of its purpose.

• gool: the root of the project, and where global settings are.
• gool.parser.XXX¹: to parse the XXX input language from concrete to abstract.
• gool.parser: the class that must be extended to write a gool.parser.XXX¹.
• gool.recognizer.XXX¹: to translate the XXX input abstract language into its GOOL abstract language representation.
• gool.recognizer.common: the common classes for all input languages.
• gool.ast: the description of the GOOL abstract language.
• gool.ast.core: the core of the GOOL abstract language.
• gool.ast.__________: the hard-coded libraries of the GOOL abstract language.
• gool.generator.YYY²: to translate the GOOL abstract language into YYY output concrete language.
• gool.generator.common: the classes that must be extended to write a gool.generator.YYY².
• gool.generator: useful across generators (batching, parameterization etc.)
• gool.executor.YYY²: native compilation and running of the YYY output concrete language.
• gool.executor.common: the classes that must be extended to write a gool.executor.YYY².
• gool.executor: useful across executors (running a command etc.)
• logger: useful classes for logging.
• webgool.application: main class for launching the webgool server.
• webgool.server: websoket manager.

The src/ folder also contains:

• main/resources: Some resources associated with the java generator and recognizer packages (output languages templates for different kind of libraries and some properties files that helps generators to associate input language libraries with GOOL abstract virtual libraries).
• test: GOOL tests.

In the GOOL project root folder one will find:

• index.html: the webgool client entry file.
• Elements/: the folder that contains the webgool client source codes (javascripts and css files).
• build.gradle and associated gradle files: the files used by gradle to build the project.

Chains of events

Tip: on Eclipse, press control and click on a name to see its declaration.

Parsing

What happens, in the GOOL system, when concrete java gets parsed into abstract GOOL?

• Initially this is the job of GOOLCompiler.concreteJavaToAbstractGool(...)
• which delegates it to JavaParser.parseGool(...)
• which calls Sun's java parser thereby obtaining abstract java trees and
• which launches JavaRecognizer.scan() on each abstract java tree
• which calls a JavaRecognizer.visitClass(...) on each abstract java class
• which creates an abstract GOOL class and fills it by doing an accept(...) on each element of the abstract java class
• which calls a JavaRecognizer.visitSomething(...) on this element
• which creates the corresponding abstract GOOL element, possibly filled-in by doing an accept(...) on its sub-elements, etc.
• until the leafs of the abstract Java tree are reached.

Generating

What happens, in the GOOL system, when abstract GOOL gets flattened into concrete target?

• Initially this is the job of GOOLCompiler.abstractGool2Target(...)
• which delegates it to GeneratorHelper.printClassDefs(...)
• which works out the target language for each class, and delegates it to the right CodePrinter.print(...)
• which creates the corresponding file (i.e. two strings, one with its full name and one with its content), and fills it with ClassDef.getCode()
• which, via CodePrinter.processTemplate(...), tells velocity to fill in the template class.vm with itself.
• Take generators.java.templates.class.vm for instance. Velocity fills it with the content of each element of the class, (fields, methods, etc.) by calling a .toString() upon them
• which calls JavaCodeGenerator.getCode(...) upon itself
• which generates the appropriate character string for the element in the concrete target language
• which may require performing .toString() upon sub-elements
• which calls JavaCodeGenerator.getCode(...) upon sub-elements, etc.
• until the leafs of the abstract GOOL tree are reached.
• Finally all the created files are either sent to the web client that displays them or printed on the disk by the GOOLCompiler.printFiles(Map<String, String> files) function.

Compilation and execution of the webgool files into docker containers

The webgool server gets the possibility to compile and execute the generated files in docker containers. It is done by the function GOOLCompiler.launchHTMLExecution. Each output language uses its own docker container. The files are compiled and executed within the container and the standard and error outputs are sent to the web client.

By now the following container are used :

• java -> openjdk:8
• C++ -> reaverproject/gcc-boost:5_1_0-1.60.0
• python -> python:3.5
• C# -> mono:latest

You can change the docker container directly within the GOOLCompiler.launchHTMLExecution function.

Testing

All the tests are written in the src/test folder. No systematic unit tests have been written, only some functional tests exist. Some of them do compare some translations with a gold reference one (the gold files are given in the src/test/GOOLREFERENCEYYY² folders. Others are compiling and executing translations to verify that they are correct.

In the most cases, the tests are working this way:

• A test is launching (through ./gradlew -Dtest.single=GoolTestTheTestIWantToLaunch for example),
• which wraps a print with TestHelper.surroundWithClassMain(...) and hands it to compareResultsDifferentPlatforms(...)
• which creates a new GoolTestExecutor(input, expected) and for each of its platform triggers GoolTestExecutor.compare(...)
• which compares the expected result with that of GoolTest.compileAndRun(...)
• which is just a cleaned up version of TestHelper.generateCompileRun(...)
• which does parsing to GOOL and generating from GOOL as before with a GOOLCompiler.concreteJavaToConcretePlatform() and then seeks to execute the concrete target with ExecutorHelper.compileAndRun(), which:
• which first delegates compiling the concrete target with its native compiler to SpecificCompiler.compileToExecutable(...) and second delegates executing the compiled concrete target with the machine to SpecificCompiler.run(...).

Important data structures

Abstract Java

is documented on the web, e.g. http://docs.oracle.com/javase/8/docs/api/javax/lang/model/package-summary.html .

Platforms

• They specify the target language for an abstract GOOL class.
• They are defined in gool.generator.YYY.YYYPlatform².
• They can be considered to be an abstract GOOL PrimitiveType.
• They have a name, a CodePrinter and a SpecificCompiler.
• The CodePrinter tells how to name files, and their general format, for that target language.
• The SpecificCompiler tells how to translate each abstract GOOL element into concrete target.

ClassDefs

• They are the abstract GOOL classes.
• They are defined in gool.ast.constructs.
• They can be considered to be Dependency.
• They have a name, a target platform.
• They have a package, a parent, interfaces, modifiers, constructors, fields, methods...
• They have indications on whether they are themselves enums, interfaces...
• They have dependencies.

Dependencies

• They stand for the external classes which are used in the code of some abstract GOOL class.
• They have a name.
• During parsing process, when JavaRecognizer finds a non-primitive type, or an import, it registers it as a Dependency of the class. Moreover the classes being parsed are registered as dependencies of each other.
• Often gool.generator.YYY.YYYGenerator² maintains a register of dependencies.

How libraries are dealt with

In the beginning, the GOOL system was not making any distinction between the core versus the libraries of a language. The GOOL abstract language was representing its libraries (e.g., lists, map, system out print calls) just like it was representing its core primitives: each with a dedicated GOOL Abstract Syntax Tree node. It meant that in order to recognize and generate a new library (a class and its method), you had to:

• Create new GOOL Abstract Syntax Tree nodes dedicated to representing the library.
• Implement their recognition by the different input language Recognizers.
• Implement their generation by the different output language Generators.

This was the ''hard-coded'' solution. However, there is now an alternative, ''soft-coded'' solution, by means of the GOOL Library Manager, which is more modular and much more convenient for users who are not familiar with the whole code of the project. You can find out how a presentation of the GOOL Library Manager here: Libraries support and of its inner workings, here: The library manager.

Some nitty gritty details

• The code generation gets started by a toString() called upon the GOOL Abstract Syntax Tree. This is because for code generation, we proceed by filling gaps in some velocity templates, and velocity calls toString(). Such a toString() then gets converted into a getCode(this) by the GOOL Abstract Syntax Tree. At this point this needs to have the appropriate type of that piece of the Abstract Syntax Tree, so that the right getCode(...) gets called, as getCode(...) is overloaded. To achieve this, getCode(this) must be performed at that level of the Abstract Syntax Tree. Indeed, if the conversion had been done in some generic GOOL Node, then the overriding of getCode(...) would not have worked, since overriding is done statically in Java. This is why, in GOOL Node, we force overriding of toString() by the children nodes, but this is just a matter of converting it to a getCode(this) in each child.

• The concept of a Platform specifies a gool.generator.CodePrinter, and hence a gool.generator.CodeGenerator, together with a gool.executor.SpecificCompiler for the Target language. In other words it tells you how to generate the concrete Target, and how to execute it. That way you can have different platforms of the same Target language, which you execute differently etc. Platforms are kept track of at two levels. First of all, they are remembered in some global static register. Second of all, each abstract GOOL class knows its Platform. This is in provision for multi-platform compilation, where different pieces of the abstract GOOL tree compile to different Platforms.

The log system

A log system if redefined in the gool.logger package. To log a message anywhere in the code one can call
a static print method of the gool.logger.Log class like:

Log.d("My debug message");

The gool.logger.config.ini configuration file given in the src/main/resources defines the different printers used according to the ini4j standards. A new printer can be directly defined in this file. The existing ones can also be enabled or disabled by setting TRUE or FALSE the enable field.

Five levels of logs are available : ALL, DEBUG, INFO, WARN, ERROR. Each one can be selected directly by the static method used to print:

• Log.a(message) do log the message for all level.
• Log.d(message) do log the message for all and debug levels.
• Log.i(message) do log the message for all, debug and info levels.
• Log.w(message) or Log.w(exception) do log the message or the exception for all, debug, info and warn levels.
• Log.e(message) or Log.e(exception) do log the message or the exception for all, debug, info, warn and error levels.

1: We use XXX to denote an input language (cpp or java)

2: We use YYY to denote an output language (cpp, java, csharp or python))