-
Notifications
You must be signed in to change notification settings - Fork 6
Getting Started
- Simple C++: We try to avoid as much as possible high-level C++ mechanisms such as templates, pointers to members, multi-class inheritance, etc. to favor portability, memory-efficiency and readability
- Public access: We usually make everthing in a class public. Members that are not meant to be publicly accessible are prefixed by an underscore (_). Qualia is designed for experimenters, researchers, creators and hackers, so you should be free to play with stuff if you want (and if you're ready to accept the consequences).
- Few dependencies: In order to be cross-platform, we try to use only the most basic, common tools on all platforms. We thus stick to tools that are available on AVR platforms: the C Standard Library and part of the Arduino/Wiring library. We try as much as possible to avoid using the C++ Standard Library.
Additional notes:
- in order to facilitate cross-platform development, the basic Arduino/Wiring functions, definitions and macros are made available when compiling for computer or non-Arduino AVR.
- Some tools are provided that only work on a computer platform: they are located in the
src/qualia/computerfolder in the source repository. - Our coding philosophy is largely inspired from that of the Torch 3 machine learning library. See chapter 2 of the documentation (PDF).
In Qualia, you can easily switch between compiling code on a PC and on an AVR microcontroller. This is done by setting the flag platform at construction. Supported platform settings:
- computer : general-purpose computer (Linux, OSX, Windows)
- arduino : Arduino board
- avr : AVR microcontroller (not necessarily Arduino)
To these settings correspond three important macros:
is_computer()is_arduino()is_avr()
Notice: The macro is_avr() will be true on Arduino boards, since Arduinos use an AVR chip.
One of the biggest problems encountered when working with AVR chipsets is that the SRAM is very small (between 1k to 8k depending on the model) and the stack and the heap share the same memory. Therefore, using dynamic allocation can result in unexpected behaviors, especially for memory-consuming applications.
To overcome this issue, Qualia favors a model of computation where memory is allocated only at initialization. Moreover, Qualia provides a flexible memory management framework that allows for different ways of managing memory. In particular, using class StaticAllocator allows one to reserve a block of memory on the heap from which memory is to be allocated (although it isn't able to "free" memory).
For this to work, dynamic allocation needs to be performed through static class Alloc methods. Qualia also provides custom templates and macros Q_NEW, Q_ARRAY_NEW, Q_DELETE and Q_ARRAY_DELETE to be used instead of the new, new[], delete and delete[] operators.
Here is an example of use. Look at how we switch allocator to a StaticAllocator on an AVR/Arduino platform.
#if is_computer()
Allocator allocator; // use standard allocator
#else
unsigned char STATIC_BUFFER[1000];
StaticAllocator allocator(&STATIC_BUFFER); // use static allocator
#endif
Alloc::init(&allocator);
int* val = (int*) Alloc::malloc(10*sizeof(int));
MyClass* obj = Q_NEW(MyClass)(1, 2); // MyClass* bbj = new MyClass(1,2);
MyClass* arr = Q_ARRAY_NEW(MyClass,10); // MyClass* arr = new MyClass[10];
...
Alloc::free(val);
Q_DELETE(MyClass, ptr);
Q_ARRAY_DELETE(arr);The easiest way to get the latest version is to clone the git master branch:
git clone https://github.com/sofian/qualia.git
Then:
cd qualia
To compile (for computer platform):
scons
To install to default directory (/usr/local/lib):
scons install
To install to alternative directory:
scons install PREFIX=/usr/
There are optional "plugin" modules that can be compiled and installed. For now, these modules are only available on a computer platform. They are:
- bt : Behavior Tree plugin based on Libbehavior
- osc : OSC plugin (requires Liblo)
- mapper : Libmapper plugin
To compile the code with plugins, use the PLUGINS argument:
scons PLUGINS=bt,osc
scons PLUGINS=bt,osc install
A Qualia program usually consists in three interconnected parts:
- Agent
- Environment
- Qualia
Note: The complete code for this example is provided in the tools/examples/drunk folder.
You should always start any Qualia code by including the <qualia/core/common.h> header file. You should also include whatever headers you are going to be using.
#include <qualia/core/common.h>
#include <qualia/core/Agent.h>
#include <qualia/core/Environment.h>
#include <qualia/core/Qualia.h>For this example, we will create a simple "drunk" agent that randomly chooses between increasing a value (action "0") or decreasing it (action "1"). The agent thus chooses a one-dimensional action with two possible actions in the dimension. See how this is defined below by creating an ActionProperties instance and assigning it to the Action instance that will be returned by the Agent.
const unsigned int N_ACTIONS[] = { 2 };
class DrunkAgent : public Agent {
public:
ActionProperties props; // the action properties
Action action; // the action that will be returned by the agent's start() and step() methods
DrunkAgent() : props(1, N_ACTIONS), action(&props) {}
virtual ~DrunkAgent() {}We then define an init() method (in our case, we have nothing to do so we'll just print a message):
virtual void init() {
Q_MESSAGE("Initializing agent");
}And then, most importantly, we need to implement a start() and a step() methods:
virtual Action* start(const Observation* observation) {
Q_MESSAGE("Starting agent");
return step(observation);
}
virtual Action* step(const Observation* observation) {
action[0] = random(2); // picks either 0 or 1, randomly
Q_MESSAGE("Current observation: %f / Next action: %d.", observation->observations[0], action[0]);
return &action;
}
};That's it! Now our Agent is ready to make actions in an environment.
That's it for the Agent, at least for now. Now, you have noticed that this agent only takes a random action. But the result of the action is still undefined. This is because the outcome of the agent's action is the responsibility of the enviroment.
For the sake of the example, we'll imagine the agent moves along a one-dimensional grid with 100 spaces. When the agent chooses to move in one direction, it steps one space in that direction, unless it's hitting the border.
class DrunkEnvironment : public Environment {
public:
Observation observation;
DrunkEnvironment() : observation(1) {}
virtual ~DrunkEnvironment() {}
virtual void init() {
Q_MESSAGE("Initializing enviroment");
}
virtual Observation* start() {
observation[0] = 50; // we start in the middle
return &observation;
}
virtual Observation* step(const Action* action) {
// step left or right depending on the action
if (action->actions[0] == 0)
observation[0]--;
else
observation[0]++;
// constrain within grid
observation[0] = constrain(observation[0], 0.0f, 100.0f);
return &observation;
}
};Now that we have an Agent and an Environment definitions, we can start a program that will have them interact with one another, producing a behavior. Qualia provides a tool for that called ... "Qualia" (!) Here's how it works:
void main() {
DrunkAgent agent; // the agent
DrunkEnvironment environment; // the environment
// Qualia is constructed using an agent and an environment
Qualia qualia(&agent, &environment);
qualia.init();
qualia.start();
for (int i=0; i<100; i++)
qualia.step();
return 0;
}Notice: An alternative to calling
qualia.start();
for (int i=0; i<100; i++)
qualia.step();is to simply call:
qualia.episode(100);Compile by linking to the Qualia static library.
g++ -o program.o -c -I/path/to/qualia/include -I. program.cpp
g++ -o program program.o -L/path/to/qualia/lib -lqualia -lm
Alternatively, you can use scons. For compiling executables, Qualia uses the comavarscons scons build files, available in tools/comavarscons.
Yes, this is a rather dummy example. For instance, our DrunkAgent has not even a sense of what is going on: it just takes a random action, whatever it observes!
It is easy to change this behavior: all you have to do is update the start() and step() method. For example, the agent could at least choose to go the other way when it hits a border:
virtual Action* step(const Observation* observation) {
if (observation->observations[0] <= 0)
action[0] = 1; // go right
else if (observation->observations[0] >= 100.0f)
action[0] = 0; // go left
else
action[0] = random(2); // picks either 0 or 1, randomly
Q_MESSAGE("Current observation: %f / Next action: %d.", observation->observations[0], action[0]);
return &action;
}But the main thing to understand at this point is that you can easily use any agent class with the DrunkEnvironment, as long as it supports the same action-taking and observation framework (ie. the same ActionProperties and the same Observation dimension). Also, you can easily use the DrunkAgent with another Environment class, as long as it respects those constraints.