NanoCockpit

Performance-optimized Application Framework for AI-based Autonomous Nanorobotics

Features

• Camera configuration and acquisition
• State-estimation forwarding to GAP8
• Camera and state-estimation streaming over Wi-Fi

Structure

This repository is structured with one sub-directory for each component of the system, matched with the hardware components in a Crazyflie drone:

• stm32: code for the Crazyflie's main STM32 MCU, further divided in crazyflie-firmware (original Crazyflie firmware, with minor additions) and app (user application code).
• nina: code for the ESP32 NINA module on the AI-deck. Handles Wi-Fi communication, streams data received from GAP8 to the host computer.
• gap: code for the GAP8 SoC on the AI-deck. Handles camera configuration and acquisition, receives state estimation from STM32 over UART, streams over SPI to ESP32.
• client: Python code for the host computer. Provides ROS and GUI clients for the camera streamer, off-board inference and remote control of the drone.

Getting started

The following instructions allow you build and deploy a demo application build on top of the NanoCockpit framework to a Crazyflie drone with AI-deck. The application implements the PULP-Frontnet-based CNN tested in the Human pose estimation experiment, including on-board inference on the GAP8 SoC and closed-loop control on the STM32 MCU. More detailed information about each component is available in the respective subfolders.

System requirements

NanoCockpit has been tested with the following system configuration

• Ubuntu 22.04
• Python 3.9

Setup virtual environment

$ python3.9 -m venv venv
$ source venv/bin/activate
$ pip install \
  -r requirements.txt \
  -e src/client/aideck_cpx_streamer \
  -e src/client/crazyflie-clients-python

Prepare the Crazyflie

First, flash the 2021.06 firmware from cfclient. This ensures that the NRF51 MCU is flashed to a firmware version compatible with the STM32 firmware below. You only need to do this the first time you flash the demo on a Crazyflie.

STM32

$ source venv/bin/activate
$ cd src/stm32/app
$ make all -j8
$ make cload

GAP8

GAP SDK v3.8.1 is required to build this application. Configure your computer to use the GAP SDK Docker container as described here. Then activate the GAP SDK container:

$ source venv/bin/activate
$ cd src/gap
$ gap8 3.8.1

NanoCockpit contains a number of GAP application examples. Select the PULP-Frontnet example, then compile it and flash it to the AI-deck:

> cd examples/pulp-frontnet
> make clean
> make all flash

NINA ESP32

ESP-IDF v5.3.1 is required to build this application. Configure your computer to use the ESP-IDF Docker container as described here. Then activate the ESP-IDF container:

$ source venv/bin/activate
$ cd src/nina
$ esp 5.3.1

> idf.py clean; idf.py menuconfig; idf.py all
> openocd -f interface/ftdi/olimex-arm-usb-ocd-h.cfg -f board/esp-wroom-32.cfg -c "adapter_khz 20000"  -c 'program_esp build/partition_table/partition-table.bin 0x8000 verify' -c 'program_esp build/bootloader/bootloader.bin 0x1000 verify' -c 'program_esp build/aideck_cpx_streamer.bin 0x10000 verify reset exit'

Crazyflie Client

$ source venv/bin/activate
$ cfclient

Literature review

In our paper, we review the body of work on nanorobotics over the last five years and demonstrate both the high research interest in the topic and the Crazyflie's prominent status as de-facto standard robot platform. The data to reproduce our analysis is available in docs/literature_review as a resource to other researchers that approach the nano-drone field. We search the Elsevier Scopus citation database using the query in scopus_query.txt, resulting in the 554 papers reported in scopus_dataset.csv, with our manual annotations with information about the target drone platform.

Publications

If you use NanoCockpit in an academic context, we kindly ask you to cite the following publication:

• E. Cereda, A. Giusti, and D. Palossi, ‘NanoCockpit: Performance-optimized Application Framework for AI-based Autonomous Nanorobotics’, arXiv pre-print 2601.07476, 2026 arXiv.

@inproceedings{cereda2025nanocockpit,
  author={Cereda, Elia and Giusti, Alessandro and Palossi, Daniele},
  booktitle={}, 
  title={NanoCockpit: Performance-optimized Application Framework for AI-based Autonomous Nanorobotics}, 
  year={2026},
  volume={},
  number={},
  pages={},
  note={[Under review]}
}

Contributors

Elia Cereda¹, Alessandro Giusti¹, and Daniele Palossi^1,2.

¹ Dalle Molle Institute for Artificial Intelligence (IDSIA), USI and SUPSI, Switzerland.
² Integrated Systems Laboratory (IIS) of ETH Zürich, Switzerland.