Dual CSI Cameras with External Trigger Support

[AhmedAyman3030]

First of all, thank you for the great work on Mandeye Controller and the overall ecosystem. It’s very impressive and extremely useful for LiDAR-based mapping workflows.
Is it possible to use two CSI cameras simultaneously in Mandeye Controller with external hardware trigger?
My goal is to synchronize:
Livox Mid-360 LiDAR
2 × CSI global shutter cameras
For offline point cloud colorization, so frame-level sync and timestamp alignment are important.
Is it planned?

[JanuszBedkowski]

We are working

[JanuszBedkowski]

We did not yet solved sync

[AhmedAyman3030]

Thank u for ur Replay ... I am currently researching this topic, and I’m happy to help if I can.

From what I found, to achieve reliable sync between LiDAR and cameras, we will likely need cameras that support external hardware trigger.

For example, cameras like this (CSI camera with external trigger support):
https://www.aliexpress.com/item/1005010016095116.html

With this type of camera, we can use:
External trigger signal
Same trigger sent to LiDAR + Camera
Then align timestamps for point cloud colorization.

My goal is to sync LiDAR + camera captures for offline colored point clouds as we do in manually colorization in HD-Mapping

[JanuszBedkowski]

Just let us know when You need help

[GeomPozzoli]

Isn't it possible to generate the fake PPS and GPRMC signals via the Raspberry Pi's GPIOs and then connect pins 8 and 10 of the Mid-360 to synchronize the lidar to the Raspberry Pi's time, and then use the same pin of the fake PPS to trigger the camera shutter like the one linked by Ahmed?
I saw in the services "sudo systemctl enable mandeye_fakepps.service" on the system, but I don't understand which GPIO refers to and if there is something similar for the GPRMC.

I refer to the wiki points 1.3.3.4.2.1. Synchronization via UART or 1.3.3.4.2.2. Synchronization via Ethernet:
https://livox-wiki-en.readthedocs.io/en/latest/tutorials/new_product/common/time_sync.html#gps-time-synchronization

[michalpelka]

@GeomPozzoli there is a lot things to do with synchronization with Livox (and other lidars).
The prototype Janusz posted is using:

• PTP server to synchronize Livox
• and camera is synhronized software way (it uses LibCamera). Libcamera attaches Wall time to photo. I think it allows 10-50 ms precision.

You can extend FakePPS (that fakes PPS) to generate signals to trigger cameras to do stereo-like synchronization.

In other words - in this project we synchronized Lidars both with PPS/NMEA (Fake PPS) and PTP (take a look at apendix)

[michalpelka]

I am not planning that currently (I want to evaluate if it is needed), but definetelly is possible.
I encourage you to contribute such setup or fork this project. If you need help using those resources, you can reach me.

[GeomPozzoli]

Coding and programming are far from my skills, but maybe with the help of some AI I'll be able to burn some boards or sensors........ 🤣

[GeomPozzoli]

@michalpelka I'm doing some experiments and I think I've found something that might work by adding some code without changing the code itself. However, I have a question about how the fake pps in the fake_pps.cpp file works (fork not full tested: https://github.com/GeomPozzoli/mandeye_controller ).
I made some changes to output the pps and nmea signals to specific pins, but when I connect an LED to the pps signal, the LED stays on and then turns off for a moment. From the code, it seems to work like this (unless there are any misinterpretations):

HIGH ~990ms → LOW 10ms → HIGH 100ms → HIGH ~890ms → ...

while on the livox wiki for synchronization it provides something like:

LOW ~900ms → HIGH 100ms → LOW ~900ms → ...

Is there some reason in the standard system behavior for this low/high inversion? And does changing it cause problems with the rest of the code?

Tnx for the info
Stefano

[michalpelka]

When I have time I will hook osciloscope.
Probably I was fine with Livox accepting this PPS signal and call it a night.

[GeomPozzoli]

Hi,

I’ve been working on a set of modifications to extend the current fake PPS implementation, introducing configuration via a .json file. The updated setup now exposes:

• PPS
• NMEA
• An additional trigger signal synchronized with PPS (frequency configurable in Hz)

I also revised the shooting workflow to support synchronized image acquisition. This is primarily intended for use with the Innomaker GS hardware trigger, but it also supports cameras without hardware trigger by selecting the frame closest to a software trigger event.

At the moment, the implementation is limited to a single camera. I do not have access to a Raspberry Pi 5 for testing, and the Raspberry Pi 4 does not allow truly simultaneous captures, even when using multi-camera add-ons.

I should note that the development was largely assisted by Claude.ai. While I do not have a strong programming background, after extensive iteration and refinement the system appears to be working reliably. Further validation through external scanning tests is still required to properly assess real-world performance.

All changes are currently available in the feature/pps-trigger-camera branch.
I have also added a setup guide describing the new features and configuration steps:
https://github.com/GeomPozzoli/mandeye_controller/blob/feature/pps-trigger-camera/doc/CAMERA_TRIGGER_SETUP.md

I hope this contribution may be useful for future project enhancements.

Best regards,
Stefano

[AhmedAyman3030]

@GeomPozzoli
Thanks for sharing this work!

Just a quick question — were you able to generate a properly colorized point cloud using this setup?

Thanks!