Visual Odometry and SLAM System

A C++ implementation of a Visual Odometry and SLAM system using stereo image sequences. The system processes stereo image pairs to track camera motion in real-time with 3D visualization.

Features

• Feature detection (Deep Learning based) and tracking across stereo image pairs
• Camera pose estimation using PnP
• 3D visualization of camera trajectory and map points
• Stereo depth estimation (Deep Learning based)
• Keyframe selection and management
• Map point creation and tracking
• Real-time performance
• Support for custom stereo datasets

Requirements

• C++17 compatible compiler
• CMake 3.10+
• OpenCV 4.0+
• Pangolin (for 3D visualization)
• libtorch (PyTorch C++ API)

Installation (Ubuntu/Debian)

sudo apt update
sudo apt install build-essential cmake libopencv-dev
# Install Pangolin for visualization
git clone https://github.com/stevenlovegrove/Pangolin.git
cd Pangolin
mkdir build && cd build
cmake ..
make -j$(nproc)
sudo make install

Quick Start

# Build the project
mkdir build && cd build
cmake ..
make

# Run the demo
./visual_odometry

Dataset Setup

The system uses a stereo image dataset stored in the stereo_rgb directory. The dataset follows a structure similar to KITTI format with some customizations. See data/README.md for detailed format specifications and setup instructions.

The main configuration file config.yaml in the project root contains camera calibration parameters and other settings for the visual odometry system.

Usage

Running the Visual Odometry System

# Build the project
mkdir build && cd build
cmake ..
make -j$(nproc)

# Run the visual odometry system
./visual_odometry

The application will load the stereo dataset from the stereo_rgb directory and process the frames to estimate camera trajectory.

System Components

The visual odometry system consists of several key components:

1. DataLoader: Loads stereo image pairs from the dataset
2. Frontend: Detects and tracks features across frames
3. PoseEstimator: Estimates camera pose using PnP
4. VisualSLAM: Core SLAM processing including keyframe selection and map management
5. Viewer: 3D visualization of camera trajectory and map points
6. SLAMMap: Stores and manages map points and keyframes

Current Status and Next Steps

The current implementation includes:

• Visual odometry with feature detection and tracking
• Camera pose estimation using PnP
• 3D visualization of camera trajectory
• Map point creation and management

Future development plans:

• Bundle adjustment for trajectory optimization
• Loop closure detection
• IMU fusion for improved pose estimation
• Complete SLAM pipeline with global optimization
• Multi-threading optimizations

Troubleshooting

• "Dataset not found": Check data/README.md for format requirements
• "OpenCV not found": Install with sudo apt install libopencv-dev
• "Pangolin errors": Make sure Pangolin is properly installed
• "Segmentation fault on exit": This is a known issue related to thread synchronization during shutdown

Dependencies

This project uses the following third-party libraries and frameworks:

ALIKED (Attentive Local and Implicit Keypoint Detector)

• A deep learning-based feature detection and matching framework
• Used for robust keypoint detection across stereo frames
• Repository: ALIKED_CPP
• License: MIT

ESMStereo

• Enhanced ShuffleMixer Disparity Upsampling for Real-Time and Accurate Stereo Matching
• Used for deep learning-based stereo depth estimation
• Repository: ESMStereo
• Paper: ESMStereo: Enhanced ShuffleMixer Disparity Upsampling for Real-Time and Accurate Stereo Matching

Other Dependencies

• OpenCV: Computer vision library
• Pangolin: Lightweight visualization framework
• LibTorch: PyTorch C++ API
• CMake: Build system

License

MIT License - see LICENSE file for details.