Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 58 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,73 @@
## IntelliCar
#### Application of Computer Vision Techniques for Object Detection and Recognition in Urban Environment Simulation oriented to Autonomous Driving Vehicles
中英文版
# IntelliCar
## 面向自动驾驶的城市仿真环境下计算机视觉目标检测与识别技术研究
---
##### INTRODUCTION
This project is focus on image object detection by training a model using YOLO. Simulations are done on CARLA Simulator and a integration with ROS has been developed.

## 项目概述
本项目基于YOLO深度学习模型实现图像目标检测与识别,在CARLA自动驾驶仿真平台完成环境验证,并搭建ROS通信架构实现算法与仿真环境的联动部署。
![A](imgs/portada-edit.jpg)

##### TOOLS

+ **Dataset**: We create our custom dataset, using Roboflow Platform, by the integration of some existing datasets and the addition of multiple images of scenarios with different environmental and weather conditions.
## 技术工具
- **数据集**:基于Roboflow平台构建自定义数据集,融合开源数据集并扩充多场景、多天气条件下的样本,提升模型泛化能力。
[Roboflow数据集链接](https://universe.roboflow.com/carla-awmfg/carladataset/model/5)
- **仿真平台**:采用CARLA开源自动驾驶仿真器,提供高保真城市交通环境与传感器数据支持。
[CARLA官网](https://carla.org/) | [CARLA GitHub](https://github.com/carla-simulator/carla)
- **检测模型**:基于Ultralytics YOLOv8算法构建目标检测模型,实现实时、高精度感知。
[YOLOv8 GitHub](https://github.com/ultralytics/ultralytics)
- **机器人操作系统**:使用ROS完成模型推理、感知结果转发与决策控制。
- **通信桥梁**:通过ROS-Bridge实现CARLA与ROS之间的数据交互。
[ROS-Bridge GitHub](https://github.com/carla-simulator/ros-bridge)
![A](imgs/global_diagram.png)
---
## 模型性能评估
![A](imgs/confusion_matrix_normalized.png)
![A](imgs/pr-curve.png)
![A](imgs/results.png)
---
## 实验效果展示
![A](imgs/inferencia.png)
![A](imgs/collage-demo.png)
## 交通灯识别与紧急制动系统
设计交通灯状态检测逻辑与紧急制动监督模块,实现基础安全驾驶辅助功能。
![A](imgs/traffic_light_task_diagram_.jpg)
![A](imgs/classic_color_segmentation.png)
---
## 项目文档
完整技术细节与实验分析详见:Memory.pdf
## 演示视频
[![DEMO-Video](https://img.youtube.com/vi/j6nA76eiCRw/0.jpg)](https://www.youtube.com/watch?v=j6nA76eiCRw)
---
English Optimized Version
# IntelliCar
## Research on Computer Vision-Based Object Detection and Recognition for Autonomous Driving in Urban Simulation Environments
---
## Project Overview
This project implements image object detection and recognition based on the YOLO deep learning model. All simulations are conducted on the CARLA autonomous driving simulator, and a ROS-based architecture is developed to connect perception algorithms with the simulation environment.
![A](imgs/portada-edit.jpg)
## Technical Tools
- **Dataset**: A custom dataset is built on the Roboflow platform by combining existing open datasets and augmenting samples under diverse scenarios and weather conditions to improve model generalization.
[Dataset on Roboflow Platform](https://universe.roboflow.com/carla-awmfg/carladataset/model/5)

+ **Simulator**: We use the open-source simulator for autonomous driving systems CARLA.
[CARLA web page](https://carla.org/)
[CARLA Github](https://github.com/carla-simulator/carla)

+ **Model Training**: Model based on newest version of YOLO (YOLOv8) from Ultralytics.
[YOLOv8 Github](https://github.com/ultralytics/ultralytics)

+ **ROS**: Running model inference for prediction and extra decission tasks modules.

+ **ROS-Bridge**: CARLA - ROS communication.
[ROS-Bridge Github](https://github.com/carla-simulator/ros-bridge)

- **Simulation Platform**: CARLA, an open-source autonomous driving simulator, is used to provide high-fidelity urban traffic scenes and realistic sensor data.
[CARLA Website](https://carla.org/) | [CARLA GitHub](https://github.com/carla-simulator/carla)
- **Detection Model**: A real-time object detection framework is developed based on YOLOv8 from Ultralytics.
[YOLOv8 GitHub](https://github.com/ultralytics/ultralytics)
- **ROS**: Used for model inference, result publishing, and high-level decision-making modules.
- **ROS-Bridge**: Enables reliable data communication between CARLA and ROS.
[ROS-Bridge GitHub](https://github.com/carla-simulator/ros-bridge)
![A](imgs/global_diagram.png)
---
##### MODEL PERFORMANCE:
## Model Performance
![A](imgs/confusion_matrix_normalized.png)
![A](imgs/pr-curve.png)
![A](imgs/results.png)
---
##### EXPERIMENTS RESULTS:
## Experimental Results
![A](imgs/inferencia.png)
![A](imgs/collage-demo.png)

##### TRAFFIC LIGHT STATE AND EMERGENCY BRAKE SUPERVISOR
## Traffic Light Detection & Emergency Brake Supervisor
A traffic light state recognition module and emergency brake supervisor are designed to support basic safe driving functions.
![A](imgs/traffic_light_task_diagram_.jpg)
![A](imgs/classic_color_segmentation.png)


---
##### PROJECT REPORT:
More info about the project can be found on Memory.pdf

##### FUNCTIONAL VIDEO:
## Project Report
Full technical details and experimental analysis are available in Memory.pdf.
## Demo Video
[![DEMO-Video](https://img.youtube.com/vi/j6nA76eiCRw/0.jpg)](https://www.youtube.com/watch?v=j6nA76eiCRw)