Vision-guided control system built during a robotics internship at CIRAMM (SAIT). Integrates an Intel RealSense depth camera with an ABB collaborative robot via TCP/IP, achieving ~90% detection accuracy at 30Hz.
This project integrates an Intel RealSense depth camera with ABB Robot Studio to create a vision-triggered robotic movement system. The C++ program continuously monitors the distance of objects in front of the camera and sends a trigger signal to the robot when an object enters a defined range. The RAPID program running on the robot listens for this signal and executes a pre-defined motion sequence.
- Real-time object detection using Intel RealSense depth camera
- TCP socket communication between PC and ABB Robot Studio
- RAPID program automatically executes robot motion upon receiving a trigger
- Adjustable distance thresholds for trigger activation
- Intel RealSense SDK
- C++
- ABB Robot Studio / RAPID
- TCP Socket Programming
- Load the RAPID program (
Module1_working1) onto your ABB robot or simulator. - Ensure the robot’s IP and port match the C++ program's configuration.
- Start the RAPID program and wait for a client connection.
- Install Intel RealSense SDK and configure your camera.
- Compile the C++ program.
- Ensure the IP and port match the RAPID program.
- Run the program — it will monitor the depth camera and send a trigger signal when an object enters the defined range.
- Position an object in front of the RealSense camera.
- The C++ program will detect the object’s distance.
- When the object is within the trigger range, a TCP signal is sent to the robot.
- The robot executes the pre-defined movement sequence in RAPID.
- Ensure that the TCP port is open and not blocked by firewall settings.
- Adjust
TRIGGER_DISTANCEandTOLERANCEin the C++ code to suit your setup. - The RAPID program waits for the "GO" signal before executing any movements.
This project is open-source. Feel free to use and modify it for educational and research purposes.