🖐️ Hand Tracking 3D

A real-time 3D hand visualization tool that bridges Python's MediaPipe tracking with a Three.js web environment.

✨ Features

• Real-time Tracking: Utilizes MediaPipe for high-performance hand landmark detection.
• 3D Visualization: Renders a rigged 3D hand model in the browser using Three.js.
• Dual Hand Support: Capable of tracking and identifying both left and right hands.
• Low Latency: Uses UDP/WebSocket bridging for fast data transfer between Python and the browser.
• Visual Feedback: Skeleton overlay and landmark spheres for debugging and visualization.

🛠️ Tech Stack

• Python: Captures video and processes hand landmarks using mediapipe.
• Node.js: Acts as a bridge server, receiving UDP packets from Python and broadcasting via WebSockets.
• Three.js: Renders the 3D scene and handles the rigged model animation.
• Vite: Modern frontend tooling for fast development.

📋 Prerequisites

Before you begin, ensure you have the following installed:

• Python 3.x
• Node.js (includes npm)

🚀 Installation

1. Install Python Dependencies

   pip install opencv-python mediapipe numpy

2. Install Node.js Dependencies

   npm install
   # or if you haven't initialized yet
   npm install express three ws vite

🎮 Usage

You will need three terminal instances to run the full stack.

1. Start the Bridge Server

This server acts as the middleman. It listens for tracking data from Python and sends it to the web client.

node bridge.js

Expected output: Bridge is ready! UDP listening on port 5052, WebSocket on port 5053.

2. Start the Frontend

Launch the web application to view the 3D model.

npx vite
# or
npm run dev

Open the URL shown (usually http://localhost:5173).

3. Start Hand Tracking

Run the Python script to start the camera and tracking.

python main.py

A camera window will open. Show your hands to the camera to see the 3D model move in the browser!

🔧 Troubleshooting

• Port in Use: If you see an EADDRINUSE error, ensure no other instances of the servers are running. The default ports are 5052 (UDP) and 5053 (WebSocket).
• Camera Not Found: Ensure your webcam is connected and not being used by another application.
• No Movement in 3D:
  • Check if the Bridge Server is running.
  • Ensure the browser console shows "Connected to hand tracking server".
  • Verify Python script is printing landmark data (e.g., Hands: R=True...).

📂 Project Structure

├── main.py           # Python script for MediaPipe tracking
├── bridge.js         # Node.js server (UDP <-> WebSocket bridge)
├── main.js           # Three.js logic and scene setup
├── library/          # Helper classes for finger/hand control
├── public/           # Static assets (3D models)
└── index.html        # Entry point for the web app

---

Built with ❤️ utilizing MediaPipe and Three.js