🎮 Pose-Based Game Controller

A real-time, hands-free game controller that uses Python, OpenCV, and MediaPipe to translate your body movements into in-game actions.

This project captures video from your webcam and uses the MediaPipe Pose landmark model to detect gestures for movement, aiming, jumping, and more, allowing you to play first-person games with your entire body.

🤸 Gameplay & Features

This controller is designed to replace standard keyboard and mouse inputs for first-person games. Stand in front of your camera and perform the following actions:

🏃 Movement (Walking, Running, Strafing)

• Single Step: Lift one foot to take a single, short step forward.
• Walking: Take a second step within 1.2 seconds of the first to begin walking continuously.
• Sprinting: Take a second step within 0.5 seconds of the first to begin sprinting.
• Walk Backwards: Raise both hands above your head and perform the walking motions.
• Strafe Left/Right: Turn your shoulders 90 degrees to the left or right and perform the walking motions to move sideways.

🖱️ Aiming & Mouse Control

• Look/Aim: Bring your hands close together in front of your chest to enter "Looking Mode."
  • The mouse cursor will move based on your hands' position relative to the center of your torso.
  • The direction of your hands determines the look direction (up, down, left, right, and diagonals).
  • The distance of your hands from your center determines the look speed.
• Left-Click (Hold): Raise your left hand above your head, then bring it down below your shoulder to activate and hold the left mouse button. Release by raising your hand above your shoulder or lowering it to your side.
• Right-Click (Hold): Raise your right hand above your head, then bring it down below your shoulder to activate and hold the right mouse button.
• Scroll Down: Perform a fast horizontal swipe from left to right with your left hand.
• Scroll Up: Perform a fast horizontal swipe from right to left with your right hand.

🧍 Other Actions

• Jump: Jump physically in real life to press the space key.
• Crouch: Lower your hips close to your knees to activate and hold crouch (shift key). You can walk and perform mouse clicks while crouching.

🛠️ Installation & Setup

Prerequisites

• Python: Python 3.8+
• Webcam: A standard webcam connected to your computer.
• C++ Compiler: Required by some Python libraries (usually pre-installed with build tools like Visual Studio on Windows).

1. Clone the repository to your local machine.

   git clone <your-repository-url>
   cd <repository-folder>

2. It is highly recommended to use a Python virtual environment.

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install the required dependencies. You will need to create a file named requirements.txt in your project folder with the content shown below, then run the pip command.

   pip install -r requirements.txt

4. Run the main script.

   python main.py

5. A window showing your webcam feed with pose landmarks will appear. Click on the game you wish to control and begin performing the gestures. To stop the program, click on the webcam window and press the ESC key.