Hand Gesture Classification Using MediaPipe Landmarks from the HaGRID Dataset

Overview:

Classify hand gestures using landmark data generated by MediaPipe from the HaGRID (Hand Gesture Recognition Image Dataset).

• Input: CSV file containing hand landmarks (e.g., x, y, z coordinates of keypoints) extracted from the HaGRID dataset using MediaPipe.
• Output: Trained machine learning model capable of classifying hand gestures into predefined classes.

Dataset Details:

The HaGRID dataset contains 18 classes of hand gestures, including:

Each gesture is represented by a set of hand landmarks (21 landmarks per hand) extracted using MediaPipe. The CSV file contains these landmarks(x,y,z location) along with their corresponding gesture labels.

Project Codes:

1. Model Training:

• A well-documented and executable Colab notebook uploaded to GitHub. The notebook includes:
  1. Data Loading: Loading the CSV file containing hand landmarks and labels.
  2. Data Visualization: Visualizing the hand landmarks by plotting keypoints for a few samples.
  3. Data Preprocessing: Cleaning, normalizing, and preparing the data for training (feature normalization, label encoding, splitting into train/test sets).
  4. Model Training: Implementing and comparing 3 machine learning models (Random Forest, SVM, XGBoost) for gesture classification.
  5. Evaluation: Reporting accuracy, precision, recall, and F1-score for each model, in addition to classification report and confusion matrix.
  6. Conclusion: Summarizing the results and selecting the best-performing model.

2. Model Testing:

• Testing code for the best performing model from Model Training code
• Model is tested using Mediapipe
• Output video showing gesture classification is recorded.

Other Files

• models_evaluation: CSV file that contains evalutation metrics (accuracy, precision, recall, F1-score) of each model.
• encoder: Pickle file that contains label encoding.
• xgb_model: Pickle file that contains the chosen best performing model for classificaion (XGBoost).