AI-Powered Video Proctoring System

A sophisticated real-time video proctoring system that monitors candidates during online interviews using advanced AI technologies. The system detects focus levels, unauthorized objects, and generates comprehensive integrity reports.

Features

Core Functionality

• Real-Time Face Detection - Powered by MediaPipe's BlazeFace model
• Focus Tracking - Monitors candidate attention and alerts when focus is lost for >5 seconds
• Absence Detection - Flags when no face is detected for >10 seconds
• Multiple Face Detection - Identifies when more than one person is in frame
• Object Detection - Uses TensorFlow.js COCO-SSD to detect unauthorized items:
  • Mobile phones
  • Books and notes
  • Extra electronic devices (keyboards, monitors, laptops)
  • Other suspicious objects

Reporting & Analytics

• Live Event Logging - Real-time tracking of all violations with timestamps
• Integrity Scoring - Comprehensive scoring system (0-100) based on violations
• Detailed Reports - Exportable proctoring reports with complete session analytics
• Visual Indicators - Color-coded status overlays showing current monitoring state

UX

• Clean Interface - Modern, intuitive design with professional aesthetics
• Real-Time Status - Visual feedback showing recording status and AI model loading state
• Session Management - Easy start/stop controls with candidate information tracking
• Downloadable Reports - Export detailed text reports for record-keeping

Tech Stack

Frontend

• React 18.3.1 - Modern component-based UI
• TypeScript 5.5.3 - Type-safe development
• Vite 5.4.2 - Lightning-fast build tool

AI/ML

• MediaPipe Tasks Vision - Face detection and tracking
• TensorFlow.js 4.22.0 - In-browser machine learning
• COCO-SSD Model - Object detection

Styling & UI

• Tailwind CSS 3.4.1 - Utility-first CSS framework
• Lucide React - Beautiful icon system

Backend (Ready for Integration)

• Supabase - Database and authentication ready

Prerequisites

Before running this project, ensure you have:

• Node.js (v18 or higher)
• npm or yarn
• Webcam access for video monitoring
• Modern browser with WebRTC support (Chrome, Firefox, Edge)

Installation

1. Clone the repository

   git clone https://github.com/yourusername/video-proctoring-system.git
   cd video-proctoring-system

2. Install dependencies

   npm install

3. Start the development server

   npm run dev

4. Open your browser Navigate to http://localhost:5173

Usage

Starting an Interview Session

1. Enter Candidate Information

   • Input the candidate's name on the start screen
   • Click "Start Interview Session"

2. Grant Camera Permissions

   • Allow browser access to your webcam when prompted
   • Wait for AI models to load (indicated by loading spinner)

3. Monitor the Session

   • The system will automatically start monitoring once recording begins
   • Real-time events appear in the Event Log panel
   • Status indicators show current monitoring state

4. End the Session

   • Click "Stop Session" to end monitoring
   • View the comprehensive proctoring report
   • Download the report for record-keeping

Understanding the Integrity Score

The system uses a deduction-based scoring system starting from 100:

Violation Type	Deduction	Description
Focus Lost	-2 points	Looking away for >5 seconds
No Face	-5 points	Face not visible for >10 seconds
Multiple Faces	-10 points	More than one person in frame
Suspicious Object	-15 points	Unauthorized items detected

Score Interpretation:

• 90-100: Excellent - High integrity
• 80-89: Good - Minor violations
• 70-79: Fair - Moderate concerns
• 60-69: Poor - Significant violations
• <60: Very Poor - Multiple serious violations

Detection Logic

Focus Detection Algorithm

// Focus is monitored every second
if (noFaceDetected) {
  timer++;
  if (timer >= 10) {
    logEvent('no_face', severity: 'high');
  } else if (timer >= 5) {
    logEvent('focus_lost', severity: 'medium');
  }
}

Object Detection Frequency

• Face detection: Every 1 second
• Object detection: Every 3 seconds (to optimize performance)
• Duplicate prevention: 5-second cooldown per object type

Project Structure

video-proctoring-system/
├── src/
│   ├── components/
│   │   ├── InterviewSession.tsx    # Main session container
│   │   ├── VideoMonitor.tsx        # Video feed with AI detection
│   │   ├── EventLog.tsx            # Real-time event display
│   │   └── ProctoringReport.tsx    # Detailed report view
│   ├── hooks/
│   │   ├── useFaceDetection.ts     # MediaPipe face detection
│   │   └── useObjectDetection.ts   # TensorFlow object detection
│   ├── utils/
│   │   └── detectionLogic.ts       # Scoring & event utilities
│   ├── types.ts                    # TypeScript interfaces
│   ├── App.tsx                     # Root component
│   └── main.tsx                    # Entry point
├── public/                         # Static assets
├── dist/                           # Production build
└── package.json                    # Dependencies

Configuration

Vite Configuration

The system uses custom headers for SharedArrayBuffer support:

// vite.config.ts
server: {
  headers: {
    'Cross-Origin-Embedder-Policy': 'require-corp',
    'Cross-Origin-Opener-Policy': 'same-origin',
  },
}

Detection Thresholds

Adjust thresholds in src/components/VideoMonitor.tsx:

const NO_FACE_THRESHOLD = 10;  // seconds
const FOCUS_LOST_THRESHOLD = 5; // seconds
const OBJECT_CHECK_INTERVAL = 3000; // milliseconds

Customization

Adding New Suspicious Objects

Edit src/utils/detectionLogic.ts:

export const SUSPICIOUS_OBJECTS = [
  'cell phone',
  'book',
  'laptop',
  'keyboard',
  // Add your objects here
];

Modifying Scoring System

Update calculateIntegrityScore in src/utils/detectionLogic.ts:

export const calculateIntegrityScore = (stats: DetectionStats): number => {
  let score = 100;
  score -= stats.focusLostCount * 2;    // Adjust penalty
  score -= stats.noFaceCount * 5;       // Adjust penalty
  score -= stats.multipleFacesCount * 10; // Adjust penalty
  score -= stats.suspiciousObjectCount * 15; // Adjust penalty
  return Math.max(0, Math.min(100, score));
};

Building for Production

npm run build

The optimized production build will be in the dist/ directory.

Preview Production Build

npm run preview

Testing

Type Checking

npm run typecheck

Linting

npm run lint

Privacy & Security

• Local Processing: All AI computations happen in the browser
• No Cloud Uploads: Video streams are not uploaded to external servers
• Session Storage: Data is stored locally in browser storage
• Camera Access: Used only during active sessions
• Data Persistence: Ready for Supabase integration for secure storage

Future Enhancements

Planned Features

• Eye closure/drowsiness detection
• Audio detection for background voices
• Real-time alerts for interviewers
• Multi-language support
• Advanced analytics dashboard
• Video recording with playback
• Automated report email delivery
• Integration with popular interview platforms

Database Integration

The system is ready for Supabase integration:

// Example schema structure
interface Session {
  id: string;
  candidate_name: string;
  start_time: timestamp;
  end_time: timestamp;
  integrity_score: number;
  events: ProctoringEvent[];
}

Troubleshooting

Camera Not Working

1. Ensure camera permissions are granted in browser settings
2. Check if another application is using the camera
3. Try a different browser (Chrome recommended)
4. Verify camera is properly connected

AI Models Not Loading

1. Check internet connection (models load from CDN)
2. Clear browser cache
3. Disable ad blockers that might block CDN requests
4. Check browser console for specific errors

Performance Issues

1. Close unnecessary browser tabs
2. Ensure adequate system resources (RAM, CPU)
3. Reduce video resolution in VideoMonitor.tsx:

   video: { width: 640, height: 480 } // Lower resolution

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Acknowledgments

• MediaPipe - Google's ML solutions for face detection
• TensorFlow.js - Browser-based machine learning
• COCO-SSD - Pre-trained object detection model
• React Community - For the amazing ecosystem

Project Status

---

Note: This system is designed for educational and professional interview monitoring. Always ensure compliance with local privacy laws and obtain proper consent from candidates before use.