Partial Row Inference via Streaming Mmap (PRISM)

A Linux kernel-level optimization for UVC video streams that reduces ingestion latency by enabling sub-frame delivery. By intercepting the internal uvcvideo data path via kprobes, PRISM allows userspace applications to begin processing as soon as a target row is decoded, bypassing the standard V4L2 "wait-for-completion" bottleneck.

Example Use Case could be in autonomous Driving and many CNN Applications

---

Caution

PRISM is only beneficial when your target object is reliably located within a known, fixed region of the frame.

Sub-frame delivery gives you the top N% of rows early — the rest of the frame hasn't arrived yet. If your object of interest (a ball, a face, a lane marker, an obstacle) could appear anywhere in the frame, you gain nothing — you'd still need to wait for the full frame before running detection.

---

Latency Analysis

In standard V4L2 implementations, frame buffers are only delivered to userspace after the entire frame (100% of rows) has been transferred from the hardware. PRISM overlaps the I/O transfer with computation by triggering an early wake-up the moment a row threshold is reached.

Metric	Standard V4L2	PRISM (60% Crop)
Ingestion Latency	29.4 ms	15.8 ms
Performance Gain	Base	~46% reduction

Note: Benchmarks performed on 640×480 @ 30fps YUYV stream. The fixed USB/UVC overhead (~11ms) means gains are sub-linear with crop percentage.

---

Technical Overview

PRISM utilizes Linux kprobes to hook into the uvcvideo driver's decoding pipeline (uvc_video_decode_data).

1. Kernel Hook: Monitors the incoming USB packet stream for a specific frame threshold (e.g., top 60% of rows).
2. Early Wake-up: Triggers a blocking read() on /dev/subframe the moment the threshold is met.
3. Zero-Copy Access: Exposes the partial frame data via mmap for direct userspace consumption while the remainder of the frame is still being transferred via DMA.

---

Installation, Detection & Benchmarking

1. Build & Load Module

make
sudo insmod subframe_kprobe.ko

2. Verify Hardware Latency

The included hardware benchmarking tool measures the real-world delta between DMA start and sub-frame availability:

g++ -O2 -std=c++17 timing_only.cpp -o timing_bench
sudo ./timing_bench --device /dev/video0

---

3. Detect Camera Shutter Type

Before using PRISM, verify your camera's shutter mechanism to ensure compatibility:

# Build the shutter detection tool
g++ -O2 -std=c++17 detect_shutter.cpp -o detect_shutter \
  `pkg-config --cflags --libs opencv4 2>/dev/null || pkg-config --cflags --libs opencv`

# Run detection (requires live camera stream)
sudo ./detect_shutter --device /dev/video0 --frames 100

Expected Output:

SHUTTER TYPE ANALYSIS
==============================================================
Device: /dev/video0
Frames analyzed: 100

METRICS:
  Row Gradient Variance:      14.32
  Temporal Distortion Score:  18.56
  Motion Artifact Ratio:       7.24

RESULT:
  Shutter Type: ROLLING SHUTTER
  Confidence:   87%
==============================================================

Shutter Type Interpretation:

• ROLLING SHUTTER (>70% confidence): Full compatibility with PRISM. Partial row inference will provide maximum latency reduction.
• GLOBAL SHUTTER (<70% confidence): Limited optimization potential. All rows expose simultaneously, reducing per-row timing benefits.

---

Project Structure

• subframe_kprobe.c: Kernel module implementation.
• timing_only.cpp: High-precision hardware measurement tool.
• read_subframe.c: Reference implementation for kernel event consumption.
• view_subframe.py: Visualization and verification script.
• detect_shutter.cpp: Camera shutter type detector (rolling vs global).

---

Troubleshooting

Camera Not Detected

# Verify camera device exists
ls -la /dev/video*

# Check for rolling shutter capability
v4l2-ctl --list-devices

Kernel Module Won't Load

# Check kernel version compatibility (requires 4.14+)
uname -r

# View module errors
dmesg | tail -20

Shutter Detection Fails

# Ensure adequate motion during detection
# Move camera or object in front of it during capture
sudo ./detect_shutter --device /dev/video0 --frames 150 --motion-threshold 20

Future Works

• Python Package for extracting Partial Row's.

---

⭐ If you find PRISM interesting or useful, consider starring the repo — it really helps visibility and motivates further development!