Clarification on Intra-frame (I-frame) interval in data preprocessing

[RRooyyCChheenn]

Hi,

I am currently reviewing the data preprocessing pipeline and had a question regarding tools/tools_for_hevc/step3_generate_video_mv_residual_index.py.

Specifically, I noticed that the script does not explicitly define a fixed interval for sampling full frames (Intra-frames). Could you please clarify the following:

Is there a default interval being used (e.g., every 32 or 64 frames)?

Does the script process the video entirely without full-frame sampling, or does it rely on the GOP structure predefined during the HEVC encoding step?

Understanding this sampling frequency is critical for me to align the training data correctly with the model architecture.

Thanks for your help!

[YunyaoYan]

Thanks for the question — this is a key detail of the preprocessing design.

The short answer is that Step 3 does not explicitly sample full frames at a fixed interval (e.g., every 16 or 64 frames). Instead, we first uniformly sample a fixed number of temporal positions (seq_len = 64) across the entire video, and then perform codec-guided patch selection on those sampled positions.

The fixed GOP (default 16) introduced during HEVC transcoding is mainly used to stabilize the compressed-domain signals (motion vectors and residuals), rather than to define the temporal layout of the training sequence.

We recently put together a more detailed explainer with diagrams here:
👉 https://evolvinglmms-lab.github.io/OneVision-Encoder/issues/issue-112.html

It covers:
• the difference between codec-side GOP structure and training-side sampled sequence
• how MV / residual signals are used to compute patch scores
• how Top-K patch indices are generated and used during training

Happy to clarify further if needed!