Quantile Threshold Calculation + Filtering#252

Draft

ajude2s wants to merge 21 commits intomasterfrom

quantile_calculation

Commits on Nov 19, 2025

chore: add notebooks
mali-git
committed

Commits on Jan 20, 2026

feat: Implement paired threshold filtering for JSONL files
ajude2s
committed
feat: Enhance threshold filter pipeline for paired JSONL processing
ajude2s
committed

Commits on Jan 22, 2026

Commits on Jan 23, 2026

refactor: move datatrove pipelines into data_pipelines and centralize execution helpers
AbasKhan
committed

Commits on Jan 24, 2026

feat: Add quantile data and tests for quantile pipeline functionality
AbasKhan
committed