|
| 1 | +--- |
| 2 | +import Layout from "../layouts/Layout.astro"; |
| 3 | +--- |
| 4 | +<Layout |
| 5 | + title="Notebook Research Workflow • OpenQuant-rs" |
| 6 | + description="Notebook-first research workflow for OpenQuant-rs with reproducibility and leakage controls." |
| 7 | +> |
| 8 | + <section class="hero"> |
| 9 | + <h1>Notebook-First Research Workflow</h1> |
| 10 | + <p class="muted"> |
| 11 | + Promotion path from exploratory notebook to reproducible candidate strategy, |
| 12 | + aligned with <code>docs/research_workflow.md</code>. |
| 13 | + </p> |
| 14 | + </section> |
| 15 | + |
| 16 | + <h2>Promotion Path</h2> |
| 17 | + <ol> |
| 18 | + <li>Exploratory notebook in <code>notebooks/python/*.ipynb</code>.</li> |
| 19 | + <li> |
| 20 | + Scripted experiment in <code>experiments/run_pipeline.py</code> with a TOML |
| 21 | + config. |
| 22 | + </li> |
| 23 | + <li> |
| 24 | + Artifact and parity checks in <code>python/tests/test_experiment_scaffold.py</code>. |
| 25 | + </li> |
| 26 | + <li> |
| 27 | + Candidate decision recorded as <code>decision.md</code> in the run artifact |
| 28 | + directory. |
| 29 | + </li> |
| 30 | + </ol> |
| 31 | + |
| 32 | + <h2>Local Commands</h2> |
| 33 | + <pre><code class="language-bash"># setup |
| 34 | +uv venv --python 3.11 .venv |
| 35 | +uv sync --group dev |
| 36 | +uv run --python .venv/bin/python maturin develop --manifest-path crates/pyopenquant/Cargo.toml |
| 37 | + |
| 38 | +# notebook logic smoke |
| 39 | +uv run --python .venv/bin/python python notebooks/python/scripts/smoke_all.py |
| 40 | + |
| 41 | +# reproducible experiment run |
| 42 | +uv run --python .venv/bin/python python experiments/run_pipeline.py --config experiments/configs/futures_oil_baseline.toml --out experiments/artifacts |
| 43 | + |
| 44 | +# parity + scaffold tests |
| 45 | +uv run --python .venv/bin/python pytest python/tests/test_experiment_scaffold.py -q</code></pre> |
| 46 | + |
| 47 | + <h2>Leakage and Reproducibility Checklist</h2> |
| 48 | + <ul> |
| 49 | + <li>Use event-based sampling and leakage-safe validation paths.</li> |
| 50 | + <li>Keep deterministic seeds and config-hashed run directories.</li> |
| 51 | + <li>Report gross and net metrics with turnover and cost estimates.</li> |
| 52 | + <li>Require statistical and economic gates before promotion.</li> |
| 53 | + <li> |
| 54 | + Confirm leakage checks pass: <code>inputs_aligned</code> and |
| 55 | + <code>event_indices_sorted</code>. |
| 56 | + </li> |
| 57 | + <li> |
| 58 | + Artifact bundle includes <code>manifest.json</code>, parquet outputs, and |
| 59 | + <code>decision.md</code>. |
| 60 | + </li> |
| 61 | + </ul> |
| 62 | + |
| 63 | + <h2>Anti-Patterns</h2> |
| 64 | + <ul> |
| 65 | + <li>Random CV splits on overlapping financial labels.</li> |
| 66 | + <li>Promoting on gross-only metrics without cost accounting.</li> |
| 67 | + <li>Notebook-only logic that cannot be reproduced from config.</li> |
| 68 | + <li>Hidden mutable state across notebook cells.</li> |
| 69 | + </ul> |
| 70 | +</Layout> |
0 commit comments