[v2.0.0] Phase 4: Deterministic Visual Validation Artifacts

[Anselmoo]

Goal

Add deterministic scientific evidence artifacts for core CLI/API validation so fitting accuracy can be reviewed visually as PNG exports — not only numerically.

Depends On

• [v2.0.0] Phase 1: Codebase Readiness Assessment #2084 (Phase 1: Codebase Readiness Assessment)
• [v2.0.0] Phase 2: Stabilize CI Baseline #2085 (Phase 2: Stabilize CI Baseline)
• [v2.0.0] Phase 3: Curate v2 Example and Input File Story #2086 (Phase 3: Curate v2 Example and Input File Story)

Background

The community needs reliable, sophisticated fitting results that can be inspected visually. Currently, validation is numeric only. Artifact-based visual regression allows maintainers and community members to judge fit quality and detect regressions from CI run artifacts — without needing to re-run locally.

spectrafit/generators/synthetic.py provides SyntheticSpectrum and PeakDefinition for deterministic scenario construction. spectrafit/plotting.py provides the current plotting surface. Determinism requires pinned seed, style, DPI, backend, and filename conventions.

Tasks

• TASK-013 — Define canonical validation scenarios using SyntheticSpectrum and PeakDefinition for at least three cases:
  1. Single-peak recovery (Gaussian, known parameters)
  2. Multi-peak recovery (overlapping Gaussian + Lorentzian)
  3. Representative global-fit scenario (shared parameter across two spectra)
• TASK-014 — Implement deterministic plot artifact generation for core CLI/API validation runs by extending spectrafit/plotting.py or dedicated test utilities; each scenario must emit predictable PNG outputs (pinned seed, matplotlib Agg backend, fixed DPI=150, fixed filename pattern validation_{scenario}_{timestamp_or_fixed}.png)
• TASK-015 — Add regression tests for visual outputs in tests/validation/test_visual_regression.py; use pytest-plt where appropriate; keep baselines stable through pinned style, seed, DPI, and filename conventions
• TASK-016 — Extend .github/workflows/python-ci.yml validation job to upload PNG artifacts alongside validation-report.json and validation-report.html; document how maintainers inspect failures

Acceptance Criteria

• Validation runs generate deterministic PNG artifacts locally via uv run pytest tests/validation/ -m validation
• CI uploads visual artifacts corresponding to numeric validation results
• The repository has a defined acceptance rule for visual-regression review (documented in docs/ or tests/validation/README.md)

Determinism Requirements

Setting	Value
NumPy seed	fixed per scenario (e.g., 42)
Matplotlib backend	Agg
DPI	150
Style	seaborn-v0_8-whitegrid or pinned style sheet
Filename	validation_{scenario_id}.png

Scope Constraint

First wave: core CLI/API validation only. Visual artifacts complement numeric validation; they do not replace it.

[Anselmoo]

⏳ Not yet started

Visual validation PNG artifacts (#2087) were not addressed in the current refactoring session (#2094). The session focused on API/architecture modernisation rather than generating deterministic test output images.

Leaving open for a future sprint.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.