Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ __pycache__/
.pytest_cache/
*.egg-info/
future.md
instructions.txt
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,9 @@ Supported Python API front-door intents:
- `Chart.trend()`
- `Chart.rank()`
- `Chart.compare()`
- `Chart.histogram()`
- `Chart.boxplot()`
- `Chart.violin()`
- experimental `audit_spec()`

## Scope and Non-Goals
Expand All @@ -205,6 +208,22 @@ Supported Python API front-door intents:

The audit layer uses Tufte-inspired visual integrity checks. It does not claim to be Tufte-compliant or Tufte-certified.

## Distribution Charts

Use distribution intents when the claim is about spread, shape, outliers, or typical values rather than a trend or rank.

- Histograms are good for one metric's distribution.
- Boxplots are good for robust summaries and outliers.
- Violin plots are good for shape, but they need more observations to stay readable.

Run `python examples/distribution_charts.py` to write:

- `examples/output/histogram_chart.vl.json`
- `examples/output/boxplot_chart.vl.json`
- `examples/output/violin_chart.vl.json`

These charts stay inside the same claim-first audit flow and are not a full visualization library.

## Companion Artifact

This repo was built using `ai-engineering-skills` and is intended as the analytical-integrity proof artifact companion to `context-to-action-skills`.
Expand Down
6 changes: 6 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ Explicit non-goals:
- Do not add auto-correction.
- Do not build a dashboard or chart generator.

## v0.3 distribution preview

- Chart intents: `Chart.histogram()`, `Chart.boxplot()`, `Chart.violin()`
- Distribution audit rules: numeric metric checks, sample-size thresholds, grouped category thresholds, histogram bins, violin density warnings
- Example artifact: `examples/distribution_charts.py`

## Later

- More visual intents only after the audit contract is strong
Expand Down
60 changes: 27 additions & 33 deletions artifacts/HANDOFF.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,50 +2,44 @@

RESUME PACKET

- Goal: keep `chart-contract` stable at v0.1 while building out the v0.2 agent gate in small, auditable slices.
- Workflow State: the CLI docs now mention `--fail-on READY|REVIEW|BLOCK` in addition to `--warnings-as-errors`, and the wording stays careful about `READY` as an explicit threshold.
- Branch: `main`
- Next task: commit the docs cleanup and push it if the branch looks good.
- Verification: `./.venv/bin/python -m pytest`, `./.venv/bin/chart-contract --version`, and `git diff --check`
- Read first: `README.md`, `docs/AGENT_INTEGRATION.md`, `artifacts/VERIFY.md`
- Goal: add first-class distribution intents without breaking the existing trend/rank/compare contract or CLI behavior.
- Workflow State: `Chart.histogram()`, `Chart.boxplot()`, `Chart.violin()`, distribution renderers, distribution audit rules, tests, docs, and `examples/distribution_charts.py` are all in place.
- Branch: `codex/distribution-chart-intents`
- Next task: review the diff, then stage/commit or keep iterating if a follow-up is needed.
- Verification: `./.venv/bin/python -m pytest -q`, `./.venv/bin/python examples/distribution_charts.py`, `./.venv/bin/python examples/bad_to_good_chart.py`, `./.venv/bin/chart-contract --help`, and `git diff --check`
- Read first: `README.md`, `ROADMAP.md`, `docs/AUDIT_RULES.md`, `artifacts/VERIFY.md`

## Current Repo State

- The v0.1 scope in `artifacts/SPEC.md` remains unchanged.
- The v0.2.0 roadmap is now explicit and commit-shaped.
- The package version is bumped to 0.2.0 and the changelog calls out the shipped gate surface.
- Report serialization is hardened for CLI use, and the CLI now loads specs/data from disk, emits multiple report formats, and writes file outputs when requested.
- The v0.2 trap fixtures are file-based and runnable from the CLI, with separate spec, data, and claim files for easy inspection.
- The README now advertises the CLI audit gate, and CI runs a trap smoke check in addition to pytest.
- The trend-spec audit now matches the chart-level trend rule for simple line specs, so the single-point trap is BLOCK again.
- The CI workflow now explicitly checks that BLOCK exits 1 and that `--warnings-as-errors` turns REVIEW into a nonzero exit.
- `report.verdict` is now spelled out in the README as the authoritative gate field, and `report.passed` is documented as a no-FAIL signal.
- The `--fail-on` flag is now briefly documented so users do not have to discover it only through `--help`.
- The v0.1 scope in `artifacts/SPEC.md` remains intact, and the new distribution intents live in a separate v0.3 preview section.
- The v0.2.0 agent gate is still intact: CLI loading, report serialization, trap fixtures, and CI smoke checks remain in place.
- Distribution intents now exist as `Chart.histogram()`, `Chart.boxplot()`, and `Chart.violin()` with matching Altair renderers.
- `audit_chart()` and `audit_spec()` now enforce distribution-specific checks for numeric metrics, sample sizes, grouped categories, histogram bins, and violin density warnings.
- `examples/distribution_charts.py` writes histogram, boxplot, and violin Vega-Lite specs into `examples/output/`.
- README, `docs/AUDIT_RULES.md`, `ROADMAP.md`, and `artifacts/SPEC.md` now describe the distribution preview and keep the claim-first framing explicit.
- The repo still has tracked example-output drift whenever the example scripts are run, so those artifacts need review before any commit.

## Working Commands

- `git diff --check`
- `./.venv/bin/python -m pytest tests/test_cli.py`
- `./.venv/bin/python -m pytest`
- `./.venv/bin/python -m pytest tests/test_distribution_intents.py -q`
- `./.venv/bin/python -m pytest tests/test_audit_spec.py -q`
- `./.venv/bin/python -m pytest tests/test_chart_intents.py -q`
- `./.venv/bin/python -m pytest -q`
- `./.venv/bin/python examples/distribution_charts.py`
- `./.venv/bin/python examples/bad_to_good_chart.py`
- `./.venv/bin/chart-contract --help`
- `sed -n '1,260p' ROADMAP.md`
- `sed -n '1,220p' docs/AGENT_INTEGRATION.md`
- `sed -n '1,220p' docs/AUDIT_RULES.md`

## Important Decisions

- Keep v0.2.0 concrete enough that each slice can become a commit.
- Preserve the v0.1 and later-roadmap context while avoiding v0.3 drift.
- Do not add ChartContract, auto-correction, or extra chart intents in this slice.
- Use `schema_version: "0.2"` in serialization output as the CLI-facing contract anchor.
- Use verdict names for `--fail-on` so the future exit-code mapping stays aligned with `AuditReport.verdict`.
- Keep stdout text as the default surface unless `--out` switches it to a one-line verdict summary.
- Keep trap fixtures tiny and synthetic so they stay inspectable and easy to copy-paste.
- Keep the CI smoke check on a REVIEW fixture so it stays a passing gate while still exercising the CLI front door.
- Keep the release notes aligned with the actual CLI behavior so `0.2.0` stays a truthful cut.
- Mirror chart-level trend completeness checks in spec-audit code whenever line specs are treated as trend-like.
- When adding CI smoke checks, assert the exit code explicitly instead of relying on output alone.
- Keep docs wording version-neutral when the release version has already advanced.
- Keep public CLI flags like `--fail-on` documented in the agent guidance when they are exposed in `--help`.
- Keep the distribution slice claim-first and deterministic; avoid turning the package into a full visualization library.
- Keep v0.1 scope in `artifacts/SPEC.md` stable while using a separate preview section for new intents.
- Keep the existing trend/rank/compare API and CLI behavior unchanged.
- Keep the distribution audits focused on numeric metrics, sample size, grouping, bins, and violin density warnings.
- Keep example outputs synthetic and inspectable so they stay easy to review.

## Next Recommended Task

Use the roadmap and agent-integration docs as the source of truth for the next implementation slice.
Use the roadmap, SPEC, and audit-rules docs as the source of truth for any follow-up slice.
6 changes: 6 additions & 0 deletions artifacts/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ Build `chart-contract`, a lightweight Python harness for claim-first, audited an
- deterministic PASS/WARN/FAIL findings
- docs, tests, examples, and build-proof artifacts

## v0.3 Preview

- `Chart.histogram()`, `Chart.boxplot()`, `Chart.violin()`
- distribution-specific audit rules for numeric value fields, sample size, grouped categories, histogram bins, and violin density warnings
- `examples/distribution_charts.py`

## Non-Goals

- UI, dashboards, or Streamlit
Expand Down
39 changes: 39 additions & 0 deletions artifacts/VERIFY.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,44 @@
# VERIFY

2026-07-03 - Add distribution chart intents

Environment:
- Working directory: repo root

Commands:
- `./.venv/bin/python -m pytest tests/test_distribution_intents.py -q` -> PASSED (`11 passed`)
- `./.venv/bin/python -m pytest tests/test_audit_spec.py -q` -> PASSED (`5 passed`)
- `./.venv/bin/python -m pytest tests/test_chart_intents.py -q` -> PASSED (`4 passed`)
- `./.venv/bin/python -m pytest -q` -> PASSED (`52 passed`)
- `./.venv/bin/python examples/distribution_charts.py` -> PASSED
- `./.venv/bin/python examples/bad_to_good_chart.py` -> PASSED
- `./.venv/bin/chart-contract --help` -> PASSED
- `git diff --check` -> PASSED

Changed files:
- `src/chart_contract/chart.py`
- `src/chart_contract/renderers/altair.py`
- `src/chart_contract/audit.py`
- `tests/test_distribution_intents.py`
- `tests/test_audit_spec.py`
- `tests/test_chart_intents.py`
- `examples/distribution_charts.py`
- `examples/output/histogram_chart.vl.json`
- `examples/output/boxplot_chart.vl.json`
- `examples/output/violin_chart.vl.json`
- `README.md`
- `docs/AUDIT_RULES.md`
- `ROADMAP.md`
- `artifacts/SPEC.md`
- `artifacts/HANDOFF.md`

Remaining risks:
- The example scripts rewrite tracked output artifacts, so the working tree will still show those diffs until they are reviewed or intentionally committed.
- The distribution preview is intentionally separate from the v0.1 scope, so any follow-up work should keep that boundary explicit.

Next safest task:
- Review the generated output artifacts and decide whether to commit or keep iterating on the distribution preview.

2026-06-22 - Document `--fail-on` in CLI guidance

Environment:
Expand Down
5 changes: 5 additions & 0 deletions docs/AUDIT_RULES.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,11 @@ Audits catch common analytical and visual-integrity failure modes. They do not p
| `data.not_empty` | Chart audits | `PASS` when data has rows; `WARN` when the dataset is empty. | Checks that the chart has any observations to render. | Provide data with at least one row. |
| `data.trend.min_points` | Trend chart audits | `PASS` when the trend has at least two rows; `FAIL` when it has fewer than two. | Ensures a directional trend has more than one observation. | Add historical data covering at least two time periods. |
| `data.trend.x.ordered` | Trend chart audits | `PASS` when x is ordered or datetime-like; `WARN` otherwise. | Checks that the trend axis can be read as a progression. | Use a time field, numeric sequence, or ordered categorical series. |
| `data.distribution.value.numeric` | Distribution chart audits | `PASS` when the distribution metric exists and is numeric; `FAIL` when it is missing or non-numeric. | Ensures histogram, boxplot, and violin claims use a numeric measure. | Add or convert the metric column to numeric. |
| `data.distribution.sample_size` | Distribution chart audits | `PASS` when the chart has at least 20 rows; `WARN` for 5-19; `FAIL` below 5. | Checks whether the sample is large enough to summarize shape or spread. | Collect more observations before summarizing the distribution. |
| `data.distribution.group_sample_size` | Distribution chart audits with categories | `PASS` when each non-null group has at least 10 rows; `WARN` when any group is smaller. | Checks whether grouped distributions have enough rows per category. | Aggregate small groups or collect more data for each group. |
| `readability.histogram.bins` | Histogram audits | `PASS` when bins are default/custom-controlled or an integer between 5 and 50; `WARN` when an integer falls outside that range. | Checks whether the histogram bin count stays readable. | Use a bin count between 5 and 50 unless the claim needs a custom setting. |
| `visual.violin.sample_size` | Violin audits | `PASS` when the chart has at least 30 rows; `WARN` below 30. | Checks whether a violin plot has enough rows to justify the density view. | Use a boxplot or strip/point summary when the sample is small. |
| `readability.rank.category_count` | Rank chart audits | `PASS` when categories are at or below the limit; `WARN` when there are too many. | Checks whether a rank chart stays readable. | Reduce categories or aggregate the long tail. |
| `readability.color.category_count` | Chart audits with grouped color encodings | `PASS` when group count is at or below the limit; `WARN` when there are too many. | Checks whether color encoding remains readable. | Reduce groups or facet the comparison. |
| `claim.causal_support` | Chart audits and spec audits | `PASS` when the claim is non-causal or justified; `WARN` when causal language lacks caveat/evidence. | Checks whether the claim overreaches causally. | Add a caveat or causal evidence metadata when justified. |
Expand Down
102 changes: 102 additions & 0 deletions examples/distribution_charts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
"""Distribution chart demo: histogram, boxplot, and violin."""

from __future__ import annotations

import json
from pathlib import Path

import pandas as pd

from chart_contract import Chart

OUTPUT_DIR = Path(__file__).resolve().parent / "output"


def _write_json(path: Path, payload: dict) -> None:
path.write_text(json.dumps(payload, indent=2) + "\n")


def _audit_and_write(label: str, chart: Chart, output_path: Path) -> None:
report = chart.audit()
print(f"{label}: {report.verdict_summary()}")
_write_json(output_path, chart.to_vega_lite())
print(f"Wrote {output_path}")


def main() -> None:
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)

df = pd.DataFrame(
{
"segment": ["SMB"] * 15 + ["Enterprise"] * 15,
"amount": [
4,
5,
6,
7,
5,
4,
6,
7,
8,
6,
5,
7,
6,
4,
5,
12,
13,
15,
14,
13,
12,
16,
14,
15,
13,
12,
14,
15,
13,
12,
],
}
)

histogram = Chart.histogram(
data=df,
value="amount",
claim="The amount values are spread across the observed range.",
source="synthetic.amounts",
unit="count",
title="Amount distribution",
bins=12,
)
_audit_and_write("Histogram", histogram, OUTPUT_DIR / "histogram_chart.vl.json")

boxplot = Chart.boxplot(
data=df,
x="segment",
y="amount",
claim="SMB and Enterprise have different amount summaries.",
source="synthetic.amounts",
unit="count",
title="Amount by segment",
)
_audit_and_write("Boxplot", boxplot, OUTPUT_DIR / "boxplot_chart.vl.json")

violin = Chart.violin(
data=df,
x="segment",
y="amount",
claim="SMB and Enterprise have different amount shapes.",
source="synthetic.amounts",
unit="count",
title="Amount density by segment",
)
_audit_and_write("Violin", violin, OUTPUT_DIR / "violin_chart.vl.json")


if __name__ == "__main__":
main()
Loading
Loading