Standardize Evidence Artifacts and Schema

[FScholPer]

Goal

Standardize the artifact set emitted by all onboarded repositories.

Why

Developers and governance tooling need consistent machine-readable evidence across repos.

Required Artifact Contract

Base fields (all repos)

• execution_lane (oss_default | accelerated)
• policy_decision (allow | deny, with machine-readable reason)
• test_summary
• quality_or_traceability_metrics
• dependency_manifest_summary
• license_policy_result
• reproducibility_metadata (rerun id, tool versions, environment hash)

Extended fields for docs-as-code / assurance domain

• impacted_elements — list of need IDs or argument element IDs directly impacted by the change
• impact_class per element — one of: direct_recheck | indirect_propagation | revision_required
• consistency_rule_ids — which rules triggered the impact classification
• affected_artifact_type — requirement | guideline | test_link | process_requirement | evidence
• change_scenario_id — reference to the specific change scenario class that was matched
• responsible_role — who is accountable for the outcome (default: pr_creator)
• escalation_role — who to escalate failures to (default: harness_maintainer)
• waiver_authority — who can approve waivers (default: release_approver)

Provenance fields (all repos):

• execution_timestamp — ISO 8601 timestamp
• python_version — interpreter version
• environment_hash — stable hash of execution environment
• gate_script_version — validator version

These extended fields enable the harness outer loop to grep structured traces and identify recurring failure patterns without reading raw sphinx-build output.

Queryability Rules

• Artifacts must be machine-readable (json, jsonl, or schema-validated yaml)
• Filenames must be stable and consistent across runs so simple tools such as grep and cat work reliably
• Raw stdout/stderr may be retained as optional debug artifacts, but never as the primary evidence contract
• A small summary index (evolution_summary.jsonl or equivalent) must exist so agents start from the index before drilling into traces
• The docs-as-code pilot should standardize at least gate_output.json, impacted_elements.json, and score.json per task

Tasks

• Define artifact manifest and schema versions.
• Reuse/align with existing docs-as-code schema patterns.
• Validate schema compatibility in CI.
• Document retention and discoverability.
• Define compatibility policy for schema evolution.
• Define canonical filenames and directory layout for per-run and per-task trace artifacts.

Docs-as-code pilot tasks

• Extend current metrics artifacts with required provenance fields.
• Validate artifact schemas in CI before policy decision.
• Ensure accelerated-lane artifacts remain verifiable by OSS validators.
• Add evolution_summary.jsonl or equivalent summary index for selective navigation of prior runs.
• Document the canonical per-task trace layout under runs/<iteration>/<candidate>/traces/<task_id>/.

Done When

• Pilot repos emit the same artifact contract.
• CI rejects malformed or incomplete artifact payloads.
• Schema version policy is documented and enforced.
• Agents can locate the relevant trace files for one failure class without reading raw command output dumps.

Parent: #2852