Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/ci/runtime_intelligence_gitlab_artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,12 @@ producer-lineage guard-alignment run IDs. This keeps the cross-repo marker
check file-based and does not make AIGuard a deployment decision owner.

The artifact gate is implemented by `scripts/check_runtime_intelligence_artifact_bundle.py`. It checks the generated Markdown / HTML report for the required Runtime Intelligence rows, including Lab ownership, EdgeEnv comparability, telemetry coverage-gap markers, Runtime replay duration scope with `source=entrypoint_requested_frames` traceability, Orchestrator operation feed context, compact queue/deadline/fallback operation markers with `max_total_queue_depth`, Orchestrator task event rollup, Lab EdgeEnv preservation context, Jetson/device-local preservation identity and detail labels, Orchestrator `operation_risk_summary` navigation context, AIGuard runtime operation anomalies, AIGuard `edgeenv_orchestrator_operation_risk_summary` evidence, AIGuard `edgeenv_orchestrator_task_event_rollup` evidence, remote dispatch starter event summary, `Remote fallback starter evidence`, `edgeenv_orchestrator_producer_lineage`, `runtime_history_seed_run_config_traceability`, `remote_execution_recovered_by_fallback`, and triggered deployment review rules.

The bundle manifest gate also checks the external AIGuard artifact before the
rendered report stage. In particular, `runtime_queue_overload` must preserve
`orchestrator_candidate_operation_max_total_queue_depth=7` in
`raw_context.edgeenv_regression`, so the Lab `max_total_queue_depth` report row
can be traced back to Orchestrator producer-side operation context.
When that report gate passes, its summary now emits a
`Validated Duration Traceability` section with `duration_handoff_alignment`,
`duration_source: source=entrypoint_requested_frames`,
Expand Down
5 changes: 5 additions & 0 deletions docs/portfolio/edgeenv_runtime_regression_lab_handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,11 @@ Expected Lab behavior:
- When EdgeEnv preservation context is present, Lab renders `Lab EdgeEnv preservation context` with `lab_report_preservation_context_present=True`, `lab_preservation=present`, and `lab_context=present` so the Runtime Intelligence report gate and entrypoint evidence index use the same Lab-owned marker vocabulary.
- When an EdgeEnv handoff manifest is provided, the bundle gate requires EdgeEnv-produced file keys, external AIGuard file keys, source repository mapping, artifact roles, producer contracts, and boundary flags to match Lab's Runtime Intelligence bundle contract.
- The same manifest gate requires `expected_report_markers` to preserve these exact Lab-owned report markers: `Runtime Intelligence Risk Summary`, `Runtime replay duration scope`, `Orchestrator operation feed context`, `Orchestrator task event rollup`, `Lab EdgeEnv preservation context`, `AIGuard task event rollup evidence`, `AIGuard runtime operation anomalies`, `AIGuard remote dispatch event summary`, `AIGuard remote event summary consistency`, `Remote fallback starter evidence`, `lab=Remote fallback starter evidence; evidence=remote_execution_recovered_by_fallback`, `AIGuard producer-lineage guard alignment`, and `Lab remains the final deployment decision owner.`.
- The external AIGuard artifact gate also requires `runtime_queue_overload`
raw context to preserve
`orchestrator_candidate_operation_max_total_queue_depth=7`, keeping the
rendered Lab `max_total_queue_depth` row traceable to Orchestrator's
producer-side operation context.
- The same gate summary emits `expected_report_markers: remote fallback Lab context row declared`, keeping the remote fallback row-value marker visible in file-based CI artifacts without making CI a runtime control plane.
- The same handoff gate verifies that the referenced `runtime_telemetry_history` artifact exists and preserves EdgeEnv history schema, telemetry coverage, and Runtime history seed ownership markers.
- The same handoff gate verifies that missing telemetry entries remain evidence gaps while preserving Orchestrator producer markers, owner boundary flags, and EdgeEnv mapping hints when Orchestrator context is attached.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@
"candidate_throttling_detected": true,
"baseline_queue_depth": null,
"candidate_queue_depth": 7.0,
"orchestrator_candidate_operation_max_total_queue_depth": 7.0,
"evidence_gap_count": 1.0,
"evidence_gaps": [],
"history_missing_orchestrator_context_count": 1.0,
Expand Down Expand Up @@ -508,6 +509,7 @@
"candidate_throttling_detected": true,
"baseline_queue_depth": null,
"candidate_queue_depth": 7.0,
"orchestrator_candidate_operation_max_total_queue_depth": 7.0,
"evidence_gap_count": 1.0,
"evidence_gaps": [],
"history_missing_orchestrator_context_count": 1.0,
Expand Down Expand Up @@ -1226,6 +1228,7 @@
"candidate_throttling_detected": true,
"baseline_queue_depth": null,
"candidate_queue_depth": 7.0,
"orchestrator_candidate_operation_max_total_queue_depth": 7.0,
"evidence_gap_count": 1.0,
"evidence_gaps": []
},
Expand Down Expand Up @@ -1544,6 +1547,7 @@
"candidate_throttling_detected": true,
"baseline_queue_depth": null,
"candidate_queue_depth": 7.0,
"orchestrator_candidate_operation_max_total_queue_depth": 7.0,
"evidence_gap_count": 1.0,
"evidence_gaps": [],
"history_missing_orchestrator_context_count": 1.0,
Expand Down Expand Up @@ -1844,6 +1848,7 @@
"candidate_throttling_detected": true,
"baseline_queue_depth": null,
"candidate_queue_depth": 7.0,
"orchestrator_candidate_operation_max_total_queue_depth": 7.0,
"evidence_gap_count": 1.0,
"evidence_gaps": [],
"history_missing_orchestrator_context_count": 1.0,
Expand Down Expand Up @@ -2272,6 +2277,7 @@
"candidate_throttling_detected": true,
"baseline_queue_depth": null,
"candidate_queue_depth": 7.0,
"orchestrator_candidate_operation_max_total_queue_depth": 7.0,
"evidence_gap_count": 1.0,
"evidence_gaps": [],
"history_missing_orchestrator_context_count": 1.0,
Expand Down Expand Up @@ -2574,6 +2580,7 @@
"queue_depth": 7,
"operation": {
"queue_depth": 7,
"max_total_queue_depth": 7,
"deadline_missed_count": 2,
"fallback_count": 1
},
Expand Down
25 changes: 25 additions & 0 deletions scripts/check_runtime_intelligence_bundle_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@
"aiguard_raw_context: downstream_guard_alignment preserved",
"aiguard_raw_context: producer_lineage_guard_alignment preserved",
"aiguard_raw_context: missing_telemetry_orchestrator_context preserved",
"aiguard_raw_context: max_total_queue_depth traceability preserved",
"aiguard_handoff_alignment: external required evidence types satisfied",
"expected_report_markers: Runtime Intelligence report markers declared",
"expected_report_markers: remote fallback Lab context row declared",
Expand Down Expand Up @@ -1653,6 +1654,8 @@ def _validate_guard_analysis(guard_analysis: dict[str, Any], errors: list[str])
_validate_run_config_traceability_evidence(item, index, errors)
if item.get("type") == "remote_execution_recovered_by_fallback":
_validate_remote_runtime_event_summary_evidence(item, index, errors)
if item.get("type") == "runtime_queue_overload":
_validate_queue_overload_evidence(item, index, errors)


def _validate_external_aiguard_evidence_alignment(
Expand Down Expand Up @@ -2244,6 +2247,28 @@ def _validate_coverage_gap_evidence(
_validate_aiguard_missing_orchestrator_context(edgeenv, errors)


def _validate_queue_overload_evidence(
item: dict[str, Any],
index: int,
errors: list[str],
) -> None:
edgeenv = (item.get("raw_context") or {}).get("edgeenv_regression")
_record(
isinstance(edgeenv, dict),
errors,
f"AIGuard evidence[{index}] raw_context.edgeenv_regression must be an object",
)
if not isinstance(edgeenv, dict):
return
_record(
edgeenv.get("orchestrator_candidate_operation_max_total_queue_depth")
== 7.0,
errors,
"AIGuard queue overload evidence "
"orchestrator_candidate_operation_max_total_queue_depth must be 7.0",
)


def _validate_aiguard_history_seed_context(
edgeenv_context: dict[str, Any],
errors: list[str],
Expand Down
40 changes: 40 additions & 0 deletions tests/test_runtime_intelligence_bundle_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,10 @@ def test_runtime_intelligence_bundle_manifest_gate_cli_passes(tmp_path):
)
assert "aiguard_raw_context: downstream_guard_alignment preserved" in summary
assert "aiguard_raw_context: producer_lineage_guard_alignment preserved" in summary
assert (
"aiguard_raw_context: max_total_queue_depth traceability preserved"
in summary
)
assert (
"expected_report_markers: Runtime Intelligence report markers declared"
in summary
Expand Down Expand Up @@ -1081,6 +1085,42 @@ def test_runtime_intelligence_bundle_manifest_gate_fails_for_old_guard_coverage_
assert "history missing field runs must be a list" in summary


def test_runtime_intelligence_bundle_manifest_gate_fails_for_missing_guard_max_queue_context(
tmp_path,
):
manifest = json.loads(MANIFEST.read_text(encoding="utf-8"))
guard_path = (
REPO_ROOT
/ "examples"
/ "runtime_intelligence_chain"
/ manifest["files"]["aiguard_guard_analysis"]
)
guard_analysis = json.loads(guard_path.read_text(encoding="utf-8"))
queue_evidence = next(
item
for item in guard_analysis["evidence"]
if item.get("type") == "runtime_queue_overload"
)
edgeenv_context = queue_evidence["raw_context"]["edgeenv_regression"]
edgeenv_context.pop("orchestrator_candidate_operation_max_total_queue_depth")

guard_copy = tmp_path / "aiguard_guard_analysis.json"
guard_copy.write_text(json.dumps(guard_analysis), encoding="utf-8")
manifest["files"]["aiguard_guard_analysis"] = str(guard_copy)
manifest_path = tmp_path / "bundle_manifest.json"
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
summary_path = tmp_path / "bundle_manifest_gate_summary.md"

result = manifest_gate(manifest=str(manifest_path), summary_out=str(summary_path))

assert result == 2
summary = summary_path.read_text(encoding="utf-8")
assert (
"AIGuard queue overload evidence "
"orchestrator_candidate_operation_max_total_queue_depth must be 7.0"
) in summary


def test_runtime_intelligence_bundle_manifest_gate_fails_for_bad_guard_mapping_hint(
tmp_path,
):
Expand Down
6 changes: 6 additions & 0 deletions tests/test_runtime_intelligence_evidence_chain_smoke.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,12 @@ def test_runtime_intelligence_chain_smoke_ingests_precomputed_guard_artifact():
assert guard_edgeenv_context["orchestrator_candidate_context_telemetry_source"] == (
"inferedge_orchestrator_operation_summary"
)
assert (
guard_edgeenv_context[
"orchestrator_candidate_operation_max_total_queue_depth"
]
== 7.0
)
assert guard_edgeenv_context["orchestrator_downstream_guard_alignment"][
"producer_lineage_evidence_type"
] == "edgeenv_orchestrator_producer_lineage"
Expand Down
Loading