feat: add agent artifact bundle validator#205
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new script and corresponding tests for validating deterministic agent artifact bundle payloads. The validator checks for required fields, ensures data type integrity, and scans for non-deterministic fields such as timestamps, random IDs, and UUIDs. Review feedback focuses on aligning the script with repository standards by treating null or missing list fields as empty lists and raising a RuntimeError for invalid types. Additionally, suggestions were made to verify consistency between top-level bundle fields and the nested safe_pr_gate object to prevent data discrepancies.
| def _is_string_list(value: object) -> bool: | ||
| return isinstance(value, list) and all(isinstance(item, str) for item in value) |
There was a problem hiding this comment.
According to the repository rules, null or missing values for expected list fields should be treated as empty lists, while other non-list types should trigger a RuntimeError to maintain strictness. The current implementation of _is_string_list should be updated to handle None as True and raise a RuntimeError for invalid types.
| def _is_string_list(value: object) -> bool: | |
| return isinstance(value, list) and all(isinstance(item, str) for item in value) | |
| def _is_string_list(value: object) -> bool: | |
| if value is None: | |
| return True | |
| if not isinstance(value, list): | |
| raise RuntimeError(f"Expected list, got {type(value).__name__}") | |
| return all(isinstance(item, str) for item in value) |
References
- When processing JSON data, treat null or missing values for expected list fields as empty lists, but raise a RuntimeError for other non-list types to maintain strictness.
| if not isinstance(safe_pr_gate, dict): | ||
| issues.append("bundle.safe_pr_gate must be a JSON object") | ||
| else: | ||
| issues.extend(_validate_safe_pr_gate(safe_pr_gate, ok)) |
There was a problem hiding this comment.
To ensure the integrity of the artifact bundle, the validator should verify that the top-level branch and changed_files fields are consistent with the corresponding fields inside the safe_pr_gate object. This prevents discrepancies between the bundle summary and the detailed gate state.
| issues.extend(_validate_safe_pr_gate(safe_pr_gate, ok)) | |
| issues.extend(_validate_safe_pr_gate(safe_pr_gate, ok)) | |
| if isinstance(branch, str) and isinstance(safe_pr_gate.get("branch"), str) and branch != safe_pr_gate.get("branch"): | |
| issues.append("bundle.branch must match bundle.safe_pr_gate.branch") | |
| if _is_string_list(changed_files) and _is_string_list(safe_pr_gate.get("changed_paths")): | |
| if (changed_files or []) != (safe_pr_gate.get("changed_paths") or []): | |
| issues.append("bundle.changed_files must match bundle.safe_pr_gate.changed_paths") |
| if not isinstance(validation_evidence, list): | ||
| issues.append("bundle.validation_evidence must be a list") | ||
| else: | ||
| issues.extend(_validate_validation_evidence(validation_evidence)) |
There was a problem hiding this comment.
Following the repository rule to treat null or missing list fields as empty lists and raise a RuntimeError for other non-list types, the check for validation_evidence should be updated. If it is None, it should be treated as an empty list; if it is not a list, a RuntimeError should be raised instead of appending to the issues list.
if validation_evidence is not None and not isinstance(validation_evidence, list):
raise RuntimeError("bundle.validation_evidence must be a list")
issues.extend(_validate_validation_evidence(validation_evidence or []))References
- When processing JSON data, treat null or missing values for expected list fields as empty lists, but raise a RuntimeError for other non-list types to maintain strictness.
Adds a minimal deterministic validator CLI for agent artifact bundles.
Includes:
scripts/validate_agent_artifact_bundle.pyartifacts/agent_artifact_bundle_example.json--bundlesupportScope
Local validation only. No network, no external APIs, no timestamps, no random IDs, and no broad refactors.
Validation
python -m compileall -q scripts/validate_agent_artifact_bundle.pypytest tests/test_validate_agent_artifact_bundle.py -q