Skip to content

feat: add agent artifact bundle validator#205

Merged
ProfRandom92 merged 1 commit into
mainfrom
feat/validate-agent-artifact-bundle
May 22, 2026
Merged

feat: add agent artifact bundle validator#205
ProfRandom92 merged 1 commit into
mainfrom
feat/validate-agent-artifact-bundle

Conversation

@ProfRandom92
Copy link
Copy Markdown
Owner

Adds a minimal deterministic validator CLI for agent artifact bundles.

Includes:

  • scripts/validate_agent_artifact_bundle.py
  • default validation for artifacts/agent_artifact_bundle_example.json
  • --bundle support
  • required field checks
  • ok/result consistency checks
  • safe_pr_gate status checks
  • conservative timestamp/random-id field rejection
  • focused validator tests

Scope

Local validation only. No network, no external APIs, no timestamps, no random IDs, and no broad refactors.

Validation

  • python -m compileall -q scripts/validate_agent_artifact_bundle.py
  • pytest tests/test_validate_agent_artifact_bundle.py -q

@ProfRandom92 ProfRandom92 merged commit 413b6f4 into main May 22, 2026
7 checks passed
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new script and corresponding tests for validating deterministic agent artifact bundle payloads. The validator checks for required fields, ensures data type integrity, and scans for non-deterministic fields such as timestamps, random IDs, and UUIDs. Review feedback focuses on aligning the script with repository standards by treating null or missing list fields as empty lists and raising a RuntimeError for invalid types. Additionally, suggestions were made to verify consistency between top-level bundle fields and the nested safe_pr_gate object to prevent data discrepancies.

Comment on lines +73 to +74
def _is_string_list(value: object) -> bool:
return isinstance(value, list) and all(isinstance(item, str) for item in value)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository rules, null or missing values for expected list fields should be treated as empty lists, while other non-list types should trigger a RuntimeError to maintain strictness. The current implementation of _is_string_list should be updated to handle None as True and raise a RuntimeError for invalid types.

Suggested change
def _is_string_list(value: object) -> bool:
return isinstance(value, list) and all(isinstance(item, str) for item in value)
def _is_string_list(value: object) -> bool:
if value is None:
return True
if not isinstance(value, list):
raise RuntimeError(f"Expected list, got {type(value).__name__}")
return all(isinstance(item, str) for item in value)
References
  1. When processing JSON data, treat null or missing values for expected list fields as empty lists, but raise a RuntimeError for other non-list types to maintain strictness.

if not isinstance(safe_pr_gate, dict):
issues.append("bundle.safe_pr_gate must be a JSON object")
else:
issues.extend(_validate_safe_pr_gate(safe_pr_gate, ok))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure the integrity of the artifact bundle, the validator should verify that the top-level branch and changed_files fields are consistent with the corresponding fields inside the safe_pr_gate object. This prevents discrepancies between the bundle summary and the detailed gate state.

Suggested change
issues.extend(_validate_safe_pr_gate(safe_pr_gate, ok))
issues.extend(_validate_safe_pr_gate(safe_pr_gate, ok))
if isinstance(branch, str) and isinstance(safe_pr_gate.get("branch"), str) and branch != safe_pr_gate.get("branch"):
issues.append("bundle.branch must match bundle.safe_pr_gate.branch")
if _is_string_list(changed_files) and _is_string_list(safe_pr_gate.get("changed_paths")):
if (changed_files or []) != (safe_pr_gate.get("changed_paths") or []):
issues.append("bundle.changed_files must match bundle.safe_pr_gate.changed_paths")

Comment on lines +143 to +146
if not isinstance(validation_evidence, list):
issues.append("bundle.validation_evidence must be a list")
else:
issues.extend(_validate_validation_evidence(validation_evidence))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Following the repository rule to treat null or missing list fields as empty lists and raise a RuntimeError for other non-list types, the check for validation_evidence should be updated. If it is None, it should be treated as an empty list; if it is not a list, a RuntimeError should be raised instead of appending to the issues list.

    if validation_evidence is not None and not isinstance(validation_evidence, list):
        raise RuntimeError("bundle.validation_evidence must be a list")
    issues.extend(_validate_validation_evidence(validation_evidence or []))
References
  1. When processing JSON data, treat null or missing values for expected list fields as empty lists, but raise a RuntimeError for other non-list types to maintain strictness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant