Skip to content

assert for evaluation_row#339

Merged
xzrderek merged 5 commits intomainfrom
derekx/assert-error
Nov 18, 2025
Merged

assert for evaluation_row#339
xzrderek merged 5 commits intomainfrom
derekx/assert-error

Conversation

@xzrderek
Copy link
Contributor

@xzrderek xzrderek commented Nov 18, 2025

Note

Enforces that @evaluation_test must set row.evaluation_result, updates tests to attach a dummy EvaluateResult, and adds a test asserting an error when missing.

  • Core (evaluation):
    • Add assertion in eval_protocol/pytest/evaluation_test.py to ensure every EvaluationRow has evaluation_result; raise clear AssertionError if missing.
  • Tests:
    • Import EvaluateResult and set row.evaluation_result across tests using @evaluation_test (data loaders, input rows/messages, pydantic agent, remote server, logging, IDs, env overrides, etc.).
    • Add tests/pytest/test_pytest_missing_evaluation_result.py to verify missing evaluation_result raises the expected assertion.
    • Minor adjustments to metadata assertions and async markers where applicable.

Written by Cursor Bugbot for commit 4b80012. This will update automatically on new commits. Configure here.

Copy link
Collaborator

@dphuang2 dphuang2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@xzrderek xzrderek merged commit ca8b2e8 into main Nov 18, 2025
9 checks passed
@xzrderek xzrderek deleted the derekx/assert-error branch November 18, 2025 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants