adding in the openai integration

[xzrderek]

Note

Adds OpenAI RFT integration to convert Eval Protocol evaluation functions into Python grader specs, plus example usage and tests.

• Integrations:
  • Add eval_protocol/integrations/openai_rft.py with build_python_grader_from_evaluation_test to convert evaluation-style functions into OpenAI Python grader specs by:
    • Stripping decorators/annotations via AST, renaming to _ep_eval, and embedding helper types (EvaluationRow, EvaluateResult, Message).
    • Building grade(sample, item) that maps item/sample to a duck-typed row and normalizes outputs to a float score.
  • Export in eval_protocol/integrations/__init__.py.
• Examples:
  • examples/openai_rft/example_rapidfuzz.py: demo @evaluation_test using RapidFuzz and conversion to grader.
  • examples/openai_rft/test_openai_grader.py: script to validate/run the grader via OpenAI API.
• Tests:
  • tests/test_openai_rft_integration.py: verifies grader generation from plain and wrapped functions and correct scoring behavior.

^{Written by Cursor Bugbot for commit 6fc1c36. This will update automatically on new commits. Configure here.}

[cursor]

Bug: Unhandled Exceptions Propagate, Crashing Evaluation

The call to _ep_eval on line 152 is outside the try-except block, so exceptions raised by the user's evaluation function will not be caught. Only normalization errors are caught, allowing crashes from the evaluation logic to propagate instead of gracefully returning 0.0 as intended.

 

[dphuang2]

great!