gsm8k math example

[benjibc]

Note

SingleTurnRolloutProcessor now optionally trims trailing assistant messages (on by default), with new unit tests and simplified pytest examples removing dataset adapters.

• Pytest rollout processor:
  • SingleTurnRolloutProcessor: add __init__(drop_trailing_assistant_messages=True) and strip trailing assistant messages before LLM call; use trimmed messages when building final conversation.
• Tests:
  • Add tests/pytest/test_single_turn_rollout_processor.py covering default drop and opt-out behaviors.
  • Replace GSM8K math example with tests/pytest/gsm8k/test_pytest_math_example.py using direct dataset and simple digit check.
  • Remove dataset_adapter usage from test_pytest_math_format_length.py and test_pytest_word_count_example.py.
  • Add tests/pytest/gsm8k/requirements.txt with eval-protocol.

^{Written by Cursor Bugbot for commit 5613767. This will update automatically on new commits. Configure here.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[cursor]

Bug: Empty Payload After Message Filtering

Missing validation after filtering trailing assistant messages. If all messages in row.messages are assistant messages and drop_trailing_assistant_messages=True, the messages_for_request list becomes empty, resulting in an empty messages_payload being sent to the LLM API. This will fail with an API error rather than being caught by the existing validation on line 42-43. A check should be added after the filtering loop (lines 47-49) to ensure messages_for_request is not empty before proceeding.