Skip to content

Supports explicit pytest parametrization#190

Merged
dphuang2 merged 5 commits intomainfrom
eval-protocol-v2-interface
Sep 18, 2025
Merged

Supports explicit pytest parametrization#190
dphuang2 merged 5 commits intomainfrom
eval-protocol-v2-interface

Conversation

@dphuang2
Copy link
Copy Markdown
Collaborator

@pytest.mark.parametrize(
    "completion_params",
    [
        {
            "model": "fireworks_ai/accounts/fireworks/models/deepseek-v3p1",
        },
        {
            "model": "fireworks_ai/accounts/fireworks/models/kimi-k2-instruct-0905",
        },
    ],
    ids=DefaultParameterIdGenerator.generate_id_from_dict,
)
@evaluation_test(
    input_rows=[input_rows],
    rollout_processor=SingleTurnRolloutProcessor(),
    preprocess_fn=split_multi_turn_rows,
    mode="all",
)
async def test_llm_judge_openai_responses(rows: List[EvaluationRow]) -> List[EvaluationRow]:
    return await aha_judge(rows)

@dphuang2 dphuang2 changed the title Allow for manual pytest parametrization Supports explicit pytest parametrization Sep 18, 2025
@dphuang2 dphuang2 merged commit 057e132 into main Sep 18, 2025
3 of 6 checks passed
@dphuang2 dphuang2 deleted the eval-protocol-v2-interface branch September 18, 2025 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants