Skip to content

add better logging to pydantic ai rollouts #581

add better logging to pydantic ai rollouts

add better logging to pydantic ai rollouts #581

Triggered via pull request August 29, 2025 20:36
Status Success
Total duration 5m 52s
Artifacts 4

ci.yml

on: pull_request
Lint & Type Check
1m 40s
Lint & Type Check
Matrix: test-core
Batch Evaluation Tests
1m 9s
Batch Evaluation Tests
MCP End-to-End Tests
45s
MCP End-to-End Tests
Upload Coverage
5s
Upload Coverage
Fit to window
Zoom out
Zoom in

Annotations

10 errors and 11 warnings
Lint & Type Check: eval_protocol/_version.py#L71
Expected type arguments for generic class "Callable" (reportMissingTypeArgument)
Lint & Type Check: eval_protocol/_version.py#L68
Expected type arguments for generic class "Callable" (reportMissingTypeArgument)
Lint & Type Check: eval_protocol/_version.py#L65
Expected type arguments for generic class "Callable" (reportMissingTypeArgument)
Lint & Type Check: eval_protocol/_version.py#L43
Instance variable "verbose" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L42
Instance variable "versionfile_source" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L41
Instance variable "parentdir_prefix" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L40
Instance variable "tag_prefix" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L39
Instance variable "style" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L38
Instance variable "VCS" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/__init__.py#L34
"_FIREWORKS_AVAILABLE" is constant (because it is uppercase) and cannot be redefined (reportConstantRedefinition)
Lint & Type Check: eval_protocol/_version.py#L65
Type of "HANDLERS" is partially unknown   Type of "HANDLERS" is "dict[str, Dict[str, (...) -> Unknown]]" (reportUnknownVariableType)
Lint & Type Check: eval_protocol/_version.py#L64
This type is deprecated as of Python 3.9; use "dict" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L22
This type is deprecated as of Python 3.9; use "dict" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.9; use "tuple" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.10; use "| None" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.9; use "list" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.9; use "dict" instead (reportDeprecated)
Lint & Type Check: eval_protocol/__init__.py#L82
Type of "__version__" is Any (reportAny)
Lint & Type Check: eval_protocol/__init__.py#L24
Type of "rollout" is partially unknown   Type of "rollout" is "(envs: GeneralMCPVectorEnv, policy: FireworksPolicy | LLMBasePolicy | ((...) -> Unknown), *, evaluation_rows: List[EvaluationRow] | None = None, dataset: List[Dict[Unknown, Unknown]] | None = None, model_id: str | None = None, steps: int = 512, openai_format_log_file: str | None = None, max_concurrent_rollouts: int = 8) -> List[Task[EvaluationRow]]" (reportUnknownVariableType)
Lint & Type Check: eval_protocol/__init__.py#L23
Type of "make" is partially unknown   Type of "make" is "(env_spec: str, evaluation_rows: List[EvaluationRow] | None = None, dataset: List[Dict[Unknown, Unknown]] | None = None, n: int | None = None, seeds: List[int] | None = None, model_id: str = "unknown", user_prompt_formatter: ((...) -> Unknown) | None = None) -> GeneralMCPVectorEnv" (reportUnknownVariableType)
MCP End-to-End Tests
No files were found with the provided path: coverage.xml. No artifacts will be uploaded.

Artifacts

Produced during runtime
Name Size Digest
coverage-batch-eval Expired
33.7 KB
sha256:44f4a3672d40b99585761e7723d85fca83a3aa2a438ca1d9f29356d3de6181e2
coverage-core-3.10 Expired
40.7 KB
sha256:86d5c6616802cc535416ada28d30fd693ce1c0d8883d090cb877513d1c3fb46a
coverage-core-3.11 Expired
40.7 KB
sha256:a9cb9e7c6b8fdc0d1ff0a37b1ad21cd22207f7c892a4ead36d7c4dccd45af37a
coverage-core-3.12 Expired
40.8 KB
sha256:b11b2f2cba34a7effec3f3d567c827fe53034583df940f5b42f66343d41aca19