Skip to content

Fix evaluation test type checks #570

Fix evaluation test type checks

Fix evaluation test type checks #570

Triggered via pull request August 29, 2025 04:45
Status Failure
Total duration 5m 46s
Artifacts 1

ci.yml

on: pull_request
Lint & Type Check
2m 11s
Lint & Type Check
Matrix: test-core
Batch Evaluation Tests
1m 6s
Batch Evaluation Tests
MCP End-to-End Tests
53s
MCP End-to-End Tests
Upload Coverage
Upload Coverage
Fit to window
Zoom out
Zoom in

Annotations

13 errors and 11 warnings
Lint & Type Check: eval_protocol/_version.py#L71
Expected type arguments for generic class "Callable" (reportMissingTypeArgument)
Lint & Type Check: eval_protocol/_version.py#L68
Expected type arguments for generic class "Callable" (reportMissingTypeArgument)
Lint & Type Check: eval_protocol/_version.py#L65
Expected type arguments for generic class "Callable" (reportMissingTypeArgument)
Lint & Type Check: eval_protocol/_version.py#L43
Instance variable "verbose" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L42
Instance variable "versionfile_source" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L41
Instance variable "parentdir_prefix" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L40
Instance variable "tag_prefix" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L39
Instance variable "style" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/_version.py#L38
Instance variable "VCS" is not initialized in the class body or __init__ method (reportUninitializedInstanceVariable)
Lint & Type Check: eval_protocol/__init__.py#L34
"_FIREWORKS_AVAILABLE" is constant (because it is uppercase) and cannot be redefined (reportConstantRedefinition)
Core Tests (Python 3.12)
Process completed with exit code 1.
Core Tests (Python 3.10)
Process completed with exit code 1.
Core Tests (Python 3.11)
Process completed with exit code 1.
Lint & Type Check: eval_protocol/_version.py#L65
Type of "HANDLERS" is partially unknown   Type of "HANDLERS" is "dict[str, Dict[str, (...) -> Unknown]]" (reportUnknownVariableType)
Lint & Type Check: eval_protocol/_version.py#L64
This type is deprecated as of Python 3.9; use "dict" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L22
This type is deprecated as of Python 3.9; use "dict" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.9; use "tuple" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.10; use "| None" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.9; use "list" instead (reportDeprecated)
Lint & Type Check: eval_protocol/_version.py#L19
This type is deprecated as of Python 3.9; use "dict" instead (reportDeprecated)
Lint & Type Check: eval_protocol/__init__.py#L82
Type of "__version__" is Any (reportAny)
Lint & Type Check: eval_protocol/__init__.py#L24
Type of "rollout" is partially unknown   Type of "rollout" is "(envs: GeneralMCPVectorEnv, policy: FireworksPolicy | LLMBasePolicy | ((...) -> Unknown), *, evaluation_rows: List[EvaluationRow] | None = None, dataset: List[Dict[Unknown, Unknown]] | None = None, model_id: str | None = None, steps: int = 512, openai_format_log_file: str | None = None, max_concurrent_rollouts: int = 8) -> List[Task[EvaluationRow]]" (reportUnknownVariableType)
Lint & Type Check: eval_protocol/__init__.py#L23
Type of "make" is partially unknown   Type of "make" is "(env_spec: str, evaluation_rows: List[EvaluationRow] | None = None, dataset: List[Dict[Unknown, Unknown]] | None = None, n: int | None = None, seeds: List[int] | None = None, model_id: str = "unknown", user_prompt_formatter: ((...) -> Unknown) | None = None) -> GeneralMCPVectorEnv" (reportUnknownVariableType)
MCP End-to-End Tests
No files were found with the provided path: coverage.xml. No artifacts will be uploaded.

Artifacts

Produced during runtime
Name Size Digest
coverage-batch-eval Expired
33.7 KB
sha256:f734ef0f72c25e8599d31b5b9793ff95d7dfdb3a049c031774ca7a7034cfbfe8