Skip to content
Python CI

Add type safety and error handling improvements to evaluation pipelin… #621

Sign in to view logs

Add type safety and error handling improvements to evaluation pipelin…

Add type safety and error handling improvements to evaluation pipelin… #621

Triggered via push September 1, 2025 07:53
@benjibcbenjibc
pushed caf93cf
main
Status Success
Total duration 5m 29s
Artifacts 4

ci.yml

on: push
Lint & Type Check
55s
Lint & Type Check
Matrix: test-core
Batch Evaluation Tests
1m 7s
Batch Evaluation Tests
MCP End-to-End Tests
44s
MCP End-to-End Tests
Upload Coverage
9s
Upload Coverage
Fit to window
Zoom out
Zoom in

Annotations

10 errors and 3 warnings
Lint & Type Check: eval_protocol/benchmarks/test_gpqa.py#L108
Argument of type "str | List[ChatCompletionContentPartTextParam]" cannot be assigned to parameter "text" of type "str" in function "_extract_abcd_letter"   Type "str | List[ChatCompletionContentPartTextParam]" is not assignable to type "str"     "List[ChatCompletionContentPartTextParam]" is not assignable to "str" (reportArgumentType)
Lint & Type Check: eval_protocol/benchmarks/test_gpqa.py#L85
Cannot access attribute "startswith" for class "List[ChatCompletionContentPartTextParam]"   Attribute "startswith" is unknown (reportAttributeAccessIssue)
Lint & Type Check: eval_protocol/benchmarks/test_gpqa.py#L82
"split" is not a known attribute of "None" (reportOptionalMemberAccess)
Lint & Type Check: eval_protocol/benchmarks/test_gpqa.py#L82
Cannot access attribute "split" for class "List[ChatCompletionContentPartTextParam]"   Attribute "split" is unknown (reportAttributeAccessIssue)
Lint & Type Check: eval_protocol/benchmarks/test_gpqa.py#L79
Cannot access attribute "startswith" for class "List[ChatCompletionContentPartTextParam]"   Attribute "startswith" is unknown (reportAttributeAccessIssue)
Lint & Type Check: eval_protocol/benchmarks/test_gpqa.py#L63
Cannot access attribute "startswith" for class "List[ChatCompletionContentPartTextParam]"   Attribute "startswith" is unknown (reportAttributeAccessIssue)
Lint & Type Check: eval_protocol/benchmarks/test_aime25.py#L101
No parameter named "data" (reportCallIssue)
Lint & Type Check: eval_protocol/benchmarks/test_aime25.py#L85
Argument of type "str | List[ChatCompletionContentPartTextParam]" cannot be assigned to parameter "text" of type "str" in function "_extract_boxed_text"   Type "str | List[ChatCompletionContentPartTextParam]" is not assignable to type "str"     "List[ChatCompletionContentPartTextParam]" is not assignable to "str" (reportArgumentType)
Lint & Type Check: eval_protocol/adapters/langfuse.py#L16
"Langfuse" is not exported from module "langfuse"   Import from "langfuse._client.client" instead (reportPrivateImportUsage)
Lint & Type Check: eval_protocol/adapters/bigquery.py#L197
Except clause is unreachable because exception is already handled   "Exception" is a subclass of "Exception" (reportUnusedExcept)
Lint & Type Check: eval_protocol/mcp/__init__.py#L49
Operation on "__all__" is not supported, so exported symbol list may be incorrect (reportUnsupportedDunderAll)
Lint & Type Check: eval_protocol/mcp/__init__.py#L49
Operation on "__all__" is not supported, so exported symbol list may be incorrect (reportUnsupportedDunderAll)
MCP End-to-End Tests
No files were found with the provided path: coverage.xml. No artifacts will be uploaded.

Artifacts

Produced during runtime
Name Size Digest
coverage-batch-eval Expired
33.5 KB
sha256:0dbfbf6762a4b705176fda9f34f212b67bbde87ae487c089c2d4c68aa72c0135
coverage-core-3.10 Expired
40.7 KB
sha256:7c1475b966768d92d435db5963dfd1dcdc8eadfa396e73e25077b6f8523732d1
coverage-core-3.11 Expired
40.7 KB
sha256:ffa983f654b29936c3bfc18f86cb083ec53c9c7b89e5fd1e4b790a41b56f3413
coverage-core-3.12 Expired
40.7 KB
sha256:99e7760ffefc3f6949b338df16c5b24bc1f505d18e4eecd6f8c7838478c4a057