Skip to content

warn if large datasets + force 1 run #1534

warn if large datasets + force 1 run

warn if large datasets + force 1 run #1534

Triggered via pull request December 12, 2025 20:25
Status Success
Total duration 10m 49s
Artifacts 4

ci.yml

on: pull_request
Lint & Type Check
1m 4s
Lint & Type Check
Matrix: test-core
Batch Evaluation Tests
1m 14s
Batch Evaluation Tests
MCP End-to-End Tests
52s
MCP End-to-End Tests
Upload Coverage
5s
Upload Coverage
Fit to window
Zoom out
Zoom in

Annotations

18 errors and 3 warnings
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L163
"get_tokenizer" is not a known attribute of module "tinker_cookbook" (reportAttributeAccessIssue)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L162
Variable not allowed in type expression (reportInvalidTypeForm)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L162
Variable not allowed in type expression (reportInvalidTypeForm)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L155
Argument to class must be a base class (reportGeneralTypeIssues)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L143
Variable not allowed in type expression (reportInvalidTypeForm)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L115
Argument of type "None" cannot be assigned to parameter "logger" of type "DatasetLogger" in function "__init__"   "None" is not assignable to "DatasetLogger" (reportArgumentType)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L74
Argument to class must be a base class (reportGeneralTypeIssues)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L54
Variable not allowed in type expression (reportInvalidTypeForm)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L36
Variable not allowed in type expression (reportInvalidTypeForm)
Lint & Type Check: eval_protocol/integrations/tinker_cookbook.py#L32
Argument to class must be a base class (reportGeneralTypeIssues)
Core Tests (Python 3.10)
Event loop is closed
Core Tests (Python 3.10)
Event loop is closed
Core Tests (Python 3.10)
Event loop is closed
Core Tests (Python 3.10)
Event loop is closed
Core Tests (Python 3.10)
Event loop is closed
Core Tests (Python 3.10)
Event loop is closed
Core Tests (Python 3.10)
Event loop is closed
Core Tests (Python 3.10)
Event loop is closed
Lint & Type Check: eval_protocol/mcp/__init__.py#L49
Operation on "__all__" is not supported, so exported symbol list may be incorrect (reportUnsupportedDunderAll)
Lint & Type Check: eval_protocol/mcp/__init__.py#L49
Operation on "__all__" is not supported, so exported symbol list may be incorrect (reportUnsupportedDunderAll)
MCP End-to-End Tests
No files were found with the provided path: coverage.xml. No artifacts will be uploaded.

Artifacts

Produced during runtime
Name Size Digest
coverage-batch-eval Expired
48.9 KB
sha256:7bae19258396eb5bb2ebc4a8611b924bbd079e33629c17e9a9c0c50a2fae9b73
coverage-core-3.10 Expired
63.2 KB
sha256:7a221afd71b6db3ae173bd587459fa21af1677a1dfe1bcc4411940895ab2a817
coverage-core-3.11 Expired
63.2 KB
sha256:537b1280c8cf5d031b336090bce0f417cdc0cc8c3244a4d8cde403cc9a75dec6
coverage-core-3.12 Expired
63.2 KB
sha256:a0406471ad34b5f01293a8736b13cdfcf8b58207aed49a99960666a9f89ed9f6