Skip to content

Python: dependency-bounds validator is not enforced in CI #6582

@moonbox3

Description

@moonbox3

The Python dependency-bounds validator (uv run poe validate-dependency-bounds-test) is not wired into required CI, so dependency-floor, packaging, and typing regressions can land on main and ship in a release without being caught.

Evidence

Running the validator at the python-1.8.1 release tag already fails for core at the lowest-direct resolution:

packages/core: lower failed
Task 'test' failed for packages/core at resolution 'lowest-direct'.
FAILED tests/core/test_telemetry.py::test_detect_hosted_fallback_import_error
1 failed, 3174 passed, 18 skipped, 2 xfailed

So 1.8.1 was tagged and released while the validator was red.

On current main the same validator fails with more: the telemetry test above, plus 4 harness shell-tool tests (ModuleNotFoundError: No module named 'agent_framework_tools') and 14 pyright errors on _harness/_agent.py, all from the shell-tool integration in #6451. The validator stops at the first failing task, which is why only the telemetry failure surfaced at 1.8.1.

Two latent problems this masked

  1. packages/core/agent_framework/_telemetry.py caught (ModuleNotFoundError, ValueError) around importlib.util.find_spec(...), but find_spec imports the parent package and can raise a bare ImportError, which escaped the handler. The test passes in the full test environment (where azure.ai.agentserver is already imported, so find_spec reuses the cached parent and never hits the mocked import) but fails in the validator's isolated environment. Pre-existing since Python: Fix user agent prefix #5455.

  2. The harness shell-tool integration (Python: Integrate shell tool into harness agent #6451) imports agent_framework_tools lazily because that package depends on core, so core cannot take a runtime dependency on it. But core did not declare it even as a dev or optional dependency, so any environment that builds core in isolation hits ModuleNotFoundError at test time and unresolved types under pyright.

Both are fixed in the 1.9.0 version bump (branch bump-py-ver-release-260617): broaden the except to ImportError, add an agent-framework-tools dev dependency-group to core, and guard the shell-tool tests with skipif. With those, the validator passes fully (lower and upper, all packages).

Proposal

Wire validate-dependency-bounds-test into Python CI so these regressions are caught at PR time rather than at release time:

  • Run it on PRs that touch python/, or at minimum on the release branch and nightly.
  • It exercises both lowest-direct and highest resolutions per package and runs each package's pyright pass, so it catches floor-too-low, missing-optional-dependency, and type-checking-against-isolated-env problems that the normal full-workspace test job does not.

Cost and scoping note: a full workspace run is somewhat slow (lower plus upper per package). If per-PR cost is a concern, options are to scope the run to changed packages, or run the full sweep nightly plus required-on-release-branch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    pythonIssues related to the Python codebase

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions