Skip to content

test_via_plugin.py may resolve agent from user scope, not plugin #20

@krisrowe

Description

@krisrowe

Problem

test_via_plugin.py uses --agent privacy-guard (bare name) with
--plugin-dir to verify that the agent loads via plugin discovery.
However, if ~/.claude/agents/privacy-guard.md also exists at user
scope, the test may pass by resolving from user scope (priority 4)
rather than plugin scope (priority 5).

Claude Code resolves agent names in this order:

  1. Managed settings (highest)
  2. --agents CLI JSON
  3. .claude/agents/ (project scope)
  4. ~/.claude/agents/ (user scope)
  5. Plugin agents/ (lowest)

Since user scope is checked before plugin scope, the bare name
privacy-guard matches the user-scope file first. The test appears
to verify plugin discovery but may actually be testing user-scope
resolution.

Additionally, plugin agents are namespaced: an agent named
privacy-guard in a plugin named claude-coding must be invoked as
--agent claude-coding:privacy-guard. Using the bare name silently
falls through to user scope with no error. This was confirmed via
debug logging:

Warning: agent "plugin-probe" not found. Available agents: ...,
test-plugin:plugin-probe, ...

Proposed fix

Change the test to use the namespaced agent name:

# Before
"--agent", "privacy-guard",

# After
"--agent", "claude-coding:privacy-guard",

This ensures the test proves plugin-based discovery, not user-scope
fallback.

Discovery context

The namespacing requirement was discovered empirically and confirmed
with two tests (one positive, one negative control) that verify:

  • --agent plugin-name:agent-name resolves a plugin agent
  • --agent agent-name (bare) does NOT resolve a plugin agent

The namespacing is consistent with how plugin skills work
(/plugin-name:skill-name).

Work breakdown

  • Update test_via_plugin.py to use --agent claude-coding:privacy-guard
  • Verify the test still passes with the namespaced form
  • Add a comment in the test explaining why the namespaced form is required
  • Document the namespacing requirement in CONTRIBUTING.md under
    the "Can users run claude --agent privacy-guard?" FAQ (already
    partially done — verify the current text is accurate)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions