[FEATURE] Support URL-based marketplace registration for Agent Skills discovery indexes

[Vicente-Pastor]

Summary

Extend APM marketplace support so a marketplace can be registered from a URL-based discovery index, including standards-compliant Agent Skills discovery endpoints such as:

• https://example.com/.well-known/agent-skills/index.json
• or an origin that resolves to that index, such as https://example.com

The primary UX should be:

apm marketplace add https://example.com/.well-known/agent-skills/index.json --name workbooks
apm install code-review@workbooks

and optionally also:

apm marketplace add https://example.com --name workbooks --type well-known
apm install code-review@workbooks

This keeps the existing NAME@MARKETPLACE install flow intact while allowing marketplace backends that are not GitHub repos with marketplace.json.

Why

Today APM marketplace registration assumes a Git repository source:

• apm marketplace add OWNER/REPO
• fetch marketplace.json
• install with plugin-name@marketplace-name

That works for GitHub/Git-hosted indexes, but it does not work for the open Agent Skills discovery model, where the source of truth is an HTTP origin exposing a standards-based index at /.well-known/agent-skills/index.json.

Supporting URL-based marketplace registration would:

• preserve the current name@marketplace UX
• let APM consume standards-compliant Agent Skills catalogs
• avoid inventing a separate skill install syntax
• reduce implementation duplication by treating well-known discovery as another marketplace backend
• make APM friendlier for internal enterprise domains and non-GitHub publishers

Reference Specs

• Agent Skills spec: https://agentskills.io/specification
• RFC 8615: https://datatracker.ietf.org/doc/html/rfc8615
• Discovery RFC draft / proposal: https://github.com/cloudflare/agent-skills-discovery-rfc

Related Issues

• feat: Marketplace integration -- read marketplace.json for plugin discovery + governance #501
• feat: Support Artifactory-hosted marketplace indexes #506
• [FEATURE] Skills from .well-known URIs #554

Relevant points:

• Skills are centered on SKILL.md with required name and description.
• Discovery is origin-based through /.well-known/agent-skills/.
• Publishers should expose /.well-known/agent-skills/index.json.
• The index lists skill entries with metadata such as name, type, description, url, and optional digest.
• RFC 8615 security considerations apply because /.well-known/ is origin-scoped metadata.

Design Direction

This should follow marketplace flow, not introduce a separate install model.

That means:

• registration remains apm marketplace add ... --name workbooks
• install remains apm install code-review@workbooks
• browse/search/update/remove continue to work under the marketplace command group
• the implementation generalizes the marketplace source model rather than adding a second discovery subsystem with different UX

In short:

• same install UX as current marketplaces
• new marketplace backend for URL-based discovery indexes

Update Semantics

The issue should explicitly preserve the same update model APM already uses for marketplaces:

1. Marketplace index updates

These update discovery metadata, not the installed dependency itself.

For URL-backed marketplaces:

apm marketplace update workbooks

should:

• re-fetch the registered URL-backed index
• refresh the local marketplace cache
• update what browse, search, and future name@marketplace resolutions see
• not silently change already-installed skills in the project

2. Installed dependency updates

Installed skills should continue to be controlled by normal install/update flows and the lockfile.

That means:

• apm install code-review@workbooks resolves using the current marketplace index
• the resolved artifact URL, digest, and provenance are then written into apm.lock.yaml
• once installed, the project stays pinned to what was resolved at install time
• if the publisher later changes /.well-known/agent-skills/index.json, the existing project should not drift automatically
• users must explicitly re-run install/update flows to adopt the newly published artifact

3. Publisher behavior at the well-known URL

Publishers manage updates by changing the well-known discovery index.

Examples of publisher-managed changes:

• updating a skill's description
• changing the artifact url
• publishing a new digest
• adding or removing skills from the index

APM should consume those changes only after the user refreshes marketplace metadata and then performs an install/update action.

4. Lockfile responsibility

The lockfile is the boundary between discovery and installation.

The well-known URL controls:

• what skills are discoverable
• what artifact URL and digest are currently advertised

The lockfile controls:

• what artifact was actually installed in this project
• what provenance should be shown later
• whether future installs count as an update or remain unchanged

5. Required provenance for update management

To make update behavior clear and auditable, URL-backed marketplace installs should record at least:

• marketplace name
• marketplace source type
• index URL and/or origin URL
• resolved artifact URL at install time
• digest from the index if present
• locally computed content hash of the fetched skill artifact or fetched file set

This allows APM to distinguish:

• marketplace metadata changed
• publisher-advertised artifact changed
• installed artifact changed
• lockfile still points to the previous artifact

6. Digest-first update algorithm

When the discovery index provides a digest, APM should use it as the fast-path signal for update detection.

Recommended behavior during explicit install/update flows:

1. Resolve skill-name@marketplace against the latest marketplace index.
2. If the index provides a digest:

• compare the advertised digest to the digest stored in the lockfile
• if the digest matches, treat the install as a no-op unless the user explicitly forces a reinstall
• if the digest differs, fetch the artifact and verify the fetched content matches the advertised digest before accepting the update

3. If the index does not provide a digest:

• fetch the skill files needed for installation
• compute a deterministic local content hash
• compare that hash to the value stored in the lockfile

4. After a successful install/update:

• store the advertised digest if present
• store the locally computed content hash

This gives APM two useful signals:

• publisher digest for cheap update detection
• local content hash for integrity verification and fallback when digest is missing

7. Metadata-only changes

If only index metadata changes, such as:

• description text
• ordering in the index
• non-install-affecting index fields

and the digest or local content hash does not change, APM should treat the installed skill as unchanged.

Scope

In scope

• allow apm marketplace add <URL> --name <marketplace>
• support well-known Agent Skills indexes as a marketplace backend
• resolve skill-name@marketplace-name against the registered URL-based index
• download and install the selected skill through the existing APM install pipeline
• persist provenance in apm.lock.yaml
• add trust/policy controls for URL-based marketplace origins
• document the new registration flow
• document how marketplace metadata updates differ from installed dependency updates
• document digest-first update checks with local hash fallback

Out of scope for the first iteration

• replacing the existing Git-based marketplace flow
• publisher tooling for generating /.well-known/agent-skills/
• arbitrary multi-origin search/federation
• background crawling outside explicit marketplace registration
• remote activation without install/lockfile resolution
• automatic background upgrade of installed skills when the well-known index changes

Detailed Plan

1. Generalize the marketplace source model

Goal: make one marketplace registry capable of storing either repo-backed sources or URL-backed sources.

Steps:

• Review the current MarketplaceSource shape in src/apm_cli/marketplace/models.py and list which fields are GitHub/repo-specific.
• Add a source_type discriminator such as repo, url_index, or well_known.
• Add URL-capable fields such as url, index_url, and optional origin_url.
• Decide which fields are required for each source type.
• Update to_dict() so repo-backed entries serialize exactly as they do today unless new fields are needed.
• Update from_dict() to load both old registry entries and new URL-backed entries.
• Keep existing registry files valid without migration if possible.
• If migration is required, add a one-time compatibility path and document it in code comments.

Definition of done:

• Existing repo-backed marketplace entries still load unchanged.
• New URL-backed marketplace entries can be persisted and reloaded.
• ~/.apm/marketplaces.json can contain a mix of both source types.

2. Extend apm marketplace add to accept URLs

Goal: users can register a well-known Agent Skills index without referencing a Git repo.

Steps:

• Inspect src/apm_cli/commands/marketplace.py add() and identify where input is currently assumed to be OWNER/REPO or HOST/OWNER/REPO.
• Add input detection for full HTTP/HTTPS URLs.
• Decide the CLI contract:
• support direct index URL form: apm marketplace add https://example.com/.well-known/agent-skills/index.json --name workbooks
• optionally support origin form: apm marketplace add https://example.com --name workbooks --type well-known
• If --type well-known is supported, resolve the origin to https://example.com/.well-known/agent-skills/index.json.
• Validate marketplace names using the existing NAME@MARKETPLACE rules.
• Reject unsupported URL schemes.
• Reject malformed origin URLs.
• Add explicit errors for:
• missing index
• invalid JSON
• unsupported schema
• non-HTTPS URLs if those are disallowed

Definition of done:

• apm marketplace add OWNER/REPO still works.
• apm marketplace add https://example.com/.well-known/agent-skills/index.json --name workbooks works.
• The command stores a URL-backed marketplace source and confirms registration.

3. Add parsing for well-known Agent Skills indexes

Goal: parse /.well-known/agent-skills/index.json into a structure the marketplace layer can use.

Steps:

• Create index and entry dataclasses for the discovery document.
• Parse the top-level $schema field.
• Parse the skills array.
• For each entry, parse at least:
• name
• type
• description
• url
• digest
• Normalize relative url values against the registered index URL.
• Define how unsupported entry types are handled.
• Decide whether invalid entries are skipped with warnings or fail the whole index.
• Keep enough schema/version metadata to support future compatibility checks.

Definition of done:

• A valid index produces a typed model.
• Relative entry URLs resolve correctly.
• Unsupported or malformed entries produce deterministic behavior.

4. Generalize marketplace fetching and caching

Goal: one marketplace client can fetch both Git-hosted marketplace.json and URL-based skill indexes.

Steps:

• Inspect src/apm_cli/marketplace/client.py and separate repo-specific fetch logic from generic cache logic.
• Introduce a source-type-based fetch path.
• Keep current repo-backed fetch behavior unchanged.
• Add URL-based fetch behavior for index URLs.
• Reuse the existing cache directory and TTL strategy if appropriate.
• Define cache keys that cannot collide between repo-backed and URL-backed sources.
• Decide whether the cache key should include marketplace name, resolved index URL, or both.
• Add stale-if-error behavior for URL-backed indexes if it matches current marketplace semantics.
• Decide how redirects are handled and whether the final resolved URL is persisted.
• Define what exactly apm marketplace update <name> refreshes for URL-backed marketplaces.
• Make sure marketplace update only refreshes index metadata and does not modify installed dependencies.

Definition of done:

• Repo-backed marketplaces still fetch exactly as before.
• URL-backed marketplaces can be fetched, parsed, cached, and refreshed.
• Cache invalidation works for both source types.
• Marketplace update semantics are documented in code and tests.

5. Map URL-based index entries into marketplace items

Goal: name@workbooks resolves using the same marketplace flow regardless of backend type.

Steps:

• Review src/apm_cli/marketplace/resolver.py and identify where it assumes a plugin source comes from repo metadata.
• Define a normalized internal item shape for marketplace results so both backend types can emit comparable entries.
• For URL-backed marketplaces, map each skill index entry to that normalized shape.
• Preserve user-facing fields such as name, description, and source marketplace name.
• Add enough fields for later install resolution, such as artifact URL and digest.
• Ensure find_plugin()/search-style behavior can work for URL-backed entries too.

Definition of done:

• Browse and search can list items from URL-backed marketplaces.
• Resolver can look up code-review@workbooks from a URL-backed marketplace.

6. Resolve name@marketplace into an installable artifact

Goal: the install command can resolve a marketplace item into a concrete remote artifact without changing user syntax.

Steps:

• Trace the current name@marketplace resolution path from CLI input through parse_marketplace_ref() to resolve_marketplace_plugin().
• Keep that syntax unchanged.
• Add a resolution branch for URL-backed marketplaces.
• For well-known index entries, resolve the selected skill to:
• a final artifact URL
• an optional digest
• a resolved display/provenance structure
• Decide the canonical in-memory representation passed into the install/downloader layer.
• Ensure error messages include both marketplace name and backing URL when resolution fails.
• Define how resolution behaves if the marketplace cache is stale versus freshly refreshed.

Definition of done:

• apm install code-review@workbooks resolves to a specific URL-backed skill artifact.
• Existing name@marketplace repo-backed resolution still works.

7. Download and normalize discovered skill artifacts

Goal: remote artifacts discovered through a URL-backed marketplace are converted into something APM already knows how to install.

Steps:

• Decide first-iteration supported artifact types.
• Support skill-md first.
• Decide whether archive support lands in v1 or phase 2.
• For skill-md entries:
• download the SKILL.md
• create a temporary package directory
• place the downloaded file in the expected location
• attach source/provenance metadata needed later
• For archive entries, if supported:
• download archive
• verify digest if present
• unpack to a temp directory
• reject path traversal or unsafe entries
• validate resulting directory structure
• Decide behavior when the remote artifact contains only SKILL.md and not related assets referenced from the file.
• Ensure temp directories are cleaned up on success and failure paths.
• Compute a deterministic local content hash for the fetched artifact or fetched file set after download.

Definition of done:

• A URL-discovered skill-md artifact can be turned into a local temporary package.
• That package can move through the existing validation/install path.
• A local content hash is available for lockfile storage and fallback update checks.

8. Reuse the existing package validation and skill integration flow

Goal: keep the implementation centered on the current APM install pipeline rather than a custom remote-skill path.

Steps:

• Trace how current SKILL.md packages are validated in src/apm_cli/models/validation.py.
• Trace how they are deployed through src/apm_cli/integration/skill_integrator.py.
• Feed URL-backed marketplace artifacts into this same pipeline after normalization.
• Add any missing metadata plumbing so PackageInfo and APMPackage still carry what downstream code needs.
• Verify deployment still works for supported targets such as Codex, Claude, Cursor, and VS Code.
• Verify partial install, reinstall, and update flows do not bypass the new source type.
• Verify reinstall/update flows only change deployed content when the user explicitly runs install/update, not when marketplace metadata alone is refreshed.

Definition of done:

• URL-backed marketplace installs land in the same target directories as existing skill installs.
• No separate deployment subsystem is required.
• Metadata refresh and installed-content refresh are clearly separated.

9. Tighten validation for standards-based remote skills

Goal: standards-based indexes should get standards-based validation.

Steps:

• Review the current loose SKILL.md handling in validation code.
• For URL-backed standards-based sources, require name and description.
• Validate skill name rules from the Agent Skills spec:
• lowercase letters, numbers, and hyphens only
• 1 to 64 characters
• no leading/trailing hyphen
• no consecutive hyphens
• Preserve optional fields when present:
• license
• compatibility
• metadata
• allowed-tools
• Decide whether non-compliant remote skills are rejected or allowed behind a compatibility flag.
• Keep backward compatibility for existing non-standard Git-hosted skill installs unless intentionally changed in a separate issue.

Definition of done:

• Standards-based URL-backed skills are validated against the spec.
• Existing legacy skill installs are not accidentally broken.

10. Capture provenance in the lockfile

Goal: installs remain auditable and reproducible even when the backing index changes.

Steps:

• Review current marketplace provenance fields in the lockfile.
• Add any new fields needed to represent URL-backed marketplace discovery.
• Store at least:
• marketplace name
• marketplace source type
• index URL and/or origin URL
• resolved artifact URL
• digest from the index if present
• locally computed content hash
• Decide how update behavior works when the same skill name later points to a different URL or digest.
• Ensure deps list, deps tree, and view can surface useful provenance for these installs.
• Define how APM determines whether a later install/update should be reported as no change, metadata changed only, or artifact changed.
• Define precedence rules:
• if digest is present, compare digest first
• if digest differs, fetch and verify content before accepting the update
• if digest is absent, fall back to local content hash comparison

Definition of done:

• Lockfile data is sufficient to explain where the skill came from.
• The recorded provenance distinguishes repo-backed and URL-backed marketplaces.
• Lockfile data is sufficient to reason about future updates.
• Digest-first and hash-fallback update checks are possible from lockfile state.

11. Add security and governance controls

Goal: URL-based marketplaces introduce origin trust concerns that need explicit policy.

Steps:

• Review current policy structures under src/apm_cli/policy/.
• Decide whether URL-backed marketplace origins should be controlled by existing dependency policies or a dedicated marketplace/origin policy section.
• Add allow/deny evaluation for URL-backed marketplace origins or index URLs.
• Enforce HTTPS for production usage unless there is an explicit dev/test exception.
• Apply digest verification when digest is present in the index.
• Reuse hidden-character/content scanning before final deployment.
• Decide whether untrusted origins are blocked automatically or surfaced for explicit user approval.
• Add clear failure messaging when a policy denies a source.

Definition of done:

• URL-backed marketplace installs are subject to trust and policy checks.
• Digest mismatch and disallowed-origin failures are explicit and actionable.

12. Make all marketplace commands work with URL-backed sources

Goal: adding a new backend should not fragment the marketplace UX.

Steps:

• Update apm marketplace list to show the source kind for each registration.
• Update apm marketplace browse so it can render entries from URL-backed indexes.
• Update apm marketplace update to refresh URL-backed caches.
• Update apm marketplace remove to unregister URL-backed marketplaces cleanly.
• Ensure apm search "query@marketplace" still works from the user’s perspective.
• Add help text and examples for both repo-backed and URL-backed forms.
• Ensure command help and output clearly explain that marketplace update refreshes index metadata and does not auto-upgrade already-installed skills.

Definition of done:

• Users can add, list, browse, search, update, and remove URL-backed marketplaces using the same command group.
• Update behavior is explicit in command output/help text.

13. Document the new flow

Goal: explain the feature in terms users and maintainers can actually follow.

Steps:

• Update marketplace docs with side-by-side examples of repo-backed and URL-backed registration.
• Document the primary command:
• apm marketplace add https://example.com/.well-known/agent-skills/index.json --name workbooks
• Document install flow:
• apm install code-review@workbooks
• If supported, document origin shorthand with --type well-known.
• Explain what gets stored in apm.yml and apm.lock.yaml.
• Explain trust and policy expectations for URL-backed marketplaces.
• Explain the difference between Git-hosted marketplace.json indexes and standards-based well-known indexes.
• Add a dedicated section explaining update semantics:
• apm marketplace update <name> refreshes discovery metadata only
• install/update flows change deployed content and lockfile state
• well-known URL changes do not auto-update existing projects
• digest is used first for update detection when present
• local content hash is used for verification and fallback when digest is absent

Definition of done:

• A user can follow the docs end-to-end without needing to inspect source code.
• A user can understand exactly how marketplace refresh differs from dependency update.

14. Add tests in implementation order

Goal: land the feature with enough coverage to refactor safely.

Steps:

• Add unit tests first for:
• source model serialization/deserialization
• URL input parsing
• well-known index parsing
• relative URL resolution
• cache key generation and cache behavior
• resolver behavior for name@marketplace
• digest verification
• local content hash generation
• policy evaluation for allowed/denied origins
• marketplace update behavior for URL-backed sources
• Add integration tests next for:
• registering a marketplace from a direct index URL
• browsing/searching a URL-backed marketplace
• installing skill-name@workbooks
• writing lockfile provenance for a URL-backed marketplace install
• updating and removing URL-backed marketplaces
• confirming that marketplace update does not modify installed artifacts on disk
• confirming digest-match no-op behavior
• confirming digest-mismatch triggers fetch and verification
• confirming hash fallback works when digest is absent
• blocking untrusted or invalid origins
• Add acceptance coverage last for:
• end-to-end add -> browse/search -> install
• refresh marketplace metadata and verify no deployment drift until explicit install/update
• mixed projects containing repo-backed and URL-backed marketplaces

Definition of done:

• Tests cover both happy paths and the main failure paths.
• Existing repo-backed marketplace tests still pass.
• Update semantics are explicitly tested.
• Digest-first and hash-fallback behavior are explicitly tested.

15. Deliver in phases

Goal: keep the first merge small enough to review and ship.

Phase 1:

• Generalize marketplace source model.
• Support apm marketplace add https://example.com/.well-known/agent-skills/index.json --name workbooks.
• Parse well-known indexes.
• Enable browse/search for URL-backed marketplaces.
• Support skill-md install via name@marketplace.

Phase 2:

• Add lockfile provenance enhancements.
• Add policy/trust enforcement.
• Add digest verification.
• Add local content hash generation.
• Document and test update semantics.

Phase 3:

• Add archive support if deferred.
• Add origin shorthand such as --type well-known if deferred.
• Polish docs and compatibility behavior.

Acceptance Criteria

• A user can register a marketplace from a URL-based Agent Skills index.
• A user can run apm install code-review@workbooks after registering workbooks from a URL-based index.
• Existing repo-backed marketplace behavior continues to work unchanged.
• URL-backed marketplace installs reuse the existing APM install/deploy pipeline.
• apm.lock.yaml captures marketplace name plus backing URL provenance, advertised digest when present, and a local content hash.
• apm marketplace update <name> refreshes only marketplace metadata and does not silently upgrade already-installed skills.
• Explicit install/update flows use digest-first checks and local hash fallback to determine whether the artifact changed.
• Invalid indexes, unsupported entry types, bad digests, and untrusted origins fail with actionable errors.
• Documentation clearly explains the new marketplace registration flow and update behavior.

Open Questions

• Should --type well-known be required for origin URLs, or should APM infer it?
• Should both direct index URL and origin URL forms be supported in v1?
• Should URL-based marketplace sources be limited to well-known Agent Skills indexes in the first iteration, or generalized to arbitrary JSON indexes?
• Should first iteration support only skill-md, or also archives?
• Should strict Agent Skills validation apply only to URL-backed standards-based sources, or to all skill installs?

Notes on Current Code Fit

Relevant existing areas that appear reusable:

• marketplace caching and index parsing under src/apm_cli/marketplace/
• marketplace source modeling in src/apm_cli/marketplace/models.py
• marketplace registry in src/apm_cli/marketplace/registry.py
• marketplace resolution in src/apm_cli/marketplace/resolver.py
• dependency normalization in src/apm_cli/models/dependency/reference.py
• package validation in src/apm_cli/models/validation.py
• skill deployment in src/apm_cli/integration/skill_integrator.py
• install and resolver pipeline under src/apm_cli/commands/install.py and src/apm_cli/deps/
• policy infrastructure under src/apm_cli/policy/

This should be implemented as a generalization of the current marketplace model so the user-facing install experience remains name@marketplace regardless of whether the marketplace is backed by a Git repo or a URL-based discovery index.

[sergio-sisternes-epam]

Great design writeup, @Vicente-Pastor -- this covers UX, update semantics, digest handling, and lockfile provenance thoroughly.

A few notes for context:

• [FEATURE] Skills from .well-known URIs #554 is the original community request from @jonathanhefner for .well-known URI support. This issue supersedes it as the detailed design proposal -- we should close [FEATURE] Skills from .well-known URIs #554 as a duplicate pointing here once we confirm @jonathanhefner agrees.
• feat: Support Artifactory-hosted marketplace indexes #506 (closed) added Artifactory-hosted marketplace indexes. The URL-based backend proposed here generalizes that pattern further -- the implementation should build on the MarketplaceSource abstraction introduced there.
• The MCP .well-known standardization referenced in [FEATURE] Skills from .well-known URIs #554 (SEP-2127: MCP Server Cards - HTTP Server Discovery via .well-known modelcontextprotocol/modelcontextprotocol#2127) may also be relevant as a future extension point.

The design direction of treating this as a new marketplace backend (not a separate install model) is the right call -- it preserves the existing name@marketplace UX and avoids fragmenting the install surface.

[Vicente-Pastor]

Thanks @sergio-sisternes-epam! Good callout on #554 and #506 — I've been building directly on the MarketplaceSource abstraction from #506 as you suggested.

On the extension point you mentioned — I had actually planned ahead for this. The implementation in #691 was deliberately designed to keep the door open for direct URL-based package installs (not just marketplace discovery). I've opened #692 to track that next step explicitly, with the specific design constraints that #676 already satisfies:

• source_type is an open string (not a closed enum) — ready for "url-package" or similar
• _fetch_url_direct() is generic (plain HTTPS GET, no Agent Skills assumptions)
• _detect_index_format() is extensible for apm.yml detection
• LockedDependency uses neutral source_url / source_digest names (not marketplace-specific)
• Archive extraction (archive.py) is decoupled from marketplace logic, ready for reuse by the package installer

The MCP .well-known standardization you referenced (modelcontextprotocol/modelcontextprotocol#2127) would slot in naturally since we already resolve .well-known/agent-skills/index.json — adding another .well-known path would be a small extension.

See #692 for the full design constraints and capability roadmap.

[Vicente-Pastor]

Implementation Report — URL-based Marketplace Registration

Branch: feature/676-url-based-marketplace
PR: #691
Tests: 364 passing (11 test files — 7 new + 4 modified)
Approach: Strict TDD — every feature red→green→refactor, followed by deep code review + external review

---

Design Decisions

Core architecture: new marketplace backend, not a separate install model

URL-based sources are a new source_type on the existing MarketplaceSource model, preserving the skill@marketplace install UX. This builds directly on the abstraction from #506 (Artifactory) and avoids fragmenting the install surface.

• source_type is an open string ("github" / "url"), not a closed enum — future types like "url-package" slot in without refactoring (see [FEATURE] URL-based marketplace work Extension #692)
• owner/repo become optional (default "") for URL sources — no schema migration needed
• to_dict() omits GitHub-only fields for URL sources; from_dict() defaults to "github" for legacy dicts without source_type → full backward compat

URL detection strategy

URL detection happens before OWNER/REPO parsing in the add command — https:// prefix routes to the URL branch immediately. http:// is rejected with a clear HTTPS error. Scheme detection is case-insensitive (HTTPS://, Https:// all accepted).

.well-known auto-resolution

Bare origin URLs (e.g. https://example.com) are automatically resolved to /.well-known/agent-skills/index.json per RFC 8615. URLs that already contain a path are used as-is. Query strings are preserved (supports token-authenticated endpoints like https://cdn.corp.com?token=abc).

Agent Skills index parsing

parse_agent_skills_index() strictly enforces the RFC v0.2.0 schema:

• $schema field MUST be present and match https://schemas.agentskills.io/discovery/0.2.0/schema.json
• Skill names validated per RFC rules: 1-64 chars, lowercase alphanumeric + hyphens, no leading/trailing/consecutive hyphens
• Invalid entries (bad names, missing digests, non-dict entries) are logged and skipped, not fatal
• Both skill-md and archive types accepted

Format auto-detection

_detect_index_format(data) inspects top-level keys: "skills" → Agent Skills, "plugins" → legacy GitHub, else "unknown". _parse_manifest() is a single dispatcher used by all code paths (fresh fetch, cache hit, 304, stale fallback) — consolidating parsing logic was a key review finding.

Fetch layer: _fetch_url_direct()

Deliberately kept generic — plain HTTPS GET, no Agent Skills assumptions. This allows future callers (e.g. URL-based package installs per #692) to reuse it without importing marketplace internals.

• HTTPS scheme enforced at fetch time (not just command layer)
• Post-redirect validation: if a server redirects from HTTPS to HTTP, the redirect is rejected (prevents HTTPS→HTTP downgrade attacks)
• Returns FetchResult(data, digest, etag, last_modified) — frozen dataclass

Caching strategy

• Cache key for URL sources: sha256(url)[:16] (filesystem-safe, collision-resistant)
• Same 1-hour TTL as GitHub sources
• ETag/Last-Modified conditional refresh: sends If-None-Match/If-Modified-Since; on 304, resets TTL without network write
• Stale-while-revalidate: on network error, serves expired cache with warning (prevents hard failures from transient outages)

Digest / integrity verification

• sha256:<64 hex chars> computed on raw response bytes at fetch time
• Stored in .meta.json alongside cache data
• expected_digest parameter on _fetch_url_direct — extension point for lockfile-aware re-fetch
• Per-skill content-level digest stored in MarketplacePlugin.source["digest"] — verified at discovery, full verification deferred to install time

Lockfile provenance

LockedDependency gained source_url and source_digest fields (names intentionally format-neutral, not marketplace-specific) for future traceability and reproducible installs.

Archive support

marketplace/archive.py provides standalone extraction utilities:

• Formats: .tar.gz, .zip (auto-detected from magic bytes + extension)
• Safety: path traversal, absolute paths, Windows drive-letter/UNC paths, symlinks, hardlinks, null bytes, decompression bomb (512 MB limit)
• Decoupled from marketplace logic — reusable by future package installer

Resolver changes

• resolve_plugin_source() handles skill-md/archive types as parallel code paths
• Non-GitHub HTTPS URLs returned as-is (pass-through) instead of rejected
• Empty marketplace_owner guard: raises ValueError immediately rather than allowing downstream crashes

Error handling strategy

Three-tier error handling in the add command:

1. MarketplaceFetchError → displayed directly (fetch-layer message is already user-friendly)
2. ValueError → wrapped with "Invalid index format:" prefix (parser-layer validation)
3. All other exceptions → "Failed to register marketplace:" prefix (unexpected errors)

---

Implementation Steps

Step 1 — Extend MarketplaceSource model (gaps #1, #7)

• url: str = "", source_type: str = "github", is_url_source property
• to_dict()/from_dict() with backward compat
• 14 tests in test_marketplace_url_models.py

Step 2 — URL detection in marketplace add (gap #3)

• _resolve_index_url() helper, http:// rejection, name from hostname
• Tests in test_marketplace_url_commands.py

Step 3 — URL fetch path in client.py (gaps #2, #8)

• _cache_key() SHA-256, _fetch_url_direct(), _detect_index_format(), _parse_manifest()
• FetchResult frozen dataclass, fetch_marketplace() URL branch
• 18 tests in test_marketplace_url_client.py

Step 4 — Agent Skills index parser (gaps #5, #13, #15)

• parse_agent_skills_index(), $schema enforcement, _is_valid_skill_name()
• Tests in test_marketplace_url_models.py

Step 5 — Digest / integrity verification (gap #9)

• FetchResult.digest, _write_cache() stores digest, _is_valid_digest()
• LockedDependency.source_url / source_digest
• 27 tests across digest, lockfile provenance

Step 6 — ETag/conditional refresh (gap #10)

• If-None-Match/If-Modified-Since, 304 handling, _read_stale_meta()
• Stale-while-revalidate fallback
• Parametrized tests for conditional headers + cache

Step 7 — Archive type support (gap #14)

• archive.py: detection, extraction, safety guards
• 43 tests in test_marketplace_archive.py

Step 8 — Fix list/remove/browse/resolver (gaps #4, #6, #12)

• is_url_source display guards, clear_marketplace_cache(source=), resolver URL pass-through
• Tests in URL commands + resolver suites

---

External Review Summary

48 items resolved across bugs, design, tests, exceptions, comments, and naming. Highlights:

• S1: Post-redirect HTTPS enforcement (prevents HTTPS→HTTP downgrade)
• DR1: from_dict() raises ValueError on unknown source_type (fail-fast)
• DR4: Query string preserved in _resolve_index_url()
• DR6: Case-insensitive scheme detection
• All bug fixes, design improvements, docstrings, and naming cleanups verified with tests

5 architecture items deferred to a dedicated refactoring PR:

• N1: Rename archive.py → extraction.py
• N2: Move ArchiveError to errors.py, inherit from MarketplaceError
• N5: Move _resolve_index_url from commands to client.py
• N8: Move _WELL_KNOWN_PATH alongside _MARKETPLACE_PATHS
• G8: Move archive.py from marketplace/ to utils/

---

Test Inventory

File	Count	Coverage
test_marketplace_url_client.py	67	URL fetch, cache, digest, ETag, format detection, HTTPS enforcement, redirect validation
test_marketplace_url_models.py	61	URL source fields, Agent Skills parser, name validation, digest format
test_marketplace_resolver.py	45	All resolver paths (GitHub + URL), skill-md/archive types
test_marketplace_archive.py	43	Archive safety, extraction, format detection, decompression bomb
test_marketplace_models.py	33	GitHub models + parser (pre-existing)
test_marketplace_url_commands.py	29	URL add/list/remove/update, query string, case-insensitive scheme
test_marketplace_client.py	20	GitHub client paths (pre-existing, adapted for tuple return)
test_marketplace_url_resolver.py	15	URL source resolution, PluginNotFoundError
test_marketplace_commands.py	15	GitHub commands (pre-existing)
test_lockfile_provenance.py	14	Lockfile source_url/source_digest round-trips
test_marketplace_registry.py	10	Registry CRUD (pre-existing)
test_marketplace_install_integration.py	7	Install pre-parse, marketplace ref detection
test_marketplace_errors.py	5	Error class hierarchy (pre-existing)
Total	364	364 / 364 passing

---

Files Changed (vs main)

File	Type	Summary
src/apm_cli/marketplace/models.py	Modified	MarketplaceSource URL fields, parse_agent_skills_index(), _is_valid_skill_name(), _is_valid_digest(), manifest provenance fields
src/apm_cli/commands/marketplace.py	Modified	URL detection, _resolve_index_url(), list/remove/update URL display, error handling
src/apm_cli/marketplace/client.py	Modified	_fetch_url_direct(), FetchResult, _detect_index_format(), _parse_manifest(), _cache_key() SHA-256, _read_stale_meta(), ETag/conditional, redirect validation
src/apm_cli/marketplace/resolver.py	Modified	Non-GitHub URL pass-through, skill-md/archive types, empty URL guard
src/apm_cli/marketplace/archive.py	New	Archive detection, extraction, safety guards (path traversal, symlinks, decompression bomb)
src/apm_cli/deps/lockfile.py	Modified	source_url / source_digest on LockedDependency
tests/unit/marketplace/test_marketplace_url_models.py	New	61 tests for URL source model and Agent Skills parser
tests/unit/marketplace/test_marketplace_url_client.py	New	67 tests for URL fetch, cache, digest, format detection
tests/unit/marketplace/test_marketplace_url_commands.py	New	29 tests for URL commands
tests/unit/marketplace/test_marketplace_url_resolver.py	New	15 tests for URL resolver
tests/unit/marketplace/test_marketplace_archive.py	New	43 tests for archive safety and extraction
tests/unit/marketplace/test_lockfile_provenance.py	Modified	7 new provenance round-trip tests
tests/unit/marketplace/test_marketplace_client.py	Modified	Adapted for _read_cache() tuple return
tests/unit/marketplace/test_marketplace_resolver.py	Modified	Updated non-GitHub URL test

---

Future Extension

Issue #692 tracks extending this work to support installing full APM packages directly from a URL. The current implementation was designed to not foreclose this direction — see #692 for specific design constraints already satisfied.