Skip to content

feature/add shopify app get-review-requirements command#7707

Draft
brandonreid wants to merge 1 commit into
mainfrom
feature/add_shopify_app_get-review-requirements_command
Draft

feature/add shopify app get-review-requirements command#7707
brandonreid wants to merge 1 commit into
mainfrom
feature/add_shopify_app_get-review-requirements_command

Conversation

@brandonreid
Copy link
Copy Markdown
Contributor

@brandonreid brandonreid commented Jun 3, 2026

Warning

We'll need an actual monorail event created before this can be reviewed and merged.

WHY are these changes introduced?

The AI Toolkit ships a shopify-app-review agent skill that walks a
developer's local codebase against the App Store review requirements before
they submit. Today the skill instructs the agent to web-fetch the
requirements directly from
https://shopify.dev/docs/apps/launch/app-store-review/app-store-ai-self-review-requirements.

That works, but it gives us no signal about which apps have actually
attempted a local self-review. We'd like that signal so we can:

  • track adoption of the skill,
  • potentially use it as a trust input when ranking apps for human review.

Since developers building for the App Store already have the Shopify CLI
installed, routing the requirements fetch through a CLI command gives us
the auto-emitted Monorail event (with local app attribution and the
standard analytics opt-out) for free.

WHAT is this pull request doing?

Adds a new command:

shopify app get-review-requirements

It fetches the canonical App Store review requirements (including the agent
verification guidance) from shopify.dev and prints them to stdout as
markdown — the format the agent skill needs to evaluate the codebase.

Key behaviors:

  • Extends BaseCommand (not AppLinkedCommand) — no auth, no remote
    linking, works inside or outside an app directory so agents and humans
    can call it freely.
  • When run inside an app directory, the local shopify.app.toml is loaded
    best-effort so the Monorail event carries app-level attribution
    (api_key from the local client_id, app_name_hash, app_path_hash,
    app_scopes, project_type, …). Outside an app dir the event is still
    emitted, just without app_* fields.
  • One new public Monorail field, cmd_app_review_requirements_fetch_status,
    so we can distinguish successful fetches from network/HTTP failures
    without parsing error messages.
  • Standard CLI analytics opt-out (SHOPIFY_CLI_NO_ANALYTICS /
    OPT_OUT_INSTRUMENTATION) applies.

Not in this PR: updating the shopify-app-review skill in
Shopify/ai-toolkit-source to shell out to this command instead of
web-fetching directly. That'll be a follow-up once this lands and is
released.

Files of interest:

  • packages/app/src/cli/commands/app/get-review-requirements.ts — command
  • packages/app/src/cli/services/app/get-review-requirements.ts — fetch + best-effort local app load
  • packages/cli-kit/src/public/node/monorail.ts — new schema field
  • packages/app/src/cli/index.ts — command registration

How to test your changes?

From inside an app directory:

shopify app get-review-requirements | head -20

You should see the requirements markdown printed to stdout, starting with
Section 1. Policy.

To verify the Monorail payload locally:

SHOPIFY_CLI_ALWAYS_LOG_ANALYTICS=true OPT_OUT_INSTRUMENTATION=true \
  shopify app get-review-requirements --verbose 2>&1 >/dev/null \
  | awk '/Analytics event sent/,/^}/'

Inside a linked app dir you should see api_key, app_name_hash,
app_path_hash, app_scopes, project_type and
cmd_app_review_requirements_fetch_status: 200 in the payload. From a
non-app directory the same command runs cleanly but emits the event
without the app_* fields.

Opt-out:

OPT_OUT_INSTRUMENTATION=true shopify app get-review-requirements --verbose 2>&1 \
  | grep "Skipping command analytics"

Post-release steps

Open a follow-up PR in Shopify/ai-toolkit-source to update the
shopify-app-review skill to invoke shopify app get-review-requirements instead of fetching the requirements page
directly.

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is user-facing — I've identified the correct bump type (patch for bug fixes · minor for new features · major for breaking changes) and added a changeset with pnpm changeset add

@brandonreid brandonreid requested a review from a team as a code owner June 3, 2026 16:35
Copilot AI review requested due to automatic review settings June 3, 2026 16:35
@brandonreid brandonreid requested a review from a team as a code owner June 3, 2026 16:35
@github-actions github-actions Bot added the Area: @shopify/cli @shopify/cli package issues label Jun 3, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/monorail.d.ts
@@ -76,6 +76,7 @@ export interface Schemas {
             cmd_app_validate_valid?: Optional<boolean>;
             cmd_app_validate_issue_count?: Optional<number>;
             cmd_app_validate_file_count?: Optional<number>;
+            cmd_app_review_requirements_fetch_status?: Optional<number>;
             cmd_dev_tunnel_type?: Optional<string>;
             cmd_dev_tunnel_custom_hash?: Optional<string>;
             cmd_dev_urls_updated?: Optional<boolean>;

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Shopify CLI command to fetch and print the App Store AI self-review requirements (including agent verification guidance) as markdown, with basic Monorail attribution/telemetry to support tooling that evaluates a local app codebase against the latest requirements.

Changes:

  • Introduces shopify app get-review-requirements command and service to fetch the requirements doc from shopify.dev and print it to stdout.
  • Records fetch status in Monorail metadata and attempts best-effort app attribution (api_key) for analytics.
  • Updates CLI docs/manifest and adds a changeset for release versioning.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/cli/README.md Documents the new shopify app get-review-requirements command in the CLI README.
packages/cli/oclif.manifest.json Registers the new command in the oclif manifest (help/metadata output).
packages/cli-kit/src/public/node/monorail.ts Adds a new Monorail schema field for recording fetch HTTP status.
packages/app/src/cli/services/app/get-review-requirements.ts Implements the underlying fetch + metadata attribution logic.
packages/app/src/cli/index.ts Wires the new command into the app CLI command map.
packages/app/src/cli/commands/app/get-review-requirements.ts Adds the new oclif command implementation and flags.
.changeset/app-get-review-requirements.md Declares release bumps for @shopify/app and @shopify/cli-kit.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/cli/README.md
(https://shopify.dev/docs/apps/launch/app-store-review/app-store-ai-self-review-requirements) to stdout as markdown,
including the agent verification guidance an AI agent needs to evaluate each requirement against a local codebase.

Designed to be invoked by the `shopify-app-review` agent skill from the Shopify AI Toolkit as part of a local
Comment on lines +27 to +31
export async function getReviewRequirements(options: GetReviewRequirementsOptions): Promise<string> {
await loadAppForAttribution(options)

const markdown = await fetchRequirementsMarkdown()
return markdown
@brandonreid brandonreid marked this pull request as draft June 4, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: @shopify/cli @shopify/cli package issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants