Skip to content

Use crawler signatures#7907

Open
graygilmore wants to merge 5 commits into
mainfrom
gg-theme-dev-crawler-signatures
Open

Use crawler signatures#7907
graygilmore wants to merge 5 commits into
mainfrom
gg-theme-dev-crawler-signatures

Conversation

@graygilmore

@graygilmore graygilmore commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds support for Shopify-managed storefront request headers during local theme development in an effort to eliminate legitimate CLI users being flagged as a bot.

  • fetches reusable headers during theme dev startup when available
  • sends them with storefront render, proxy, session-cookie, and password-check requests
  • wires the same behavior into theme app extension previews
  • keeps this best-effort so local development continues if the headers cannot be fetched

Note

Does not work in combination with the --password flag.

@github-actions github-actions Bot added the no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. label Jun 23, 2026
@graygilmore graygilmore force-pushed the gg-theme-dev-crawler-signatures branch 2 times, most recently from 0187042 to d4b78c1 Compare June 23, 2026 21:43
@github-actions github-actions Bot added Area: @shopify/cli @shopify/cli package issues and removed no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. labels Jun 23, 2026
Comment thread .changeset/theme-dev-crawler-signatures.md Outdated
Crawler signature values are request secrets, so verbose request logging should treat Signature, Signature-Input, and Signature-Agent like other sensitive headers before the dev server starts sending them.
@graygilmore graygilmore force-pushed the gg-theme-dev-crawler-signatures branch 2 times, most recently from f8ff3b4 to 15f866b Compare July 2, 2026 20:36
@Shopify Shopify deleted a comment from github-actions Bot Jul 2, 2026
@Shopify Shopify deleted a comment from github-actions Bot Jul 2, 2026
@graygilmore

Copy link
Copy Markdown
Contributor Author

/snapit

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

🫰✨ Thanks @graygilmore! Your snapshot has been published to npm.

Test the snapshot by installing your package globally:

pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20260702204507

Caution

After installing, validate the version by running shopify version in your terminal.
If the versions don't match, you might have multiple global instances installed.
Use which shopify to find out which one you are running and uninstall it.

@graygilmore graygilmore force-pushed the gg-theme-dev-crawler-signatures branch 3 times, most recently from 18087d6 to 1187c62 Compare July 2, 2026 22:47
Some internal Admin GraphQL fields are only available on unstable. Let raw Admin requests opt into a specific version while preserving the existing latest-supported-version behavior by default.
Add a theme helper that reuses an active CLI-managed crawler signature for the storefront domain or creates one through the internal Admin mutation when needed. The helper returns only the request headers the dev server needs.
Thread CLI crawler signature headers through the theme dev session so render requests, proxied storefront requests, session-cookie requests, and password checks all reach Storefront Renderer as verified Shopify crawler traffic.
Use the same Admin-backed crawler signature helper in app dev theme extension previews and pass the resulting headers into the shared theme dev server session.

Assisted-By: devx/1fdba809-4102-4cb7-a3de-1ab7dd624a29
@graygilmore graygilmore force-pushed the gg-theme-dev-crawler-signatures branch from 1187c62 to 92d78d2 Compare July 2, 2026 23:22
@graygilmore graygilmore marked this pull request as ready for review July 2, 2026 23:23
@graygilmore graygilmore requested review from a team as code owners July 2, 2026 23:23
@graygilmore graygilmore requested a review from EvilGenius13 July 2, 2026 23:23

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to define these types ourselves because these fields are hidden and only available to our client so they don't get automatically generated like our other GraphQL queries/mutations do.

const storefrontPasswordPromise = await isStorefrontPasswordProtected(options.adminSession).then((needsPassword) =>
needsPassword ? ensureValidPassword(options.storePassword, options.adminSession.storeFqdn) : undefined,
)
const [crawlerSignatureHeaders, isPasswordProtected] = await Promise.all([

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run these both at the same time since one doesn't block the other.

headers = cleanHeader({
...headers,
...defaultHeaders(),
...(host === ctx.session.storeFqdn ? ctx.session.crawlerSignatureHeaders : {}),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This proxy handles a few kinds of requests so we only want to add the headers when the hosts match.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/api/admin.d.ts
@@ -9,9 +9,10 @@ import { TypedDocumentNode } from '@graphql-typed-document-node/core';
  * @param query - GraphQL query to execute.
  * @param session - Shopify admin session including token and Store FQDN.
  * @param variables - GraphQL variables to pass to the query.
+ * @param version - API version.
  * @returns The response of the query of generic type <T>.
  */
-export declare function adminRequest<T>(query: string, session: AdminSession, variables?: GraphQLVariables): Promise<T>;
+export declare function adminRequest<T>(query: string, session: AdminSession, variables?: GraphQLVariables, version?: string): Promise<T>;
 export interface AdminRequestOptions<TResult, TVariables extends Variables> {
     /** GraphQL query to execute. */
     query: TypedDocumentNode<TResult, TVariables>;

@EvilGenius13 EvilGenius13 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the latest snap build confirmed everything is still working and can find the necessary info in our logs. 🚀 Thanks for the PR @graygilmore

@graygilmore graygilmore requested a review from isaacroldan July 3, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: @shopify/cli @shopify/cli package issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants