-
Notifications
You must be signed in to change notification settings - Fork 1.8k
fix: Add more A/B testing examples #1490
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
arnavnagzirkar
wants to merge
1
commit into
vercel:main
Choose a base branch
from
arnavnagzirkar:fix-27
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| NEXT_PUBLIC_LD_CLIENT_SIDE_ID= | ||
| EDGE_CONFIG= |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "root": true, | ||
| "extends": "next/core-web-vitals" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # See https://help.github.com/articles/ignoring-files/ for more about ignoring files. | ||
|
|
||
| # Dependencies | ||
| /node_modules | ||
| /.pnp | ||
| .pnp.js | ||
|
|
||
| # Testing | ||
| /coverage | ||
|
|
||
| # Next.js | ||
| /.next/ | ||
| /out/ | ||
|
|
||
| # Production | ||
| /build | ||
|
|
||
| # Misc | ||
| .DS_Store | ||
| *.pem | ||
|
|
||
| # Debug | ||
| npm-debug.log* | ||
| yarn-debug.log* | ||
| yarn-error.log* | ||
|
|
||
| # Local ENV files | ||
| .env.local | ||
| .env.development.local | ||
| .env.test.local | ||
| .env.production.local | ||
|
|
||
| # Vercel | ||
| .vercel | ||
|
|
||
| # Turborepo | ||
| .turbo | ||
|
|
||
| # typescript | ||
| *.tsbuildinfo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # Enabled to avoid deps failing to use next@canary | ||
| legacy-peer-deps=true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| --- | ||
| name: A/B Testing with LaunchDarkly | ||
| slug: ab-testing-launchdarkly | ||
| description: Reduce CLS and improve performance from client-loaded experiments at the edge with LaunchDarkly | ||
| framework: Next.js | ||
| useCase: | ||
| - Edge Config | ||
| - Edge Middleware | ||
| css: Tailwind | ||
| deployUrl: https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fvercel%2Fexamples%2Ftree%2Fmain%2Fedge-middleware%2Fab-testing-launchdarkly&project-name=ab-testing-launchdarkly&repository-name=ab-testing-launchdarkly&integration-ids=oac_8DFUMlauSkqeQhdGHpL5xbWp&env=NEXT_PUBLIC_LD_CLIENT_SIDE_ID,EDGE_CONFIG&envDescription=LaunchDarkly%20client-side%20ID%20and%20Edge%20Config%20connection%20string&envLink=https%3A%2F%2Fdocs.launchdarkly.com%2Fhome%2Fgetting-started | ||
| demoUrl: https://ab-testing-launchdarkly.vercel.app | ||
| relatedTemplates: | ||
| - ab-testing-simple | ||
| - ab-testing-statsig | ||
| - feature-flag-launchdarkly | ||
| --- | ||
|
|
||
| # A/B Testing with LaunchDarkly | ||
|
|
||
| This example shows how to do A/B testing at the edge using [LaunchDarkly](https://launchdarkly.com) and [Edge Config](https://vercel.com/docs/storage/edge-config). Experiment variants are assigned in Edge Middleware, so there is no client-side flicker and no extra round trip. | ||
|
|
||
| ## Demo | ||
|
|
||
| https://ab-testing-launchdarkly.vercel.app | ||
|
|
||
| ## How to Use | ||
|
|
||
| You can choose from one of the following two methods to use this repository: | ||
|
|
||
| ### One-Click Deploy | ||
|
|
||
| **Note:** Before clicking `Deploy`, complete the [Set up LaunchDarkly](#set-up-launchdarkly) section below to create a flag and obtain your client-side SDK key. | ||
|
|
||
| [](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fvercel%2Fexamples%2Ftree%2Fmain%2Fedge-middleware%2Fab-testing-launchdarkly&project-name=ab-testing-launchdarkly&repository-name=ab-testing-launchdarkly&integration-ids=oac_8DFUMlauSkqeQhdGHpL5xbWp&env=NEXT_PUBLIC_LD_CLIENT_SIDE_ID,EDGE_CONFIG&envDescription=LaunchDarkly%20client-side%20ID%20and%20Edge%20Config%20connection%20string&envLink=https%3A%2F%2Fdocs.launchdarkly.com%2Fhome%2Fgetting-started) | ||
|
|
||
| ### Clone and Deploy | ||
|
|
||
| Execute [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app) with [pnpm](https://pnpm.io/installation) to bootstrap the example: | ||
|
|
||
| ```bash | ||
| pnpm create next-app --example https://github.com/vercel/examples/tree/main/edge-middleware/ab-testing-launchdarkly ab-testing-launchdarkly | ||
| ``` | ||
|
|
||
| #### Set up environment variables | ||
|
|
||
| Copy [.env.example](./.env.example) to `.env.local`: | ||
|
|
||
| ```bash | ||
| cp .env.example .env.local | ||
| ``` | ||
|
|
||
| Fill in the variables as described in the next section. | ||
|
|
||
| #### Set up LaunchDarkly | ||
|
|
||
| 1. Create a free account at [launchdarkly.com](https://launchdarkly.com). | ||
| 2. In the LaunchDarkly dashboard, go to **Feature flags** and click **Create flag**. | ||
| 3. Set the flag key to `ab-testing-example`. | ||
| 4. Choose **String** as the flag type and add two variations: | ||
| - `control` (default off variation) | ||
| - `experiment` (default on variation) | ||
| 5. Set up a targeting rule that splits traffic 50/50 between `control` and `experiment`. | ||
| 6. Turn the flag on for your environment. | ||
| 7. Go to **Account settings > Projects**, select your project, and copy the **Client-side ID** for your environment. | ||
| 8. Add it to `.env.local` as `NEXT_PUBLIC_LD_CLIENT_SIDE_ID`. | ||
|
|
||
| #### Set up the LaunchDarkly Vercel Integration | ||
|
|
||
| Install the [LaunchDarkly Vercel Integration](https://vercel.com/integrations/launchdarkly) for your project. The integration syncs LaunchDarkly flag rules into Edge Config so they can be read at the edge without a network call to LaunchDarkly. | ||
|
|
||
| Once installed, set the `EDGE_CONFIG` environment variable in `.env.local` with the Edge Config connection string provided by the integration. | ||
|
|
||
| Next, run Next.js in development mode: | ||
|
|
||
| ```bash | ||
| pnpm dev | ||
| ``` | ||
|
|
||
| Deploy it to the cloud with [Vercel](https://vercel.com/new?utm_source=github&utm_medium=readme&utm_campaign=edge-middleware-eap) ([Documentation](https://nextjs.org/docs/deployment)). | ||
|
|
||
| ## How it works | ||
|
|
||
| On each request to `/`, the middleware: | ||
|
|
||
| 1. Reads (or generates) a user ID from a cookie. | ||
| 2. Initializes the LaunchDarkly SDK using flag rules cached in Edge Config. | ||
| 3. Calls `variation()` to get the assigned bucket (`control` or `experiment`). | ||
| 4. Rewrites the request to `/{bucket}`. | ||
|
|
||
| Because the flag rules are read from Edge Config rather than fetched from LaunchDarkly on every request, the evaluation adds minimal latency. The user ID cookie ensures the same user always sees the same variant. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| export const UID_COOKIE = 'uid' | ||
| export const EXPERIMENT_FLAG_KEY = 'ab-testing-example' | ||
| export const BUCKETS = ['control', 'experiment'] as const | ||
| export type Bucket = (typeof BUCKETS)[number] | ||
| export const FALLBACK_BUCKET: Bucket = 'control' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| import { NextRequest, NextResponse } from 'next/server' | ||
| import { init } from '@launchdarkly/vercel-server-sdk' | ||
| import { createClient } from '@vercel/edge-config' | ||
| import { | ||
| UID_COOKIE, | ||
| EXPERIMENT_FLAG_KEY, | ||
| FALLBACK_BUCKET, | ||
| BUCKETS, | ||
| type Bucket, | ||
| } from './lib/constants' | ||
|
|
||
| const IS_UUID = /^[0-9a-f-]+$/i | ||
|
|
||
| const edgeConfigClient = createClient(process.env.EDGE_CONFIG) | ||
|
|
||
| export const config = { | ||
| matcher: '/', | ||
| } | ||
|
|
||
| export async function middleware(req: NextRequest) { | ||
| let userId = req.cookies.get(UID_COOKIE)?.value | ||
| let hasUserId = !!userId | ||
|
|
||
| if (!userId || !IS_UUID.test(userId)) { | ||
| userId = crypto.randomUUID() | ||
| hasUserId = false | ||
| } | ||
|
|
||
| let bucket: string = FALLBACK_BUCKET | ||
|
|
||
| try { | ||
| // Create a fresh LaunchDarkly client per request to avoid shared promise issues | ||
| // across requests in Edge Middleware (unlike Edge Functions, cache() is not available) | ||
| const ldClient = init( | ||
| process.env.NEXT_PUBLIC_LD_CLIENT_SIDE_ID!, | ||
| edgeConfigClient | ||
| ) | ||
| await ldClient.waitForInitialization() | ||
|
|
||
| const context = { kind: 'user', key: userId } | ||
| const flagValue = await ldClient.variation( | ||
| EXPERIMENT_FLAG_KEY, | ||
| context, | ||
| FALLBACK_BUCKET | ||
| ) | ||
|
|
||
| // Validate the returned value is a known bucket to avoid rewriting to a missing route | ||
| if (typeof flagValue === 'string' && BUCKETS.includes(flagValue as Bucket)) { | ||
| bucket = flagValue | ||
| } | ||
| } catch { | ||
| // Fall back to the default bucket if LaunchDarkly fails | ||
| bucket = FALLBACK_BUCKET | ||
| } | ||
|
|
||
| const url = req.nextUrl.clone() | ||
| url.pathname = `/${bucket}` | ||
| const res = NextResponse.rewrite(url) | ||
|
|
||
| if (!hasUserId) { | ||
| res.cookies.set(UID_COOKIE, userId, { | ||
| maxAge: 60 * 60 * 24, | ||
| }) | ||
| } | ||
|
|
||
| return res | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| /// <reference types="next" /> | ||
| /// <reference types="next/image-types/global" /> | ||
|
|
||
| // NOTE: This file should not be edited | ||
| // see https://nextjs.org/docs/pages/api-reference/config/typescript for more information. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| { | ||
| "name": "ab-testing-launchdarkly", | ||
| "repository": "https://github.com/vercel/examples.git", | ||
| "license": "MIT", | ||
| "private": true, | ||
| "scripts": { | ||
| "dev": "next dev", | ||
| "build": "next build", | ||
| "start": "next start", | ||
| "lint": "next lint" | ||
| }, | ||
| "dependencies": { | ||
| "@launchdarkly/vercel-server-sdk": "^1.3.3", | ||
| "@vercel/edge-config": "^1.1.0", | ||
| "@vercel/examples-ui": "^1.0.5", | ||
| "js-cookie": "^3.0.5", | ||
| "next": "^16.0.10", | ||
| "react": "^19.2.1", | ||
| "react-dom": "^19.2.1" | ||
| }, | ||
| "devDependencies": { | ||
| "@types/js-cookie": "^3.0.6", | ||
| "@types/node": "^20", | ||
| "@types/react": "^19", | ||
| "autoprefixer": "^10.4.15", | ||
| "eslint": "^8.47.0", | ||
| "eslint-config-next": "^13.4.19", | ||
| "postcss": "^8.4.28", | ||
| "tailwindcss": "^3.3.3", | ||
| "turbo": "^1.10.12", | ||
| "typescript": "^5" | ||
| } | ||
| } |
67 changes: 67 additions & 0 deletions
67
edge-middleware/ab-testing-launchdarkly/pages/[bucket].tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| import { GetStaticPaths, GetStaticProps } from 'next' | ||
| import { useRouter } from 'next/router' | ||
| import Cookie from 'js-cookie' | ||
| import { Layout, Text, Page, Button, Code } from '@vercel/examples-ui' | ||
| import { BUCKETS, UID_COOKIE, EXPERIMENT_FLAG_KEY } from '../lib/constants' | ||
|
|
||
| interface Props { | ||
| bucket: string | ||
| } | ||
|
|
||
| export const getStaticProps: GetStaticProps<Props> = async ({ params }) => { | ||
| return { | ||
| props: { | ||
| bucket: params?.bucket as string, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| export const getStaticPaths: GetStaticPaths<{ bucket: string }> = async () => { | ||
| return { | ||
| paths: BUCKETS.map((bucket) => ({ params: { bucket } })), | ||
| fallback: false, | ||
| } | ||
| } | ||
|
|
||
| function BucketPage({ bucket }: Props) { | ||
| const { reload } = useRouter() | ||
|
|
||
| function resetBucket() { | ||
| Cookie.remove(UID_COOKIE) | ||
| reload() | ||
| } | ||
|
|
||
| return ( | ||
| <Page className="flex flex-col gap-12"> | ||
| <section className="flex flex-col gap-6"> | ||
| <Text variant="h1">A/B Testing with LaunchDarkly</Text> | ||
| <Text> | ||
| This example uses LaunchDarkly's Edge Config integration to | ||
| assign users to experiment buckets at the edge. The assignment happens | ||
| in <Code>middleware.ts</Code> and is based on a user ID stored in a | ||
| cookie. | ||
| </Text> | ||
| <Text> | ||
| Buckets are statically generated at build time so rewrites are fast. | ||
| The middleware evaluates the <Code>{EXPERIMENT_FLAG_KEY}</Code> flag | ||
| and rewrites the request to the matching bucket page. | ||
| </Text> | ||
| <pre className="bg-black text-white font-mono text-left py-2 px-4 rounded-lg text-sm leading-6"> | ||
| bucket: {bucket} | ||
| </pre> | ||
| <Button size="lg" onClick={resetBucket}> | ||
| Reset bucket | ||
| </Button> | ||
| <Text> | ||
| Clicking reset clears your user ID cookie. On the next visit, | ||
| LaunchDarkly will assign you to a bucket again based on the flag | ||
| configuration in your LaunchDarkly project. | ||
| </Text> | ||
| </section> | ||
| </Page> | ||
| ) | ||
| } | ||
|
|
||
| BucketPage.Layout = Layout | ||
|
|
||
| export default BucketPage | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| import type { AppProps } from 'next/app' | ||
| import type { LayoutProps } from '@vercel/examples-ui/layout' | ||
| import { getLayout } from '@vercel/examples-ui' | ||
| import '@vercel/examples-ui/globals.css' | ||
|
|
||
| export default function MyApp({ Component, pageProps }: AppProps) { | ||
| const Layout = getLayout<LayoutProps>(Component) | ||
|
|
||
| return ( | ||
| <Layout | ||
| title="A/B Testing with LaunchDarkly" | ||
| path="edge-middleware/ab-testing-launchdarkly" | ||
| > | ||
| <Component {...pageProps} /> | ||
| </Layout> | ||
| ) | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.