Possible adjustments to multipage example

advanced/multipage-testing-example/.env.example

@@ -1,3 +1,5 @@
-FEATURE_FLAG_NAME=YOUR_FEATURE_FLAG_NAME
 CLIENT_SIDE_SDK_KEY=YOUR_CLIENT_SIDE_SDK_KEY
 SERVER_SIDE_SDK_KEY=YOUR_SERVER_SIDE_SDK_KEY
+FEATURE_FLAG_OPTIMIZE_PAGE=page_optimization
+FEATURE_FLAG_NETWORK_SPEED=network_speed
+FEATURE_FLAG_IMAGE_SIZE=image_size

advanced/multipage-testing-example/README.md

@@ -1,6 +1,6 @@
-The example consists of an A/B test comparing the performance of two variants of a web page. A NodeJS server serves these variants behind a feature flag using Split's NodeJS SDK. The web pages are built with Webpack and utilize the Split's RUM Agent to capture performance metrics such as page load time and Web-vitals.
+The example consists of an A/B test comparing the performance of two variants of a webpage. A NodeJS server serves these variants behind a feature flag using Split's NodeJS SDK. The webpages are built with Webpack and utilize Split RUM Agent to capture performance metrics such as page load time and Web Vitals.
 
-One of the variants (treatment 'on') includes performance optimizations, while the other (treatment 'off') doesn't. For example, the 'on' variant loads the RUM agent using dynamic imports and is built with Webpack's "production" mode, which minifies the code. In contrast, the 'off' variant loads the RUM Agent synchronously, blocking the page load, and is built with Webpack's "development" mode that doesn't minifies the code by default.
+One of the webpage variants (treatment 'on') includes performance optimizations, while the other (treatment 'off') doesn't. For example, the 'on' variant loads the RUM agent using dynamic imports and is built with Webpack's "production" mode, which minifies the code. In contrast, the 'off' variant loads the RUM Agent synchronously, blocking the page load, and is built with Webpack's "development" mode that doesn't minify the code by default.
 
 An automation script navigates the page multiple times, generating events and impressions for both treatments.
 
@@ -9,19 +9,19 @@ An automation script navigates the page multiple times, generating events and im
 This example assumes you have set up a feature flag in an environment, with traffic type 'user' and two treatments: 'on' and 'off'.
 
 1. Take a copy of `.env.example` and re-name to `.env`.
-2. Add your Split SDK keys and feature flag name to `.env`.
+2. Add your Split SDK keys and feature flag names to `.env`.
 3. Run `npm install` to install dependencies.
 4. Run `npm run serve` to build the app and start the server. The Web page will be served at `http://localhost:3000/?id=<user-id>`, where `<user-id>` is a unique identifier for the user, used by the Split SDK to bucket the user into a treatment.
-5. Run `npm run automation` to run the automation script. The script can take a few minutes to complete, as it will generate events and impressions by navigating to the Web page multiple times with different user IDs, using [Puppeteer and Chrome Headless](https://www.npmjs.com/package/puppeteer).
-6. Open the Split UI, create metrics associated to the events captured by the RUM Agent, and analyze the results. For example, in the Create Metric panel, you can create a metric for page load time, by selecting traffic type `user`, event type `page.load.time`, desired impact "Decrease", and measured as "Average of event values per user". See more about ["Creating a Metric"](https://help.split.io/hc/en-us/articles/360020586132-Creating-a-metric) and ["Metrics impact tab"](https://help.split.io/hc/en-us/articles/360020844451-Metrics-impact-tab).
+5. Run `npm run automation` to run the automation script. The script can take some time to complete, as it will generate events and impressions by navigating to the webpage multiple times with different user IDs, using [Puppeteer and Chrome](https://www.npmjs.com/package/puppeteer). You can grab a coffee. :)
+6. Open the Split UI, click on a feature flag's Metrics impact tab, and analyze the metric results. You can click 'View more' on a Metric card to visualize metric measurements. See more about the ["Metrics impact tab"](https://help.split.io/hc/en-us/articles/360020844451-Metrics-impact-tab).
 
 ![Split UI](./screenshot.png)
 
 # Contents
 
-- `/client`: source code of the Web application and its two variants.
-  - `/client/index-on.js`: entry point for the optimized variant, served for treatment `on`.
-  - `/client/index-off.js`: entry point for the default variant, served for treatment `off`.
+- `/client`: source code of the web application and its two variants.
+  - `/client/index-on.js`: entry point for the optimized variant, served for treatment `on` of the Split feature flag provided to the `FEATURE_FLAG_OPTIMIZE_PAGE` variable (after creating this flag in the Split UI, you provide the flag name in your `.env` file).
+  - `/client/index-off.js`: entry point for the default variant, served for treatment `off` of the `FEATURE_FLAG_OPTIMIZE_PAGE` Split feature flag.
 - `/webpack.config.js`: Webpack configuration to build the two variants of the application.
 - `/dist`: built static assets of the application, generated by Webpack.
 - `/server/index.js`: source code of the NodeJS server that implements the endpoint `GET /?id=<user-id>` that serves the two variants of the application behind a feature flag.
@@ -33,3 +33,7 @@ This example assumes you have set up a feature flag in an environment, with traf
 - `npm run build`: builds the two variants of the application.
 - `npm run serve`: builds the two variants of the application and starts the NodeJS server.
 - `npm run automation`: runs the automation script.
+
+# Tutorial
+
+This version of the code was used to create the [blog](https://www.split.io/blog/): 'Instant feature impact detection for webpage performance: Split’s hidden gem'. Since you are already peering into the code, check it out and see Split's powerful IFID capabilities!

advanced/multipage-testing-example/automation.js

@@ -1,6 +1,19 @@
+require('dotenv').config();
 const puppeteer = require('puppeteer');
 
-const SAMPLE_SIZE = 500;
+const getSplitClient = require('./server/split.js');
+const splitClient = getSplitClient();
+
+
+// You can optionally pass the loop variables on the command line, e.g. `npm run automation 0 200`
+let [i, SAMPLE_SIZE] = process.argv.slice(2,4).map(n => parseInt(n, 10));
+// Validate input
+if ( !( i <= SAMPLE_SIZE ) ) {
+  console.log(`Found invalid command line arguments '${i}' and '${SAMPLE_SIZE}'. Using default values instead.`);
+  i = 0;
+  SAMPLE_SIZE = 200;
+}
+
 
 // https://fdalvi.github.io/blog/2018-02-05-puppeteer-network-throttle/
 const NETWORK_CONDITIONS = {
@@ -27,32 +40,47 @@ const NETWORK_CONDITIONS = {
   }
 };
 
+
 (async () => {
   console.log('Navigation script');
 
-  // Launch browser
-  const browser = await puppeteer.launch();
+  // Launch browser, use headful mode to allow Cumulative Layout Shift (CLS) measurment
+  const browser = await puppeteer.launch({headless: false, defaultViewport: null});
 
-  for (let id = 0; id < SAMPLE_SIZE; id++) {
-    console.log(`Running ${id} of ${SAMPLE_SIZE}`);
+  for (; i < SAMPLE_SIZE; i++) {
+    console.log(`Running ${i} of ${SAMPLE_SIZE}`);
 
     // Open new page
     const page = await browser.newPage();
 
     // Emulate network conditions and disable cache to simulate new users
     await page.setCacheEnabled(false);
-    await page.emulateNetworkConditions(NETWORK_CONDITIONS.Good3G);
+
+    // Evaluate Split flag to determine what the emulated network conditions should be for this user
+    const networkSpeed  = splitClient.getTreatment(i, process.env.FEATURE_FLAG_NETWORK_SPEED);
+
+    await page.emulateNetworkConditions(
+      (networkSpeed in NETWORK_CONDITIONS)
+      ? NETWORK_CONDITIONS[networkSpeed]
+      : NETWORK_CONDITIONS.Good3G
+    );
 
     // Navigate to URL
-    await page.goto(`http://localhost:3000/?id=${id}`);
+    await page.goto (`http://localhost:3000/?id=${i}` , {waitUntil: "networkidle0", timeout: 0}); // Disabled timeout to avoid exception being thrown. If, however, the page gets 'stuck', click the refresh button.
 
-    // Perform click on an element
+    // Click on an element to start measuring First Input Delay (FID) and Interaction to Next Paint (INP) time
     await page.click('#split_logo');
 
-    // Wait some time
+    // Pause to allow time for the FID and INP measurement
     await new Promise(resolve => setTimeout(resolve, 1000));
+
+    // Close the tab so that the CLS and INP measurements are sent
+    await page.close();
   }
 
+  // Pause to allow the browser time to send the last CLS measurement
+  await new Promise(resolve => setTimeout(resolve, 1000));
+
   // Close browser
   await browser.close();
 

advanced/multipage-testing-example/client/index-off.js

@@ -2,7 +2,10 @@
 import { SplitRumAgent, webVitals } from '@splitsoftware/browser-rum-agent';
 
 SplitRumAgent
-  .setup(process.env.CLIENT_SIDE_SDK_KEY)
+  .setup(process.env.CLIENT_SIDE_SDK_KEY,
+    // set 2 second pushRate so we can view events in realtime in Split Data hub
+    { pushRate: 2 }
+  )
   .addIdentities([
     // get key from URL query parameter `id`
     { key: new URLSearchParams(window.location.search).get('id'), trafficType: 'user' }

advanced/multipage-testing-example/client/index-on.js

@@ -2,7 +2,10 @@
 import('./browser-rum-agent').then(({ SplitRumAgent, webVitals }) => {
 
   SplitRumAgent
-    .setup(process.env.CLIENT_SIDE_SDK_KEY)
+    .setup(process.env.CLIENT_SIDE_SDK_KEY,
+      // set 2 second pushRate so we can view events in realtime in Split Data hub
+      { pushRate: 2 }
+    )
     .addIdentities([
       // get key from URL query parameter `id`
       { key: new URLSearchParams(window.location.search).get('id'), trafficType: 'user' }

advanced/multipage-testing-example/client/index.html

@@ -13,6 +13,16 @@
       w.addEventListener('unhandledrejection', g.l2);
     }(window))
   </script>
+  <script>
+    window.addEventListener('load', function () {
+      let imageSize = new URLSearchParams(window.location.search).get('img');
+
+      // if the query parameter is not one of the the imgur size modifiers, then don't use it
+      if( ! ['b', 's', 't', 'm', 'l', 'h'].includes(imageSize) ) imageSize = '';
+
+      document.getElementById('street_img').src = "https://i.imgur.com/q9b5x97" + imageSize + ".png";
+    });
+  </script>
 </head>
 
 <body>
@@ -36,7 +46,7 @@ <h1><%= htmlWebpackPlugin.options.title %></h1>
     Pharetra convallis posuere morbi leo urna molestie. Duis ultricies lacus sed turpis tincidunt id aliquet. Bibendum
     enim facilisis gravida neque. Gravida quis blandit turpis cursus in hac habitasse platea.
   </p>
-  <img src="https://i.imgur.com/q9b5x97l.png" style="max-height: 400px;" />
+  <img id="street_img" />
 
   <!-- imgur embed widget -->
   <blockquote class="imgur-embed-pub" lang="en" data-id="a/8aOpHV6">

advanced/multipage-testing-example/server/index.js

@@ -3,27 +3,24 @@ const path = require('path');
 const express = require('express');
 const app = express();
 
-const { SplitFactory } = require('@splitsoftware/splitio');
-
-const client = SplitFactory({
-  core: {
-    authorizationKey: process.env.SERVER_SIDE_SDK_KEY,
-  },
-  debug: 'ERROR'
-}).client();
+const getSplitClient = require('./split.js');
+const splitClient = getSplitClient();
 
 // Split traffic to serve two variants of the Web page, using `id` query param as user key for feature flag evaluations
 // Web page variants are located at different folders: `dist/off` ('off' treatment) and `dist/on` ('on' treatment)
+
 app.use('/on', express.static(path.join(__dirname, '..', 'dist', 'on')));
 app.use('/off', express.static(path.join(__dirname, '..', 'dist', 'off')));
 app.use('/', (req, res, next) => {
   if (req.query.id) {
-    const treatment = client.getTreatment(req.query.id, process.env.FEATURE_FLAG_NAME);
-    console.log('serving treatment ' + treatment);
-    if (treatment === 'on') {
-      return res.redirect('/on' + req.url)
+
+    const optimizeAsync = splitClient.getTreatment(req.query.id, process.env.FEATURE_FLAG_OPTIMIZE_PAGE);
+    const imageSize = splitClient.getTreatment(req.query.id, process.env.FEATURE_FLAG_IMAGE_SIZE);
+
+    if (optimizeAsync === 'on') {
+      return res.redirect('/on' + req.url + '&img=' + imageSize)
     } else {
-      return res.redirect('/off' + req.url);
+      return res.redirect('/off' + req.url + '&img=' + imageSize);
     }
   }
 

advanced/multipage-testing-example/server/split.js

@@ -0,0 +1,29 @@
+require('dotenv').config();
+const { SplitFactory } = require('@splitsoftware/splitio');
+
+// the SDK client singleton instance
+let client;
+
+function getSplitClient() {
+
+  // This implementation of the singleton pattern ensures
+  // that only one instance of the SplitFactory is created.
+  // This means that only one copy of the Split (feature flag
+  // and segment) definitions are downloaded and synchronized.
+
+  if (!client) {
+    client = SplitFactory({
+      core: {
+        authorizationKey: process.env.SERVER_SIDE_SDK_KEY,
+      },
+      scheduler: {
+        impressionsRefreshRate: 2 // s - send information on who got what treatment at
+      },                          //     what time back to Split server every 2 seconds
+      debug: 'INFO'
+    }).client();
+  }
+
+  return client;
+}
+
+module.exports = getSplitClient;