Skip to content

A B Testing

Jump to bottom
Phillip Dornauer edited this page Apr 14, 2026 · 2 revisions

A/B Testing

Marble's A/B testing lets you create a Variant B for any item. Visitors are randomly bucketed into group A (original) or group B (variant). Impression and conversion counts let you measure which version performs better.

Creating a Variant

  1. Open an item in the editor
  2. In the sidebar, find the A/B Test box
  3. Click Create Variant B
  4. Give the variant a name and set the traffic split (e.g. 20% → 20% of visitors see Variant B)
  5. Save

Editing the Variant

Click Edit Variant B in the A/B Test sidebar box. You'll see all the item's fields, each showing the current Variant A value as a reference below it. Only the fields you fill in are overridden — the rest fall through to the original item.

Traffic Split

The traffic split is a percentage. A split of 30 means 30% of new visitors get Variant B, 70% get Variant A. Once a visitor is assigned, their bucket is stored in a cookie (marble_ab_{item_id}) for 30 days.

Impression Tracking

Marble tracks impressions separately for A and B:

  • Impressions A / B — how many unique visitors were assigned to each bucket
  • Impressions increment only on first assignment (new cookie), not on repeat visits

Conversion Tracking

To measure the actual outcome of the test (a click, a form submit, a purchase, a signup), call Marble::recordAbConversion($item) after the desired action in your own controller:

use Marble\Admin\Facades\Marble;
use Marble\Admin\Models\Item;

// Example: record a conversion after a successful form submit
public function submit(Request $request, Item $item)
{
    // ... handle the form ...

    Marble::recordAbConversion($item);

    return redirect()->back()->with('success', 'Thanks!');
}

Marble reads the visitor's marble_ab_{item_id} cookie and increments conversions_a or conversions_b. If the visitor has no cookie (e.g. direct link, cleared cookies), the call is a no-op.

What counts as a conversion?

Anything you want. Common examples:

Goal Where to call
Form submission After saving the FormSubmission
Newsletter signup After creating the subscriber
Purchase / checkout After order is confirmed
Button click (tracked server-side) In a dedicated tracking endpoint

Conversion Rate (CVR)

The sidebar widget shows CVR % (conversions ÷ impressions) per bucket once each bucket has at least 30 impressions. Below that threshold, raw conversion counts are shown with a "not enough data" notice — avoiding misleading conclusions from small samples.

Once enough data is collected, the leading bucket gets a winner highlight with its CVR:

A  86%  |  2 imp.  |  0% CVR
B  14%  |  14 imp. |  71% CVR  ✓ winner

The Sidebar Widget

The A/B Test box in the item sidebar shows:

  • Active / Paused status with a toggle button
  • Visual traffic split bar with a live drag slider
  • Per-bucket: impression %, impression count, conversion count, CVR (when enough data)
  • Total impressions and total conversions
  • "Not enough data" notice when conversions exist but sample is too small
  • Winner highlight once a clear leader emerges

Pausing a Test

Click Pause in the sidebar. When paused, all traffic goes to Variant A (original). Impressions and conversions stop incrementing. Click Resume to restart.

Removing a Variant

Click Delete Variant in the sidebar. This permanently removes the variant and all its values. There is no undo.

How It Works in Templates

No template changes are required. When a visitor is in bucket B, $item->value('field') automatically returns the variant value (if one exists). The frontend template is identical for both versions — Marble swaps the values transparently.

{{-- This works for both A and B automatically --}}
{{ $item->value('headline') }}
{{ $item->value('hero_image') }}

Limitations

  • Only one variant (B) is supported per item
  • Variant B shares the same blueprint and URL as the original item — it is not a separate page
  • Impression and conversion counts are approximate (cookie-based, not session-based)
  • If a user clears cookies, they may be re-bucketed and conversions may be double-counted
  • No statistical significance calculation — the 30-impression threshold is a practical minimum, not a p-value

Clone this wiki locally