A/B Test Statistical Analysis

This project performs statistical analysis on an A/B test experiment to determine whether a product or marketing change leads to improved conversion rates.

The analysis applies hypothesis testing using a Two-Proportion Z-Test to evaluate statistical significance and provide data-driven recommendations.

Business Problem

A company launched a new webpage (Variant B) and wants to determine whether it performs better than the existing version (Control A).

The key question:

Does Variant B significantly improve conversion rates?

Dataset

The dataset contains:

user_id – Unique identifier

group – A (control) or B (variant)

converted – 1 (converted) or 0 (not converted)

Tools Used

Python

Pandas

NumPy

SciPy (Two-Proportion Z-Test)

Statistical Methodology

Calculate conversion rates for both groups

Perform Two-Proportion Z-Test

Compare p-value against significance level (α = 0.05)

Provide recommendation

Hypothesis

Null Hypothesis (H₀): No difference in conversion rates

Alternative Hypothesis (H₁): Variant B improves conversion rate

Output

Conversion summary table

Z-statistic

P-value

Final recommendation:

Deploy Variant B

Keep Control

Collect More Data

Business Impact

This analysis helps stakeholders:

Make statistically sound decisions

Avoid biased assumptions

Reduce financial risk

Optimize product performance