Statistical Significance Calculator

Determine if your proposed budget will generate enough conversions for a conclusive A/B test.

Testing Strategy Guides

A/B Test

Holdout Test

Weekly Budget ($)

The amount you plan to spend per week.

Proposed Test Duration (Weeks)

The number of weeks the test will run. We recommend at least 2 weeks.

Assumed Cost Per Click ($)

This is used to estimate the number of clicks your budget will generate.

Test Group Spend Proportion (%)

Control Group Test Group

Historical Conversion Rate (%)

Minimum Detectable Effect (MDE) (%)

The smallest lift or drop you want to be able to detect.

Confidence Level

Assumes a two-sided (two-tailed) test.

Statistical Power

The probability of detecting a real effect if one exists.

Total Audience Size

The total number of users in your target audience.

Holdout Group Proportion (%)

Exposed Group Holdout Group

Baseline Conversion Rate (%)

The organic conversion rate of your total audience.

Calculate Achievable Lift (MDE)

Expected Incremental Lift (%)

The incremental lift you expect to see from the campaign.

Lift Type:

Absolute

Relative

Confidence Level

Assumes a two-sided (two-tailed) test.

Statistical Power

The probability of detecting a real effect if one exists.

[Sept, 2025]

Added ability to calculate **Achievable Lift (MDE)** in both Absolute and Relative terms for Holdout Tests.
Implemented **Holdout Test Mode** to determine required audience size for incremental lift.
Added **Dynamic Split Slider** for A/B test budget distribution.
Implemented a clear **Calculation Breakdown** and actionable **Recommendations**.

This tool is actively maintained and improved.