Stop Guessing: The Simple Formula for A/B Test Duration & Budgeting

The number one mistake marketers make when running A/B tests is stopping too early. They decide to run a test for "two weeks" or based on an arbitrary budget, not realizing they haven't generated enough conversion volume to reach **statistical significance**.

To run a conclusive test, you must reverse-engineer the planning process, starting with the statistical requirement and ending with the budget. This simple 3-step formula connects your CVR, MDE, budget, and duration.

Step 1: Calculate Required Conversions (The Statistical Goal)

Your first goal is to determine the minimum number of conversions needed across both the control and test groups. This is dictated by three factors you choose:

• **Baseline CVR:** Your starting conversion rate.
• **Minimum Detectable Effect (MDE):** The smallest change you want to prove is real.
• **Confidence/Power:** Your required levels (e.g., 95% confidence, 80% power).

**Example:** If your baseline CVR is 1.0% and your MDE is 5% relative, the statistical formula might tell you that you need **30,000 total conversions**.

Step 2: Calculate Required Budget (The Financial Cost)

Once you have the required conversions, you translate that into the required budget using your historical metrics, namely the Cost Per Click (CPC) and your Baseline CVR.

1. **Required Clicks:** Divide the Required Conversions (from Step 1) by your Baseline CVR.
2. **Required Budget:** Multiply the Required Clicks by your Assumed CPC.

**Example continued:** If you need 30,000 conversions and your CVR is 1.0%, you need 3,000,000 clicks. If your CPC is $2.00, your required total budget is **$6,000,000**.

Step 3: Determine Duration (The Time Constraint)

If your required budget is $6 million, but you can only allocate $100,000 per week, you now have your minimum duration: $6,000,000 / $100,000 = **60 weeks**.

Clearly, 60 weeks is too long. The duration calculation reveals if your test plan is financially and temporally viable. If the duration is too long, you must adjust the **MDE** (make it larger) or **increase the weekly budget**.

The Rule of Seasonality

Even if your budget is huge and you could theoretically hit your required conversions in three days, you must run the test for at least **one full business cycle (14 days minimum)**. This captures user behavior on all days of the week, reducing the impact of daily fluctuations.

By following this reverse-engineering approach, you ensure that every test you launch is properly powered and guaranteed to provide a conclusive readout, whether it's a win or a loss.