Conversion
Conversion rate calculator
What's a higher conversion rate worth in orders, revenue and profit?
Version B vs A
+25.0%conversion rate
Not proven yet: 90.8% confidence, below the 95% you chose. Keep the test running: each version needs about 13,809 visitors to confirm a difference this size.
Conversion rate A
2.0%
Conversion rate B
2.5%
Confidence
90.8%
p-value 0.092. Two-sided test, so it also catches a version that's worse.
Run the test for at least
29days
Each version needs about 21,109 visitors to spot a 20% lift (from 2% to 2.4%) with 95% confidence. Round up to 5 full weeks.
Visitors per version
21,109
Total visitors
42,218
Full weeks
5
Days needed for other lifts
| Lift | Visitors per version | Days |
|---|---|---|
| 10% | 80,682 | 108 |
| 15% | 36,693 | 49 |
| 20% | 21,109 | 29 |
| 30% | 9,798 | 14 |
| 50% | 3,826 | 6 |
Uses 80% power: if the lift is real, a test this size catches it four times out of five.
Book a strategy callWant tests that find real winners? 30 minutes with Dair, the founder.
THE MATH
What statistical significance means
A result is significant when the gap between the versions is too big to be luck. At 95% confidence, a gap this size would turn up by chance less than 5% of the time if A and B really performed the same. Below that, you can't tell a better version from a lucky week.
Confidence = how unlikely the gap would be if both versions performed the same
Visitors needed = grows as the lift gets smaller: halve the lift, and you need about four times the visitors
Worked example
The numbers already in the first calculator.
- Version A
- 100 ÷ 5,000 = 2.0%
- Version B
- 125 ÷ 5,000 = 2.5%
- Lift
- +25.0%
- Confidence
- 90.8%
At 95%Not proven yet
Visitors needed per versionabout 13,809
GOOD TESTING HABITS
How to run a test you can trust
Three habits that keep a lucky result from passing for a real winner.
Decide the length first
Checking every day and stopping at the first good-looking result is an easy way to crown a false winner. Work out the length with the calculator above, then leave the test alone.
Run whole weeks
Shoppers behave differently on weekends and paydays. Running full weeks means the result covers every kind of day, not just the ones that fell inside the test.
Test big changes on small traffic
With fewer visitors, only big differences show up in time. Test a new offer or a rebuilt product page, not a button color. See what a win is worth with the conversion rate calculator, or how we run tests in conversion optimization.
FAQ
Questions about A/B testing.
Don't see your question? Ask Dair directly on the call.
Still have a question?
Email or call. Dair replies personally.
Until each version has the number of visitors the duration calculator gives you, and for at least one full week, ideally two, so every day of the week is covered. Stores with less traffic may need several weeks, or bigger changes to test.
No. Results swing a lot early on, and if you check every day and stop at the first good-looking number, you will often crown a winner that is not real. Decide how long the test runs before you start, and stick to it.
95% is the usual choice. Use 90% for small changes that are easy to undo, and 99% for big decisions like a new price.
Because small differences are hard to tell apart from luck. The number grows fast as the lift you want to find gets smaller: halve it and you need about four times the visitors.
You can, but it takes longer. This calculator checks conversion rate, the most common test goal. Revenue per visitor also depends on order values, which vary much more, so it needs more visitors to call. Watch both, but pick one main goal before you start.
More free tools
See all tools Customers & email
Customer lifetime value calculator
What's a customer worth over time, and how much can I pay to win one?
Ads & offers
Break-even ROAS calculator
What ROAS do my ads need to break even, and to make a profit?
