Which A/B test goes first? A free sheet that ranks your ideas.
A question we get a lot: how do you decide which experiment gets priority? Once the list of test ideas grows, you need a way to pick that doesn’t come down to who argues hardest.
We put our answer in a free Google Sheet called The A/B Test Log. It keeps every test in one place, from the first idea to the result.
Make your copy of The A/B Test Log →
How do you decide which test goes first?
We use ICE. Every idea gets three scores from 1 to 5. Impact is how much the change could move your numbers. Confidence is how sure you are that it will, based on data and past experience. Ease covers the time and money it takes to build: the less it takes, the higher the score.
The sheet multiplies the three (Impact × Confidence × Ease) and ranks every idea automatically, so the highest score gets rank 1 and you know which test to run next.
We like ICE because it’s quick. The downside is that every score is a judgment call. That’s why the backlog asks you to describe the idea and why you expect it to help before anyone scores it. A vague idea gets a vague score.
Can you change what you score on?
Yes. What you score on depends on the company. Rename or swap a criterion, or give one score more weight than the others.
The Score formula is =IF(COUNT(I2:K2)<3,"",I2*J2*K2). Change it to =IF(COUNT(I2:K2)<3,"",I2*J2*K2^2) and Ease counts double, which suits a team where developer time is the bottleneck. Fill the new formula down the whole column and the ranking updates with it.
What else is in the sheet?
The sheet is built for website tests: booking flows, forms, checkouts, landing pages. Besides the backlog it has six tabs.
Start Here explains how the file works. Commandments lists ten principles for running experiments, such as deciding your confidence threshold before the test starts and sticking to it.
Experiment Design is where you write a test down before it goes live. The sheet suggests a form for the hypothesis: if we apply [this change], then we expect [these metrics to change] for [this group of users], because of [this behavioural reason]. Alongside it you fill in the primary and secondary metric, the page or flow, weekly visitors and conversions, the devices, and the first and last full day of the test.
Experiment Results records how the test ended (win, fail or inconclusive) and what you learned from it. The A/B Testing Significance Calculator checks whether a result is significant, so the calculation stays in the same file. In Settings you list your own pages or flows, their steps and the metrics you test on. Those lists feed the dropdowns in the other tabs.
How do you get it?
Open the link. Google asks whether you want to make a copy, and that copy lands in your own Drive, where you can edit it as you like. You need a Google account for this. If something in your copy breaks, make a fresh one from the same link.
Make your copy of The A/B Test Log →
Where have we used a log like this?
We kept one through a twelve-month test programme on the booking flows of a European optician group. Every result went into it, and we wrote up the tests that lost with the same care as the winners. The case shows what that looks like in practice.