The results screen says which variation won, then shows the numbers that led there. Whether to ship it is your call.
Every test has one goal it is judged on. The screen reads that goal for every variation, compares each one against the baseline, and says whether the difference is big enough to act on. The other metrics you attached are shown too, but they never decide the test.
Numbers update every two minutes while a test runs. The screen says when it last did.
The verdict card
The first line is the answer. It is one of these.
| It says | What it means |
|---|---|
| [Variation] wins | Every condition holds and this variation beat the baseline. The line under it says by how much. |
| [Baseline] wins | Every condition holds and no variation beat the page you already have. |
| [Variation] wins, but a limit was breached | It won on the goal and a guardrail went past its limit. Read the guardrails card before you act. |
| Still collecting | Not enough evidence yet. The line under it says what is still short, and when it will be ready at this pace. |
| [Variation] is ahead | The chance to beat baseline has reached your bar, but the range still includes no change. Not a result. Keep it running. |
| No meaningful difference | Every variation landed within the difference your project treats as too small to matter. |
| No clear winner | The test ended and nothing cleared the bar. That is a result: the change was not big enough to find at this size. |
| Traffic split is wrong | The variations did not get the traffic they were set to get, so the comparison cannot be trusted. |
| No result yet | Nothing has been counted against this test yet. |
| No verdict while filtered | The report is narrowed to some dates or one slice. A verdict is only shown for the whole run and every visitor. |
The screen never tells you to ship. It says what the data found, and you decide what to do with it. What you decided is recorded when you press Stop.
Under a winning line there is a rough range for what the difference comes to in a month: the ends of its likely range, per visitor, times the visitors a day this test has seen, times thirty. It is a rough figure, not a forecast.
The card raises the traffic split only when something is wrong with it. A split that came out as you set it says nothing and stays off the card.
How to read the table
- Find the row tagged Winner. While a test is still collecting, the row in front is tagged Ahead, which is not a result.
- Read Estimated lift range. A range that still crosses zero means the lift could be nothing.
- Read Chance to beat baseline. It has to reach your required certainty, 95% unless your project says otherwise, before a verdict is called. On the sequential engine the column is Result instead.
- Check the guardrails card, if the test has any, before you act.
What each label means
The table has one row per variation. Two of its columns appear only on a goal that measures an amount rather than a rate.
| Label | What it means |
|---|---|
| Variation | The name you gave it, tagged Baseline, Winner once the verdict is ready, or Ahead while it is still collecting. |
| Visitors | People who entered the test on this variation. |
| Conversions | How many of them did the goal at least once, with the rate that comes to beneath. |
| Total revenue / AOV | On a money goal only: what this variation took altogether, with the amount per buyer beneath. A goal measuring something that is not money is headed Total with Per converter beneath. |
| Estimated lift range | How much better this variation is than the baseline, as a share of it, with the 95% confidence range beneath. Unlike a one-off interval it stays valid however often you look. Shown as an amount instead when a percentage would not read, and back as a percentage once it does. |
| Chance to beat baseline | How often this variation's true number beats the baseline's, given what both have done so far. Shown on the Bayesian engine, which is the default. |
| Result | On the sequential engine only, in place of the chance. Winner when the whole range is above zero, Worse when it is below, Not yet otherwise. |
Secondary metrics and the breakdown
Secondary metrics are the other metrics you attached. They show change and a likely range, and no chance to beat baseline, because three metrics against one baseline is three chances of a false winner. They explain a result. They never decide it.
Breakdown slices the goal by who the visitors were: device, channel, country, browser, new against returning, source or campaign. The eight largest slices are shown. Same rule as the secondary metrics: ranges only, and a range that crosses zero earns no colour.
Good to know
- The verdict is only shown for the whole run and every visitor. Narrow the dates and it goes away until you clear them.
- Raw numbers (CSV) in the menu exports every row with the statistics on the deciding goal.
- A winner called in its first week has not been checked for a lift that fades. The card says so. Keep the test running if that matters to you.
- With more than one variation, each range is drawn at a higher confidence than your setting, so the test as a whole still keeps it. The heading over the column says which: 95% with one variation, 97.5% with two, 98.3% with three. A test with more variations takes longer to call a winner.
- From the tenth day, the card compares the lift in the first 7 days with the lift since. When the later lift is much lower, a Lift is fading tag appears with both numbers. The verdict does not change: the total was counted correctly. Read the daily chart before you act on it.
Related articles
- How the stats work: the methods behind the chance, the range and the bars, and what we measured about them.
- Your first A/B test: how a test is built and what step three asks for.
- Metrics: the goal a test is judged on, the cap on extreme totals, and how earlier activity sharpens a result.
- A/A test: the same screen read as a check on your setup.