How Squoosh decides a winner¶
When an experiment finishes, Squoosh files a verdict: keep the original, ship the variant, or call it too close. This page explains how that decision is made and how to read it.
The two inputs¶
Squoosh decides using two independent reads of the same experiment:
- An expert reviewer. Squoosh's reviewer reads both versions of the page and compares them the way an experienced conversion analyst would — layout, copy, friction, and what changed between A and B.
- Shopper outcomes. Every synthetic shopper makes a decision on each version. Squoosh measures how those decisions differ between the original and the variant, and how confident that difference is statistically.
The reviewer explains why a version performed the way it did; the shopper outcomes establish how sure Squoosh is that the difference is real. The verdict combines both.
The three outcomes¶
| Verdict | What it means | What to do |
|---|---|---|
| Variant B wins | The variant beat the original on your goal. | Ship the variant. |
| Keep the original | The variant did worse than the original. | Keep the original page live. Don't ship the variant. |
| Too close to call | Neither version clearly separated from the other. | Re-test with a sharper change. |
"Keep the original" is a real result, not a failure. A change that costs you conversions is worth knowing about before you ship it. When the original wins, Squoosh tells you to keep it — it never frames a loss as a win.
The 80% confidence bar¶
Squoosh calls a winner at 80% confidence, not the 95% a traditional A/B tool waits for.
This is deliberate. An 80% bar lets you learn from a test and move to the next idea faster, which matters when you're running many experiments. The trade-off is a higher chance of a false positive than a 95% bar would allow.
If you need more certainty before you ship, re-run the experiment with a larger audience. A bigger sample narrows the result and can clear a higher bar.
The verdict always shows the confidence it reached, with the 80% mark on the meter so you can see how far past the bar the result landed. Squoosh never reports more than 98% confidence — it doesn't claim absolute certainty.
Note
Squoosh needs a minimum number of shoppers to decide on each version before it will file any verdict. A run that ends with too few completed shoppers stays inconclusive rather than guessing.
Where the verdict can disagree with the table¶
The headline can reflect the reviewer's read while the results table shows the raw shopper rate, and the two can point in different directions. You might see a + number for the variant in the table and still be told to keep the original — the change didn't earn the lift, so Squoosh doesn't credit it. When this happens, the verdict says so directly rather than letting the raw number mislead you.
You always decide¶
Squoosh never applies a change to your site. There is no "ship" or "deploy" button. The verdict is a recommendation backed by evidence; rolling out the variant is your call.
What you see after a run¶
The results page reads top to bottom in one order:
- The verdict — the outcome, the lift, and the confidence.
- Experiment Insights — a plain-language summary of what drove the result.
- The evidence — the per-version results, the change by segment, and what individual shoppers did and said.
See Reading lift and confidence for how to read the numbers in the verdict band.