How Squoosh decides a winner¶
When an experiment finishes, Squoosh files a verdict: keep the original, ship the variant, or call it too close. This page explains how that decision is made and how to read it.
Which result mode applies¶
The rigor level you choose in the wizard sets how Squoosh arrives at the lift:
| Shoppers | Method | What the lift is |
|---|---|---|
| Standard or Medium | Estimated lift | Squoosh's AI reviewer compares the two pages and grounds its read in what these shoppers did. It is a directional read, not a statistically conclusive result. The sections below describe this standard verdict. |
| High and above (Two URLs) | Measured lift | Squoosh counts verified conversions on each page across paired synthetic shoppers and reports the difference with a 95% sampling interval. The result page is labeled Village conversion lift. |
A measured run does not use the standard confidence meter or the reviewer override. It measures recorded conversions independently of shopper feature feedback and reports sampling intervals and coverage. At High the interval is wide (about ±27 percentage points), so No clear conversion winner is a common, honest outcome; it narrows at higher levels. The shoppers are synthetic, so it measures synthetic shoppers, not your customers. See Conversion-evidence results.
The VWO / Optimizely / AB Tasty / Convert and Feature flag variant modes and demo workspaces offer Standard and Medium only and report an estimated lift, because a measured result needs a separately hosted variant. The wizard names the method before you launch, in the note under the Rigor slider and in the Method cell of Review & run. Each experiment retains the result mode it started with.
The two inputs¶
Squoosh decides using two independent reads of the same experiment:
- An expert reviewer. Squoosh's reviewer reads both versions of the page and compares them the way an experienced conversion analyst would — layout, copy, friction, and what changed between A and B.
- Shopper outcomes. Every synthetic shopper makes a decision on each version. Squoosh measures how those decisions differ between the original and the variant, and how confident that difference is statistically.
The reviewer explains why a version performed the way it did; the shopper outcomes establish how sure Squoosh is that the difference is real. The verdict combines both.
The three outcomes¶
| Verdict | What it means | What to do |
|---|---|---|
| Variant B wins | The variant beat the original on your goal. | Ship the variant. |
| Keep the original | The variant did worse than the original. | Keep the original page live. Don't ship the variant. |
| Too close to call | Neither version clearly separated from the other. | Re-test with a sharper change. |
"Keep the original" is a real result, not a failure. A change that costs you conversions is worth knowing about before you ship it. When the original wins, Squoosh tells you to keep it — it never frames a loss as a win.
A filed verdict — the outcome, the lift, and the confidence against the 80% bar.
The 80% confidence bar¶
Squoosh calls a winner at 80% confidence, not the 95% a traditional A/B tool waits for.
This is deliberate. An 80% bar lets you learn from a test and move to the next idea faster, which matters when you're running many experiments. The trade-off is a higher chance of a false positive than a 95% bar would allow.
If you need more certainty before you ship, re-run the experiment, or re-test with a sharper difference between A and B.
The verdict always shows the confidence it reached, with the 80% mark on the meter so you can see how far past the bar the result landed. Squoosh never reports more than 98% confidence — it doesn't claim absolute certainty.
Note
Squoosh needs a minimum number of shoppers to decide on each version before it will file any verdict. A run that ends with too few completed shoppers stays inconclusive rather than guessing.
Where the verdict can disagree with the table¶
The headline can reflect the reviewer's read while the results table shows the raw shopper rate, and the two can point in different directions. You might see a + number for the variant in the table and still be told to keep the original — the change didn't earn the lift, so Squoosh doesn't credit it. When this happens, the verdict says so directly rather than letting the raw number mislead you.
You always decide¶
Squoosh never applies a change to your site. There is no "ship" or "deploy" button. The verdict is a recommendation backed by evidence; rolling out the variant is your call.
What you see after a run¶
The results page reads top to bottom in one order:
- The verdict — the outcome, the lift, and the confidence.
- Why it won — a plain-language summary of what drove the result, the issues shoppers hit, and what individual shoppers said.
- The evidence — the per-version results and the change by segment.
See Reading lift and confidence for how to read the numbers in the verdict band.