Reading an experiment readout
Significant means a 95%-confidence test against control with at least 100 exposures per arm — never a bare comparison of raw rates.
A readout is one row per arm, and every number on it except the raw counts is computed against a single reference point: the control arm.
What "significant" is actually testing
Significant means one specific test passed: a two-proportion z-test at 95% confidence between an arm and control. It never runs at all until both arms have at least 100 exposures — below that floor the label is not enough data, which is a different claim from "not significant yet." A test with 40 exposures on one arm hasn't failed to find a difference; it hasn't been checked.
Only ever against control
Every non-control arm is compared to control and nothing else. With three or more arms on one experiment, "B beat C" isn't a number the readout produces — only "B beat control" and "C beat control" are.
The control row itself is the visible edge of that: it has nothing to be compared with, so its significance column always reads not enough data and its uplift is always blank, however many exposures it has collected. That is the shape of the table, not a verdict on the control arm — read control's own exposures and conversion rate instead.
Uplift also goes blank — not 0% — whenever control has converted no one yet. Dividing by a zero rate produces no number rather than a misleading one, so a missing baseline is never dressed up as "no improvement".
The readout's columns
| Name | Type | Default | Description |
|---|---|---|---|
exposures | count | — | Visitors bucketed into this arm and counted — see Running an A/B experiment for when a bucketed visitor is not counted. |
conversion rate | percentage | — | Conversions divided by exposures for this arm alone. |
uplift | percentage, or blank | — | This arm's rate compared to control's, as a percentage change. Always blank on the control row, and blank whenever control has converted no one. |
significance | label | — | not enough data below the 100-exposure floor on either arm, and always on the control row; otherwise not significant or significant from the z-test against control. |
winner | flag | — | Set on the experiment's declared winner if one has been chosen, or on the leading arm once its own test against control comes back significant — never on a bare leader by rate alone. |
How the winner is chosen
If someone has explicitly declared a winner on the experiment, that arm wins, full stop. Failing that, the readout looks at whichever arm has the highest conversion rate — but only flags it as the winner if its own significance test against control came back significant. A small-sample arm sitting on a higher raw rate is left unmarked rather than declared a winner it hasn't earned.
Declaring a winner is not the same as shipping it
There are two separate things called "the winner", and only one of them changes what visitors see.
Completing the experiment with a winner set is a record of the decision. It settles the readout — but a completed experiment is no longer among the running experiments a page loads, so the container stops getting assignments altogether and falls back to its Default arm. Finishing a test this way rolls the page back to the original section, not forward onto the arm that won.
What actually ships the result is the Winner on the section container itself. Locked there, it routes all traffic to that arm ahead of any assignment and whatever the weights say — except a visitor who matches one of the container's own audience arms, who still gets that audience arm ahead of the lock. So the order is: declare the winner on the experiment, then lock that arm on the container — or copy the winning content into the Default arm and unwrap. See Which version of a page a visitor gets for where that lock sits in the precedence.