Skip to content
Docs
Manage Experiments

Rollouts

A rollout serves a change to a share of your visitors, with no test around it and no verdict.

On this page 5

A rollout serves a change to a share of your visitors. There is no second version and nothing to compare, so there is no winner.

Use one when the question is already answered. A test that found a winner stops when you stop it, and the change stops with it. A rollout is that same change without the test around it, so it stays on the page while your engineers build it properly. It is also the way to put a change live for a tenth of your traffic first and watch nothing break.

How it differs from a test

Test Rollout
Versions Two or more, compared One
Metrics Required, one decides it None
Result A verdict No verdict
What it reports A full report How many visitors it reached
How it ends You stop it, or a schedule You switch it off

A rollout is judged by nobody. No metrics, no significance, no traffic split check. Those all exist to choose between versions, and a rollout has one version. If you want to know whether a change works, run a test.

How to set one up

  1. Open Rollouts in the sidebar and click New rollout.
  2. Type a Name, such as "Green checkout button". Save it.
  3. On the Pages tab, say which pages carry the change.
  4. On the Changes tab, open the editor. It is the same editor a test variation uses, and it opens the page you named under Pages.
  5. On the Audience tab, narrow who it reaches, or leave it to reach everybody who matches your pages.
  6. On the Settings tab, set the Share of traffic.
  7. Press Switch on.

A new rollout starts as a draft and reaches nobody until you switch it on.

What each label means

Where Label What it means
Settings Name What you will recognise it by in a list next year. Up to 120 characters.
Settings Share of traffic A share of the visitors its rules match, not of everybody. From 0.01% to 100%.
Pages Page to edit The page the editor opens. Up to 2048 characters. Your homepage if you leave it empty.
Pages Page rules Which pages carry the change. Up to forty rules. Any one of them matching is enough.
Changes The change Authored on your own site, in the same editor a test variation uses.
Audience Audience Who it reaches, built the same way as a test's. See Audiences.

On the rollouts list:

Column What it shows
Rollout Its name, and whether it is a draft, live or off.
Changes How many changes it carries.
Reached How many visitors have seen it. "Nobody yet" until one does.

A share of zero is refused. A rollout reaching nobody is one that is switched off, so use Switch off rather than setting the share to nothing.

Good to know

  • Switch off is the undo. It is one press and it takes effect on the next page load. Reach for it first if something looks wrong, before you start editing.
  • A rollout does not take part in mutual exclusion and does not reserve traffic from any test. A visitor can be in a test and see a rollout at the same time.
  • Because of that, a rollout changing the same element a running test changes will muddy that test. Check what is live before you switch one on.
  • Reached counts visitors, not page views. One person seeing the change five times counts once, so the number is smaller than your traffic and is meant to be.
  • The share is of matching visitors, not of all your traffic. Ten per cent of a rollout scoped to your pricing page is ten per cent of pricing page visitors.
  • Each visitor gets a consistent answer. Somebody in the share on Monday is in it on Tuesday, on the same browser.
  • Write the name for somebody reading it in a year. "Green checkout button" tells them what is on the page. "Test 4 winner" does not.