Skip to content
Docs
Getting started

Introduction

What you can test, the kinds of test there are, and how a test goes from an idea to a result.

On this page 4

SplitStat lets you test changes to your website on real visitors and tells you which version works better. You show one group of visitors your page as it is and another group a changed version, and SplitStat counts what each group goes on to do.

It will not name a winner until the numbers can support one. A test that is still too close to call says so, rather than guessing.

What you can do

Type What it does Use it when
A/A test Shows both groups the same page. You want to check that the numbers on your site can be trusted.
A/B test Changes the page a visitor is already on: a headline, a button, an image, a section moved or hidden. You want to try a change to one page without building anything.
Split URL test Sends visitors to different addresses, such as /pricing and /pricing-new. The new version is a separate page, or a change too large for an editor.
Rollout Shows a change to a share of your visitors, with no test around it. You have already found a winner and want to keep it on the page.

You make the changes for an A/B test and a rollout in the visual editor, which opens on your own site. You point and click. No code is needed.

How a test works

  1. Install the tag. Add one script to your site. It is what shows each visitor their version and records what they do.
  2. Say what success looks like. Choose the events your site already sends, such as a purchase or a sign-up, and turn them into metrics.
  3. Build the test. Name it, write down what you expect to happen, choose the pages it runs on, make your changes and pick who sees it.
  4. Start it. Each visitor is placed in one version and stays in it. A visitor who comes back on the same browser sees the same page every time.
  5. Read the result. The result screen shows how each version is doing against the original, and names a winner once there is one.

Every test is built in six steps: Setup and hypothesis, Traffic and variants, Goals and metrics, Audience and targeting, Configuration, and Preview and confirm. You can close a test halfway through and come back to it later.

Words you will see

Word What it means
Project One website. Everything in SplitStat belongs to a project.
Variation One version of the page in a test.
Original The page as it is today. Every test has one.
Baseline The variation the others are compared with. It is the original unless you change it.
Event Something a visitor does that your site records, such as a click, a sign-up or a purchase.
Metric A number worked out from events, such as the share of visitors who bought something.
Goal The metric a test is judged on. A test can watch other metrics, but its goal decides it.
Audience Who a test is shown to, such as visitors from one country or on a phone.

Before you start

You need three things:

  • An account and a project. A project is made when you sign up. Add one more for each extra site.
  • Access to your site's code, or somebody who has it, to add the tag once. After that, most tests need no further help from a developer.
  • Some traffic. A test needs enough visitors to tell a real difference from chance. A page with a few dozen visits a week will take a long time to give an answer.