Skip to content
Docs
Measuring

Mutual exclusion

A mutual exclusion is a group of tests that share traffic, so a visitor only ever lands in one of them.

On this page 6

A mutual exclusion is a group of tests that share traffic. A visitor lands in at most one member of the group, ever.

Most tests can run side by side without trouble. Put two together in a mutual exclusion when they could change each other's results, such as a pricing test and a checkout test in the same funnel. A visitor who saw both would be answering two questions at once, and neither answer would be clean.

Included on Growth from 250K tested visitors up and Enterprise.

How it works

Think of the group as a line of traffic with 10,000 places on it. A test takes its part of the line when it starts, from whatever nobody else holds, and keeps that part until it stops.

State Does it hold a part of the line
Draft or scheduled No. Joining a group costs nothing.
Running Yes, its whole traffic share.
Paused Yes. It keeps its part.
Stopped or finished No. Its part goes back to the group.

A paused test keeps its space on purpose. A pause is off today and back tomorrow with the visitors it already placed. Handing its part to another test in between would put those visitors in two tests at once, which is the one thing a mutual exclusion exists to prevent.

Free space in the group is not a holdout. Nothing is recorded about the visitors who land there, so there is nothing to compare them against later.

How to set one up

  1. Open Project settings and find the mutual exclusions section.
  2. Click New mutual exclusion.
  3. Type a Name, such as "Checkout work".
  4. Add a line under What it is for, if you want one.
  5. Tick the tests to include under Tests in this exclusion, then save.

You make a group here and pick it on a test, never the other way round. On the test, the group is chosen at step five, Configuration.

What each label means

Label What it means
Name What you will pick from on a test. No two groups in a project can share one. Up to 80 characters.
What it is for Optional. Shown beside the group everywhere it appears. Up to 200 characters.
Tests in this exclusion Which tests belong. Up to twenty.

On a test, at step five:

What you see What it means
Sharing traffic with NAME The test fits. It says what it will take and what is free right now.
NAME does not have enough free traffic It cannot start. Lower its traffic share, or stop a test you are done with.

The share a test takes from the group is its ordinary traffic share, set on the traffic step. Nothing extra to set.

What it costs you

This is the trade, and it is worth knowing before you build a group.

A visitor in one member is a visitor no other member can have. Three tests sharing one group each get a third of the traffic they would have had alone, so each one waits about three times as long for an answer.

So group two tests when they genuinely interfere. Do not group every test in the project by default.

Good to know

  • A new test starts at a 100% traffic share, which cannot fit a group that is already partly taken. Adding such a test to a group lowers its share to what is free, so the number you see on the traffic step is one that can actually run. If nothing is free, the share is left alone and the screen names who is holding the space.
  • A running or paused test cannot be taken out of a group. Its visitors would then be free to land in the other members too. Stop it first.
  • A test already running on its own cannot join a group. Its visitors were placed without the group, and some of them are standing on parts the other members hold. Add it before it starts next time.
  • Drafts, scheduled tests and finished ones can come and go at any time, even while other members are running. Nobody already in a test moves.
  • When a test takes a part of the line and no other member holds one, the group is dealt again from scratch. Otherwise the space a finished test left behind would go straight to the next test along with everybody that test had already shown something to.
  • Deleting a group empties it first, so the same rules apply. A group holding a running or paused test cannot be deleted until that test stops.