How to Configure Mutually Exclusive Experiment Groups in Optimizely

When high-velocity experimentation teams run multiple tests simultaneously, overlapping experiments on the same page, checkout funnel, or user journey risk traffic collision. When a visitor enters two tests at once, interaction effects make it impossible to isolate which change drove conversions, and dynamic traffic shifts can trigger Sample Ratio Mismatch (SRM).

In Optimizely, teams solve this by configuring Exclusion Groups (Mutually Exclusive Groups). An exclusion group acts as a traffic container that pools multiple experiments together and partitions audience traffic using a shared, deterministic bucketing seed. A user allocated to Experiment A will never be exposed to Experiment B within the same group.

When to Use Mutually Exclusive Groups (and When to Avoid Them)

Mutual exclusion protects data integrity, but it comes with a trade-off: traffic dilution. Splitting a finite audience pool across multiple tests reduces sample size per test and extends the time needed to achieve statistical significance.

Experimentation leaders should apply mutual exclusion deliberately:

How Optimizely Handles Mutual Exclusion Internally

By default, Optimizely evaluates experiments independently. For every experiment, Optimizely takes the user identifier (User ID or Bucketing ID) combined with the Experiment ID, passes them through a deterministic hashing algorithm (MurmurHash3), and maps the resulting integer into one of 10,000 traffic buckets (0 to 9,999).

Because independent experiments use different hash combinations, a user can land in winning buckets across multiple concurrent experiments at the same time.

When experiments belong to an Exclusion Group, Optimizely alters the bucketing sequence:

1. Shared Bucketing Domain: The hashing function hashes the user identifier against the Exclusion Group ID first.
2. Deterministic Partitioning: The 10,000 buckets are allocated across the member experiments according to their configured traffic percentages.
3. Strict Isolation: If a user falls into the bucket range assigned to Experiment A, they are eligible only for Experiment A and cannot be evaluated for Experiment B or C within that group.

Decision Precedence and Evaluation Order

In Optimizely Feature Experimentation and Web Experimentation, decisions follow a strict evaluation hierarchy:

1. Rule Status Check: Validates that the flag rule or experiment is active.
2. Forced Variations: Overrides all other logic (e.g., QA forced variations).
3. User Allowlisting / Whitelisting: Explicit user ID assignments.
4. User Profile Service (UPS): Stored historical assignments from prior user sessions.
5. Audience Targeting: Checks if the user meets predefined audience attributes.
6. Exclusion Group Evaluation: Verifies whether the user belongs to another experiment in the group. If allocated to another test, they are excluded immediately.
7. Traffic Allocation & Variation Assignment: Allocates the user to a specific variant within the selected experiment.

Step-by-Step Configuration: Optimizely Web Experimentation

For teams running client-side or edge-delivered experiments:

1. Create the Exclusion Group

1. In the Optimizely Web Experimentation dashboard, navigate to Experiments.
2. Click the Exclusion Groups tab. (Note: If your organization uses Optimizely Personalization, navigate to Optimizations > Exclusion Groups).
3. Click Create New Exclusion Group.
4. Enter a descriptive Name (e.g., Checkout_Funnel_Q2_Exclusion) and an optional Description summarizing the initiative.

2. Add Experiments and Assign Traffic Percentages

1. Select the draft experiments you intend to include in the exclusion group.
2. Define the traffic allocation for each experiment within the group:
   • For example, if you have two tests, you can assign 50% of the group traffic to Experiment 1 and 50% to Experiment 2.
   • Alternatively, reserve a holdout group (e.g., 40% to Experiment 1, 40% to Experiment 2, and 20% holdout unallocated).
3. Click Create Exclusion Group (or Save).

3. Verify Experiment Targeting

Ensure all experiments inside the exclusion group share the same target page conditions and activation triggers, or that their audience eligibility overlaps appropriately.

Step-by-Step Configuration: Optimizely Feature Experimentation

For server-side, mobile app, and full-stack engineering teams using Optimizely Feature Experimentation:

1. Create the Exclusion Group in the Flags Dashboard

1. Navigate to Flags and click the Exclusion Groups tab.
2. Click Create New Exclusion Group.
3. Enter the group Name and Description.
4. Select the target Environment (e.g., Production, Staging). Exclusion groups are environment-scoped.
5. Click Create Exclusion Group.

2. Assign Flag Experiment Rules to the Group

1. Navigate back to Flags and select the feature flag that contains your experiment rule.
2. Select your environment and click into the specific Experiment Rule.
3. Under the Configure Rule section, locate the dropdown menu: "Add this experiment to the following exclusion group".
4. Select your newly created exclusion group.
5. Specify the percentage of group traffic this experiment rule should claim.
6. Click Save Experiment.
7. Repeat this step for all companion experiments that must remain mutually exclusive.

3. Programmatic Configuration via REST API

Teams automating their experimentation deployments can also provision exclusion groups via the Optimizely REST API:

• Create Group: POST /v2/groups with payload {"name": "Cart_Redesign_Group", "environment_key": "production"}.
• Update Group: PATCH /v2/groups/{group_id} to attach experiment rules using their unique rule_id.
• List / Inspect: GET /v2/groups to audit active exclusion boundaries across environments.

4. Zero SDK Code Overhead

Exclusion groups require no custom code changes in your application codebase. When your application calls the standard client.decide(user_key, flag_key) method, the Optimizely SDK evaluates the Exclusion Group logic in memory against the downloaded datafile/ruleset.

Four Golden Rules for Experimentation Teams

To avoid corrupted data, skewed statistics, and unexplainable uplifts, experimentation teams must adhere to four governance rules:

1. Launch Experiments in an Exclusion Group Simultaneously

Do not start Experiment 1 today and add Experiment 2 to the same exclusion group next week. If visitors are already bucketed into an active group before all experiments are launched, subsequent additions alter the internal bucket boundaries, causing existing visitors to shift or biasing new visitor assignment.

2. Never Add or Remove Experiments from a Running Group

Once any experiment in an exclusion group is live:

• Do not add new experiments into the group.
• Do not remove running experiments from the group. Modifying active groups destabilizes traffic ratios, triggers Sample Ratio Mismatch (SRM) alerts, and invalidates statistical tests. If a test finishes early, pause the experiment rule rather than dismantling the group.

3. Keep Consistent Audience Targeting

If Experiment A targets all desktop users and Experiment B targets only mobile users, placing them in the same exclusion group wastes traffic. Because bucketing occurs across the group, desktop users allocated to Experiment B will see no variation, starving Experiment A of valuable desktop sample volume. Keep audience criteria aligned within the same group.

4. Beware of User Profile Service (UPS) Overrides

In Feature Experimentation architectures utilizing a User Profile Service (UPS) for sticky variation persistence, UPS lookups evaluate before exclusion groups. If a user already had a saved assignment from a previous standalone launch, the UPS will return that historical variation, bypassing exclusion logic. Use OptimizelyDecideOption.IGNORE_USER_PROFILE_SERVICE when running clean isolation experiments.

References

• Use mutual exclusion - Feature Experimentation docs
• How bucketing works - Feature Experimentation docs
• Optimizely's automatic sample ratio mismatch detection