# A/B Testing

### Overview

> **Note:** A/B testing is available by request. [Contact support](/docs/troubleshooting/troubleshooting.md#contact-support) to enable this feature for your account.

The A/B testing experiments feature splits your website visitors into two groups:

* **Test Group**: Users who see and interact with Alhena AI
* **Control Group**: Users who don't see Alhena AI (widget is disabled)

### How It Works

1. **Create an Experiment**: Define your audience split (e.g., 50% test, 50% control) in the Analytics dashboard
2. **User Assignment**: When a visitor loads your website, they are automatically assigned to a group based on their unique fingerprint
3. **Consistent Experience**: The same user will always be assigned to the same group throughout the experiment
4. **Track Metrics**: Monitor key metrics like add-to-cart value and checkout revenue for each group

### The `experiment:loaded` Event

When an active experiment is running, the Alhena widget fires the `experiment:loaded` event after determining which group the user belongs to. This event allows you to:

* Conditionally load other SDKs based on the experiment group
* Customize your page behavior for control group users
* Implement your own tracking logic

#### Event Signature

```javascript
window.gleenWidget.on('experiment:loaded', (experimentGroup) => {
    // experimentGroup is either 'test' or 'control'
});
```

**Parameters:**

* `experimentGroup` (string): Either `'test'` (user sees Alhena AI) or `'control'` (user doesn't see Alhena AI)

### Example: Loading Zendesk for Control Group

A common use case is to show an alternative support solution (like Zendesk) to users in the control group who don't see Alhena AI:

```html
<script src="https://app.alhena.ai/js/gleen-widget/main.js"></script>

<script>
    window.gleenWidget.on('experiment:loaded', (experimentGroup) => {
        if (experimentGroup === 'control') {
            // Dynamically load Zendesk for control group users
            const script = document.createElement('script');
            script.id = 'ze-snippet';
            script.src = 'https://static.zdassets.com/ekr/snippet.js?key=7460dcd8-6a51-442e-9675-9781f5954552';
            document.body.appendChild(script);
        }
    });
</script>
```

This ensures:

* **Test group** users see Alhena AI
* **Control group** users see Zendesk instead
* Fair comparison between the two solutions

### Important Notes

* The `experiment:loaded` event only fires when an **active** experiment is running
* User assignment is deterministic and based on their fingerprint UUID, ensuring consistency across sessions
* Users in the **control group** will not see the Alhena AI widget (it's automatically disabled)
* Make sure to set up your **cart and checkout events** before starting an experiment to track revenue impact

### Setting Up Experiments

#### Prerequisites

Before creating an experiment, ensure you have:

1. **Cart Events**: Implemented using `gleenWidget.sendCartEvent()`
2. **Checkout Events**: Implemented using `gleenWidget.sendCheckoutEvent()`

See the [Cart and Checkout Event API documentation](/docs/developer-reference/website-sdk/cart-checkout-events.md) for setup instructions.

#### Creating an Experiment

1. Navigate to **Analytics > Experiments** in your dashboard
2. Click **Create Experiment**
3. Enter experiment name and set audience split
4. Verify cart and checkout events are being received
5. Click **Start Experiment**

#### Monitoring Results

The experiment dashboard shows real-time metrics for both groups:

* **Exposure**: Number of unique users in each group
* **Add to Cart Value**: Total cart value added
* **Checkout Revenue**: Total revenue from completed checkouts
* **Delta**: Percentage difference between test and control groups

#### Ending an Experiment

When you're ready to conclude:

1. Navigate to your experiment in the dashboard
2. Click **End Experiment**
3. Review the final results and statistical significance
4. The system automatically determines if the experiment was Won, Lost, or Inconclusive

If you don't end the experiment, it will end automatically after 30 days.

### How It Works Under the Hood

When a user visits your website:

1. The Alhena widget loads and checks if an experiment is active
2. The user is assigned to a group based on their unique fingerprint using a deterministic algorithm
3. The `experiment:loaded` event fires with either `'test'` or `'control'`
4. For control group users, the Alhena widget is automatically disabled

This ensures:

* **Consistency**: The same user always sees the same experience
* **Even distribution**: Users are evenly split between groups
* **No bias**: Assignment is random and not based on user behavior

### Best Practices

1. **Run for adequate duration**: Allow at least 1-2 weeks to gather meaningful data
2. **Don't modify mid-experiment**: Changing audience split invalidates results
3. **Track other metrics**: Use the event to send experiment group to your analytics platform
4. **Test your integration**: Verify the control group experience works as expected
5. **Statistical significance**: Wait for enough data before drawing conclusions

### FAQ

**Q: Can I run multiple experiments simultaneously?**

A: No, only one experiment can be RUNNING at a time per bot profile.

**Q: What happens to users after an experiment ends?**

A: All users will see the normal Alhena AI widget (no experiment assignment).

**Q: Can I change the audience split after starting?**

A: No, this would invalidate the experiment. You'll need to end and create a new one.

**Q: Is the event fired on every page load?**

A: Yes, the event fires each time the widget loads and detects an active experiment.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://alhena.gitbook.io/docs/developer-reference/website-sdk/ab-testing.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.