Experiments are the best way to test multiple index and search configuration combinations and pick the best one that fulfills your business needs. SearchAssist allows you to create experiments with two or more variants to continuously improve search relevance and user experience.

Overview

Consider the following scenarios:

Scenario 1: You have configured an index, and tuned search configurations to optimize search results. You have run it against test data in a controlled environment, but will these settings work with real-time data?

Scenario 2: You have deployed your search application and have analyzed its performance. You want to tweak the index or search configuration or both a little. You have cloned the existing configuration and made the necessary changes. How can you ensure that these tweaks would work?

Using Experiments, you can find out which index/search configuration is more effective. It helps you:

• Create variants (up to 4) using a combination of previously created indices and search configurations.
• Run them live by splitting the traffic to each variant.
• See which one performs best based on the metrics like clicks and click-through rate.

Internally, every search is associated with a unique user identifier. This serves two purposes:

• It ensures randomness. The application creates sets of users – one for each variant. Whenever a new user arrives, they are randomly distributed into either of the variants, based on a hash of their unique user identifier.
• It maintains the same distribution. If a user is assigned a variant, they continue with the same variant thus ensuring that the experiment conclusion is reliable.

You can further control the experiments by:

• Specifying the percent of traffic assigned to each of the variants.
• Setting the duration of an experiment.

Add Experiments

To add an experiment, follow the below steps:

1. Click the Analytics tab on the top and select Experiments from the drop-down list.
2. On the Experiments page, click + Add Experiment on the top-right.
3. On the New Experiment dialog box, enter a name in the Name of Experiment field.
4. Click + Add Variant.
5. Enter a name in the Variant Name field.
6. Select an index configuration from the Index drop-down list.
7. Select a configuration from the Search Configuration drop-down list.
8. Repeat steps  4 through 7 for multiple variants for a maximum of 4.
9. Under the Traffic section, adjust the slider to set a value indicating the percent of the user query traffic that should be assigned to each variant.
10. In the Duration field, enter the number of days the experiment needs to be run, up to a maximum of 90 days.
11. Click Add.

Run Experiments