Run an experiment

Now that you’ve set up an experiment, you can run it. While an experiment is running:

  • Fusion receives queries from the search app.

  • For the queries that identify a user, Fusion routes the specified percentage of traffic through the experiment. Fusion apportions the queries from different users among the variants in accordance with traffic weights.

  • Fusion records what users do after receiving search results, for example, what links they click.

We recommend that you only run one experiment at a time.

Running an experiment involves three steps:

  1. Activate the experiment – Activating an experiment turns on the logic that sends queries through the experiment.

  2. Users submit queries and interact with results – Queries and signals about interactions with results come from a search app. In this topic, we explain how to verify that data is flowing through the experiment correctly.

  3. Deactivate the experiment – When enough data have been collected for metrics to be significant, you can deactivate the experiment. To run the experiment again, just activate it again. With each activation, you get new metrics.

Tip
Metrics will calculate periodically while an experiment is running. You can also generate metrics for a running experiment manually. Just run the metrics job(s) for the metrics you want to generate.

Activate an experiment

You can activate an experiment in either of these ways. The outcome is identical.

From the experiment
  1. With the app open, navigate to Analytics Analytics > Experiments.

  2. In the left pane, click the experiment you want to activate. In the upper right, click Activate.

From the query profile
  1. With the app open, navigate to Querying Querying > Experiments. In the left pane, click the experiment you want to activate. In the upper right, click Activate Experiment.

Note
If you stop Fusion while an experiment is running, then Fusion restarts the experiment automatically the next time you start Fusion.

Verify that data is coming into the experiment

Signals resulting from requests that flow through the experiment are stored in the _signals collection associated with the primary collection. You can use the Query Workbench or App Insights to examine this collection to verify that requests are being distributed among your experiment’s query pipelines.

Tip
Don’t modify a running experiment. If you need to make a change, stop the experiment, make the modifications, and then start a new experiment that uses the modified object.

Deactivate an experiment

You can deactivate an experiment in either of these ways. The outcome is identical.

From the experiment
  1. With the app open, navigate to Analytics Analytics > Experiments.

  2. In the left pane, click the experiment you want to deactivate. In the upper right, click Deactivate.

From the query profile
  1. With the app open, navigate to Querying Querying > Experiments. In the left pane, click the experiment you want to deactivate. In the upper right, click Deactivate Experiment.

Next step

You’ve run the experiment. Next, you will analyze the experimental results.