Run an Experiment
Now that you have set up an experiment, you can run it. While an experiment is running:
-
Managed Fusion receives queries from the search app.
-
For the queries that identify a user, Managed Fusion routes the specified percentage of traffic through the experiment. Managed Fusion apportions the queries from different users among the variants in accordance with traffic weights.
-
Managed Fusion records what users do after receiving search results, for example, what links they click.
We recommend that you only run one experiment at a time.
Running an experiment involves three steps:
-
Activate the experiment. Activating an experiment turns on the logic that sends queries through the experiment.
-
Users submit queries and interact with results. Queries and signals about interactions with results come from a search app. In this topic, we explain how to verify that data is flowing through the experiment correctly.
-
Deactivate the experiment. When enough data have been collected for metrics to be significant, you can deactivate the experiment. To run the experiment again, just activate it again. With each activation, you get new metrics.
| Metrics will calculate periodically while an experiment is running. You can also generate metrics for a running experiment manually. Just run the metrics job(s) for the metrics you want to generate. |
Activate an experiment
You can activate an experiment in either of these ways. The outcome is identical.
-
With the app open, navigate to Analytics
> Experiments. -
In the left pane, click the experiment you want to activate. In the upper right, click Activate.
-
With the app open, navigate to Querying
> Experiments. In the left pane, click the experiment you want to activate. In the upper right, click Activate Experiment.
| If you stop Managed Fusion while an experiment is running, then Managed Fusion restarts the experiment automatically the next time you start Managed Fusion. |
Verify that data is coming into the experiment
Signals resulting from requests that flow through the experiment are stored in the _signals collection associated with the primary collection.
You can use the Query Workbench or App Insights (if available) to examine this collection to verify that requests are being distributed among your experiment’s query pipelines.
| Do not modify a running experiment. If you need to make a change, stop the experiment, make the modifications, and then start a new experiment that uses the modified object. |
Deactivate an experiment
You can deactivate an experiment in either of these ways. The outcome is identical.
-
With the app open, navigate to Analytics
> Experiments. -
In the left pane, click the experiment you want to deactivate. In the upper right, click Deactivate.
-
With the app open, navigate to Querying
> Experiments. In the left pane, click the experiment you want to deactivate. In the upper right, click Deactivate Experiment.
Next step
You have run the experiment. Next, you will analyze the experimental results.