Insights Reports and Signal Data Requirements

Table of Contents

Signal Types
Required Fields

Although there are many fields that are relevant to signals, this topic describes which fields are required for each type of signal. Without these fields, reports and dashboards within App Insights may not function as expected.

Signal Types

There are five types of signals:

Annotation. Annotation signals are generated when a user bookmarks, likes, or comments on a document. Annotation signals are likewise generated when the user removes a bookmark, like, or comment.

Annotation signals are generated by App Studio. If you are not using App Studio, this type of signal is not relevant to your search application.
Click. Click signals are generated when a user clicks on a page element that is being monitored by the search app. Click signals are sent from the search app to Fusion.
Login. Login signals record information about specific users when they log in to an application. This includes a time stamp and various session details.
Request. A request signal is generated by a front-end search app and captures the raw user query and other contextual information about a user and their journey through the search app.
Response. Response signals are automatically generated by a query pipeline when the signals feature is enabled for a collection.

Because response signals and their fields are automatically generated, this topic does not cover what response signal fields are required.

Required Fields

Signals

The following table describes which fields are required for annotation, click, login, and request signals.

Requests (or queries) can also require additional, available user data for the search.

Field Type Description Example Required

Field	Type	Description	Example	Required
`id`	string	Unique ID for the signal.	`b0ee5307-6223-4150-ac5a-d0d8113aa480`	✅ Annotation ✅ Click ✅ Login ✅ Request
`doc_id`	string	Product ID or Item ID of the clicked result.	`NMDDV`	✘ Annotation ✅ Click ✘ Login ✘ Request
`date`	timestamp	Timestamp of when the signal was generated. This timestamp follows Unix epoch time formatting.	`1518717749409`	✅ Annotation ✅ Click ✅ Login ✅ Request
`user_id`	string	Unique ID for the user that generated the signal.	`admin`	✅ Annotation ✅ Click ✅ Login ✅ Request
`session`	string	Unique ID for the user’s browser session.	`ef4e00cd-91bb-45b4-be80-e81f9f9c5b27`	✅ Annotation ✅ Click ✅ Login ✅ Request
`host`	string	Host name of the server which is hosting the app that is generating the signal.	`x.y.z`	✅ Annotation ✅ Click ✅ Login ✅ Request
`ip_address`	string	IP address of the user that generated the signal.	`80.6.99.35`	✅ Annotation ✅ Click ✅ Login ✅ Request
`app_id`	string	Name of the application that is generating the signal.	`myApp`	✅ Annotation ✅ Click ✅ Login ✅ Request
`annotation_type`	string	Type of the annotation signal, either "comment", "bookmark", or "like". Required fields for "comment": `comment`. The comment itself. `target`. The target that the comment refers to. Required fields for "bookmark": `title`. The bookmark title. `url`. The bookmark url. Required fields for "like": `` - `` -	`bookmark`	✅ Annotation ✘ Click ✘ Login ✘ Request
`query`	string	Terms of the query.	`ipad`	✘ Annotation ✅ Click ✘ Login ✅ Request
`fusion_query_id`	string	Unique ID for the `query` that is automatically generated from the Fusion response signal.	`ABkaEA11`	✘ Annotation ✅ Click ✘ Login ✘ Request
`filter`	array of string	List of filters associated with the query, which in turn is associated with signal.	`[“type/tablet”,”category/electronics”]`	✘ Annotation ✅ Click ✘ Login ✅ Request
`ctype`	string	Type of click.	`result`	✘ Annotation ✅ Click ✘ Login ✘ Request
`res_pos`	number	Position of the clicked result within the list of results.	`3`	✘ Annotation ✅ Click ✘ Login ✘ Request
`res_offset`	number	Result page.	`2`	✘ Annotation ✅ Click ✘ Login ✘ Request
`url`	string	URL of the page that the signal originated from.	`http://localhost:8080/products/search`	✘ Annotation ✘ Click ✘ Login ✅ Request
`path`	string	URL path of the page that the signal originated from.	`/search`	✘ Annotation ✘ Click ✘ Login ✅ Request
`page_title`	string	Title of the page that the signal originated from.	`Search Page`	✘ Annotation ✘ Click ✘ Login ✅ Request

id

string

Unique ID for the signal.

b0ee5307-6223-4150-ac5a-d0d8113aa480

✅ Annotation

✅ Click

✅ Login

✅ Request

doc_id

string

Product ID or Item ID of the clicked result.

NMDDV

✘ Annotation

✅ Click

✘ Login

✘ Request

date

timestamp

Timestamp of when the signal was generated. This timestamp follows Unix epoch time formatting.

1518717749409

✅ Annotation

✅ Click

✅ Login

✅ Request

user_id

string

Unique ID for the user that generated the signal.

admin

✅ Annotation

✅ Click

✅ Login

✅ Request

session

string

Unique ID for the user’s browser session.

ef4e00cd-91bb-45b4-be80-e81f9f9c5b27

✅ Annotation

✅ Click

✅ Login

✅ Request

host

string

Host name of the server which is hosting the app that is generating the signal.

x.y.z

✅ Annotation

✅ Click

✅ Login

✅ Request

ip_address

string

IP address of the user that generated the signal.

80.6.99.35

✅ Annotation

✅ Click

✅ Login

✅ Request

app_id

string

Name of the application that is generating the signal.

myApp

✅ Annotation

✅ Click

✅ Login

✅ Request

annotation_type

string

Type of the annotation signal, either "comment", "bookmark", or "like".

Required fields for "comment":

comment. The comment itself.
target. The target that the comment refers to.

Required fields for "bookmark":

title. The bookmark title.
url. The bookmark url.

Required fields for "like":

`` -
`` -

bookmark

✅ Annotation

✘ Click

✘ Login

✘ Request

query

string

Terms of the query.

ipad

✘ Annotation

✅ Click

✘ Login

✅ Request

fusion_query_id

string

Unique ID for the query that is automatically generated from the Fusion response signal.

ABkaEA11

✘ Annotation

✅ Click

✘ Login

✘ Request

filter

array of string

List of filters associated with the query, which in turn is associated with signal.

[“type/tablet”,”category/electronics”]

✘ Annotation

✅ Click

✘ Login

✅ Request

ctype

string

Type of click.

result

✘ Annotation

✅ Click

✘ Login

✘ Request

res_pos

number

Position of the clicked result within the list of results.

3

✘ Annotation

✅ Click

✘ Login

✘ Request

res_offset

number

Result page.

2

✘ Annotation

✅ Click

✘ Login

✘ Request

url

string

URL of the page that the signal originated from.

http://localhost:8080/products/search

✘ Annotation

✘ Click

✘ Login

✅ Request

path

string

URL path of the page that the signal originated from.

/search

✘ Annotation

✘ Click

✘ Login

✅ Request

page_title

string

Title of the page that the signal originated from.

Search Page

✘ Annotation

✘ Click

✘ Login

✅ Request

Insights Reports

The table below describes which fields are required to generate the various reports within App Insights:

Report Signal Types Used Required Field Type Description Example

Report	Signal Types Used	Required Field	Type	Description	Example
Search queries	✘ Annotation ✅ Click ✘ Login ✅ Request ✅ Response	`query`	string	Terms of the query.	`physics`
Facets used	✘ Annotation ✘ Click ✘ Login ✅ Request ✘ Response	`filter_field`	array of string	List of filters associated with the query, which in turn is associated with signal.	`["categories_s"]`
Facet filters applied	✘ Annotation ✘ Click ✘ Login ✅ Request ✘ Response	`filter`	array of string	List of applied filters associated with the query, which in turn is associated with signal.	`["categories_s/math.OA"]`
Application servers	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	`host`	string	Host name of the server that is hosting the app, which in turn is generating the signal.	`lucidworks`
Response times	✘ Annotation ✘ Click ✘ Login ✘ Request ✅ Response	`time`	number	Response time (in milliseconds).	`17`
URLs clicked	✘ Annotation ✅ Click ✘ Login ✘ Request ✘ Response	`url`	string	URL of the page that was selected from the search results. This typically results from a click signal.	`http://arxiv.org/pdf/astro-ph/0611688v1`
Search pages	✘ Annotation ✘ Click ✘ Login ✅ Request ✘ Response	`page_title`	string	Title of the page that the signal originated from.	`Search Preview`
Search platforms	✘ Annotation ✅ Click ✘ Login ✅ Request ✅ Response	`platform`	string	Name of the search platform.	`fusion`
Visitor countries	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	`geo_country_name`	string	Originating country of the user.	`United States`
Visitor cities	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	`geo_city_name`	string	Originating city of the user.	`San Francisco`
Browsers	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	`browser_name`	string	Name of the browser that generated the signal.	`Firefox`
Operating System	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	`os_name`	string	Name of the operating system used by the user.	`Ubuntu`
Device types	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	`os_device`	string	Type of the device used by the user.	`Computer`
Websites people are coming from	✘ Annotation ✅ Click ✘ Login ✘ Request ✘ Response	`referrer_domain`	string	URL of the page that the signal originated from.	`http://www.google.com`
Users	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	`user_id`	string	Unique ID for the user that generated the signal.	`admin`
User domains	N/A	`domain`	string	Domain of the user that generated the signal.	`lucidworks.com`
Types of event	✅ Annotation ✅ Click ✅ Login ✅ Request ✅ Response	`type`	string	Type of the signal.	`click`
Head Tail analysis	✘ Annotation			This report is populated by running a head tail job in Fusion. With a collection called `foo`, for example, run the `foo_head_tail` job.	N/A
✅ Click	type	string	N/A
✅ Click	query	string	N/A
✘ Login			N/A
✘ Request			N/A
✅ Response	count_i	number	N/A

Search queries

✘ Annotation

✅ Click

✘ Login

✅ Request

✅ Response

query

string

Terms of the query.

physics

Facets used

✘ Annotation

✘ Click

✘ Login

✅ Request

✘ Response

filter_field

array of string

List of filters associated with the query, which in turn is associated with signal.

["categories_s"]

Facet filters applied

✘ Annotation

✘ Click

✘ Login

✅ Request

✘ Response

filter

array of string

List of applied filters associated with the query, which in turn is associated with signal.

["categories_s/math.OA"]

Application servers

✘ Annotation

✅ Click

✘ Login

✅ Request

✘ Response

host

string

Host name of the server that is hosting the app, which in turn is generating the signal.

lucidworks

Response times

✘ Annotation

✘ Click

✘ Login

✘ Request

✅ Response

time

number

Response time (in milliseconds).

17

URLs clicked

✘ Annotation

✅ Click

✘ Login

✘ Request

✘ Response

url

string

URL of the page that was selected from the search results. This typically results from a click signal.

http://arxiv.org/pdf/astro-ph/0611688v1

Search pages

✘ Annotation

✘ Click

✘ Login

✅ Request

✘ Response

page_title

string

Title of the page that the signal originated from.

Search Preview

Search platforms

✘ Annotation

✅ Click

✘ Login

✅ Request

✅ Response

platform

string

Name of the search platform.

fusion

Visitor countries

✘ Annotation

✅ Click

✘ Login

✅ Request

✘ Response

geo_country_name

string

Originating country of the user.

United States

Visitor cities

✘ Annotation

✅ Click

✘ Login

✅ Request

✘ Response

geo_city_name

string

Originating city of the user.

San Francisco

Browsers

✘ Annotation

✅ Click

✘ Login

✅ Request

✘ Response

browser_name

string

Name of the browser that generated the signal.

Firefox

Operating System

✘ Annotation

✅ Click

✘ Login

✅ Request

✘ Response

os_name

string

Name of the operating system used by the user.

Ubuntu

Device types

✘ Annotation

✅ Click

✘ Login

✅ Request

✘ Response

os_device

string

Type of the device used by the user.

Computer

Websites people are coming from

✘ Annotation

✅ Click

✘ Login

✘ Request

✘ Response

referrer_domain

string

URL of the page that the signal originated from.

http://www.google.com

Users

✘ Annotation

✅ Click

✘ Login

✅ Request

✘ Response

user_id

string

Unique ID for the user that generated the signal.

admin

User domains

N/A

domain

string

Domain of the user that generated the signal.

lucidworks.com

Types of event

✅ Annotation

✅ Click

✅ Login

✅ Request

✅ Response

type

string

Type of the signal.

click

Head Tail analysis

✘ Annotation

This report is populated by running a head tail job in Fusion. With a collection called foo, for example, run the foo_head_tail job.

N/A

✅ Click

type

string

N/A

✅ Click

query

string

N/A

✘ Login

N/A

✘ Request

N/A

✅ Response

count_i

number

N/A

Fusion jobs

The following table specifies required fields for specific Fusion jobs:

Fusion job Field Required

Fusion job	Field	Required
click_signal_aggr	`user_id`	no
`query`	yes
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	depends
`type`	yes
`session_id`	optional
`id`	N/A
`res_offset`	optional
`res_pos`	optional
`filters`	optional
`type = response`	depends
`dependency`	N/A
phrase_extraction	`user_id`	no
`query`	yes
`doc_id`	no
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	N/A
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
spell_correction	`user_id`	no
`query`	yes
`doc_id`	no
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	N/A
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
synonym_detection	`user_id`	no
`query`	yes
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	N/A
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	phrase, misspelling
head_tail	`user_id`	no
`query`	yes
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	N/A
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
_user_item_prefs_agg	`user_id`	yes
`query`	no
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	optional
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
user_query_history_aggr	`user_id`	yes
`query`	yes
`doc_id`	no
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	optional
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
als_recommender	`user_id`	yes
`query`	yes
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	optional
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
bpr_recommender	`user_id`	yes
`query`	yes
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	optional
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
query_query_collaborative	`user_id`	yes
`query`	yes
`doc_id`	no
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	optional
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
query_query_session_based	`user_id`	no
`query`	yes
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	no
`type`	yes
`session_id`	yes
`id`	N/A
`res_offset`	N/A
`res_pos`	N/A
`filters`	N/A
`type = response`	N/A
`dependency`	N/A
experiments	`user_id`	yes
`query`	yes
`doc_id`	yes
`count_i`	yes
`fusion_query_id`	yes
`type`	yes
`session_id`	yes
`id`	yes
`res_offset`	yes
`res_pos`	yes
`filters`	yes
`type = response`	yes
`dependency`	N/A

click_signal_aggr

user_id

query

yes

doc_id

yes

count_i

yes

fusion_query_id

depends

type

yes

session_id

optional

id

N/A

res_offset

optional

res_pos

optional

filters

optional

type = response

depends

dependency

N/A

phrase_extraction

user_id

query

yes

doc_id

count_i

yes

fusion_query_id

type

yes

session_id

N/A

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

spell_correction

user_id

query

yes

doc_id

count_i

yes

fusion_query_id

type

yes

session_id

N/A

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

synonym_detection

user_id

query

yes

doc_id

yes

count_i

yes

fusion_query_id

type

yes

session_id

N/A

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

phrase, misspelling

head_tail

user_id

query

yes

doc_id

yes

count_i

yes

fusion_query_id

type

yes

session_id

N/A

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

_user_item_prefs_agg

user_id

yes

query

doc_id

yes

count_i

yes

fusion_query_id

type

yes

session_id

optional

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

user_query_history_aggr

user_id

yes

query

yes

doc_id

count_i

yes

fusion_query_id

type

yes

session_id

optional

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

als_recommender

user_id

yes

query

yes

doc_id

yes

count_i

yes

fusion_query_id

type

yes

session_id

optional

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

bpr_recommender

user_id

yes

query

yes

doc_id

yes

count_i

yes

fusion_query_id

type

yes

session_id

optional

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

query_query_collaborative

user_id

yes

query

yes

doc_id

count_i

yes

fusion_query_id

type

yes

session_id

optional

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

query_query_session_based

user_id

query

yes

doc_id

yes

count_i

yes

fusion_query_id

type

yes

session_id

yes

id

N/A

res_offset

N/A

res_pos

N/A

filters

N/A

type = response

N/A

dependency

N/A

experiments

user_id

yes

query

yes

doc_id

yes

count_i

yes

fusion_query_id

yes

type

yes

session_id

yes

id

yes

res_offset

yes

res_pos

yes

filters

yes

type = response

yes

dependency

N/A