Appkit can capture any and all interaction users have with an Appkit powered application. This includes what queries were typed, filters applied, sort order used, and links opened. The information is stored in a highly scalable embedded search index which can be used to:

Data mine or analyze information through a regular Appkit user interface
Empower and aid users in content discovery by grouping together similar queries using statistical methods and suggesting alternatives to users seeking similar content
Perform auditing and compliance functions in secure applications

Activity processing

The activity tracking framework supports these endpoints for processing activity events:

Tracking Fusion Signals
Logging

Tracking Fusion Signals

Appkit provides support for the Fusion REST Signals API, allowing user clicks and the creation of annotations (bookmarks, topics etc) to be recorded and tracked.

Incorporate signals

This section describes how to incorporate Fusion Signals activity tracking into your Appkit application.

1. Add the Fusion Signals dependency

Signals tracking in Fusion requires the Fusion Signals module. To enable this module, insert these within the dependencies tag of your pom.xml:

<dependency>
    <groupId>twigkit</groupId>
    <artifactId>twigkit.activity.tracker</artifactId>
    <version>${project.parent.version}</version>
</dependency>

<dependency>
    <groupId>twigkit</groupId>
    <artifactId>twigkit.message.service.fusion.producer</artifactId>
    <version>${project.parent.version}</version>
</dependency>

2. Add fusion.conf to activity tracking resources

To access this module when Appkit starts up, create a fusion.conf file in resources/conf/message/service/fusion.conf. In that file, configure the query-profile. This is the REST-API endpoint that will ingest signal data into a signals collection. For example:

query-profile: my-query-profile
signals-index-pipeline: _signals_ingest
commit: true
async: true

Fusion Signals config

Here, image_catalog is the name of a primary collection that will be used to generate an auxiliary collection consisting of activity tracking data.When accessing Fusion using a service account, the credentials will be pulled from the configuration given in the services/api/fusion.conf file, which should contain these parameters to enable basic authentication:

impersonate: true
userName: joebloggs
password: Enc(<encoded password>)

For more information about how authentication is used when accessing the Fusion Signals endpoint, see the Authentication section below.Several optional parameters are also provided as shown. The first, signals-index-pipeline, can be used to define an index pipeline that will to be used to convert the raw JSON signal data into a set of Solr documents. As stated in the Fusion Signals API documentation, if no pipeline is defined, then the preconfigured _signals_ingest pipeline will be used. Both the remaining two parameters, commit and async, are booleans. If commit is set to true, a Solr commit will occur at the end of indexing allowing a persistent record of the activity to be kept. If async is set to true, signals will be indexed in asynchronous mode. In this mode, an autoCommit is issued with each signal and failures are not reported. The default is false.

Creating a collection for signals tracking

Using the Fusion UI, when a primary data collection is created, this also creates a collection for the signals. By default, the name of this signals collection is by <primary collection>_signals, where <primary collection> should be substituted with the name of the primary data collection. After data has been indexed and ingested into the primary collection, the Appkit Fusion Signals module can be used to populate the signals collection.

Tracking user clicks and annotations

Currently, Appkit will send information on user clicks that trigger URL requests as well as when a user creates an annotation, for example, a bookmark or comment.A signal event of type click will include, amongst other things, information about the URL that was clicked as well as the number of times that URL was clicked.A signal event of type annotation will include, amongst other things, information about the annotation target, the collection, and the creator.For a full list of properties that are stored, refer to the signals collection associated with your application.

Authentication

Service account impersonation requires credentials for the service account and these are stored in the Fusion platform configuration. These credentials will be pulled from the configuration given in the services/api/fusion.conf file.If the credentials for a service account are not provided or cannot be found, a check will be made to see if a Fusion session cookie is available. Typically, this will be present when the user is known to the Fusion server and the Session API has been set up accordingly.

Logging

Appkit provides support for tracking the activity of specific events and logging them either to the console or a file depending on how your application is set up.

Log activity events

To log activity events, perform the steps in this section.

1: Update the POM

To log activity events, first add these dependencies to your pom.xml file:

<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.1.1</version>
</dependency>
<dependency>
    <groupId>twigkit</groupId>
    <artifactId>twigkit.activity.logger</artifactId>
    <version>${project.parent.version}</version>
</dependency>

2: Create an activity logging configuration file

In conf/activity/tracking, place a logger.conf file. In the file, set the property logging-enabled equal to true:

logging-enabled: true

If logging-enabled is set to false, then all logging will be disabled.

3: Create a logback.xml file

Create a logback.xml file in /resources and configure this to either output activity data to the console or a file (or both).For example, to output logging activity to the console:

<configuration debug="true" scan="true">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <withJansi>false</withJansi>
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} %highlight(%-5level) %cyan(%logger{36}) - %msg %n</pattern>
        </encoder>
    </appender>

    <logger name="twigkit" level="INFO"/>
    <logger name="twigkit.activity.subscribers" level="TRACE"/>

    <root level="ERROR">
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>

To output logging activity to a file, while all other logging is redirected to the console:

<configuration debug="true" scan="true">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <withJansi>false</withJansi>
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} %highlight(%-5level) %cyan(%logger{36}) - %msg %n</pattern>
        </encoder>
    </appender>

    <appender name="activity-log" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>./logs/activity.log</file>
        <append>true</append>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">

            <!-- Daily rollover. Make sure the path matches the one in the file element or else
             the rollover logs are placed in the working directory. -->
            <fileNamePattern>./logs/activity_%d{yyyy-MM-dd}.%i.log</fileNamePattern>

            <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>5MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>

            <!-- keep 30 days' worth of history -->
            <maxHistory>30</maxHistory>

        </rollingPolicy>
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} - %msg %n</pattern>
        </encoder>
    </appender>

    <logger name="twigkit" level="INFO"/>

    <logger name="twigkit.activity.subscribers" level="TRACE" additivity="false">
        <appender-ref ref="activity-log"/>
    </logger>

    <root level="ERROR">
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>

The location of the file is set in the file tag and is relative to the root of the application.

Example

A typical example of what you might see in the log is:

33:51.711 - type:response,user:user,query:US,hits:2437,page:1,time:104
33:52.907 - type:response,user:user,query:US,hits:2437,page:1,time:106
33:54.191 - type:click,username:user,URL:http://twigkit.com/kapps_kyc/customers/519/,host:twigkit.com

Where, for example, the type refers to whether the event is a click or response, the user is the name of the user, query shows the query terms, hits is the number of hits returned in the response, page is the page number, and URL is the URL associated with the click.

Query Tracking

To track user-generated queries on your page, add a <track:query> tag with a reference to the query object that captures the user’s input:

<track:query query="query" path="/search" page-title="Search Page"></track:query>

To see all related attributes, see the track:query tag documentation.

Click Tracking

To track click events, wrap a <track:clicks> tag around any result list to be tracked:

<track:clicks query="query" type="People Results">
<search:result-list response="response">
    <search:result>
        ...
    </search:result>
</search:result-list>
</track:clicks>

To see all related attributes, see the track:clicks tag documentation. Alternatively, to track a static URL, wrap the address in your href as shown. Optionally, you can use the _type parameter to provide a click type: /twigkit/secure/services/url?url=http://www.google.com&_type=Other_Sites"

Introduction to Fusion

Getting Data In

Getting Data Out

Operations

Reference

Developer Docs

Neural Hybrid Search

Release Notes

Activity Tracking

Activity processing

Incorporate signals

1. Add the Fusion Signals dependency

2. Add fusion.conf to activity tracking resources

Fusion Signals config

Creating a collection for signals tracking

Tracking user clicks and annotations

Authentication

Log activity events

1: Update the POM

2: Create an activity logging configuration file

3: Create a logback.xml file

Example

Query Tracking

Click Tracking

Introduction to Fusion

Getting Data In

Getting Data Out

Operations

Reference

Developer Docs

Neural Hybrid Search

Release Notes

​Activity processing

​Query Tracking

​Click Tracking

Activity processing

Query Tracking

Click Tracking