Create Ray Model Deployment

Managed Fusion 5.9.12 uses Ray to deploy machine learning (ML) models into production. This job deploys a Ray deployment into the Managed Fusion cluster. To create the job, sign in to Managed Fusion and click Collections > Jobs. Then click Add+ and in the Model Deployment Jobs section, select Create Ray Model Deployment. You can enter basic and advanced parameters to configure the job. If the field has a default value, it is populated when you click to add the job. To enter advanced parameters in the UI, click Advanced. For detailed information to deploy available Managed Fusion Ray models, see the:

Deploy the sentiment-general Model. This model is a general purpose sentiment prediction model that is trained on short sentences and text for intent prediction. For more information, see Sentiment analysis and prediction.
Deploy the sentiment-reviews Model. This model is trained on customer reviews and longer text. The model also supports attention weights output that can be used for highlighting the tokens that provide stronger sentiment. For more information, see Sentiment analysis and prediction.
Deploy a pre-trained scispaCy model. This model is trained on biomedical text.
Set Up a Pre-Trained Cold Start Model for Smart Answers. This model is trained on a variety of corpuses and tasks. For more information, see Smart answers.
Set Up a Pre-Trained Cold Start Model for Smart Answers. This model is trained on a variety of corpuses and tasks that supports 16 languages. For more information, see Smart answers.

Deploy the sentiment-general Model

This topic explains how to deploy the sentiment-general pre-trained sentiment prediction model. This is a general-purpose sentiment prediction model, trained on short sentences. It is suitable for short texts and for intent prediction.

Install the model in Managed Fusion

Navigate to Collections > Jobs.
Select New > Create Seldon Core Model Deployment.
Configure the job as follows:
- Job ID. The ID for this job, such as deploy-sentiment-general.
- Model Name. The model name of the Seldon Core deployment that will be referenced in the Machine Learning pipeline stage configurations, such as sentiment-general.
- Docker Repository. The value is lucidworks.
- Image Name. The value is sentiment-general:v1.0.
- Kubernetes Secret Name for Model Repo. This value is left empty.
- Output Column Names for Model. The value is [label, score].
Click Save.
Click Run > Start.

Configure the Machine Learning pipeline stages

In your index or query pipelines add Machine Learning stage and specify sentiment-general in the Model ID field (or a custom model name that was used during deployment).

Configure the Machine Learning index stage

In your index pipeline, click Add a Stage > Machine Learning.
In the Model ID field, enter the model name you configured above, such as sentiment-general.

In the Model input transformation script field, enter the following:

var modelInput = new java.util.HashMap()
modelInput.put("text", doc.getFirstFieldValue("text"))
modelInput

In the Model output transformation script field, enter the following:

doc.addField("sentiment_label_s", modelOutput.get("label")[0])
doc.addField("sentiment_score_d", modelOutput.get("score")[0])

Save the pipeline.

Configure the Machine Learning query stage

In your query pipeline, click Add a Stage > Machine Learning.
In the Model ID field, enter the model name you configured above, such as sentiment-general.

In the Model input transformation script field, enter the following:

var modelInput = new java.util.HashMap()
modelInput.put("text", request.getFirstParam("q"))
modelInput

In the Model output transformation script field, enter the following:

{/* // To put into request */}
request.putSingleParam("sentiment_label", modelOutput.get("label")[0])
request.putSingleParam("sentiment_score", modelOutput.get("score")[0])

{/* // To put into query context */}
context.put("sentiment_label", modelOutput.get("label")[0])
context.put("sentiment_score", modelOutput.get("score")[0])

{/* // To put into response documents. NOTE: This can be done only after Solr Query stage */}
var docs = response.get().getInnerResponse().getDocuments();
var ndocs = new java.util.ArrayList();

for (var i=0; i<docs.length;i++){
  var doc = docs[i];
  doc.putField("query_sentiment_label", modelOutput.get("label")[0])
  doc.putField("query_sentiment_score", modelOutput.get("score")[0])
  ndocs.add(doc);
}

response.get().getInnerResponse().updateDocuments(ndocs);

Save the pipeline.

Model output

Both of the pre-trained models output the following:

a label, negative or positive
a score from -2 to 2

The sentiment-reviews:v1.0 model also optionally outputs tokens and their corresponding attention weights, that is, the weight that each token carries in the sentiment prediction. The total of all attention weights is always 1, that is, each value represents a percentage of the total weight. In the example below, “awesome” has the highest weight because it expresses the strongest sentiment compared to other tokens in the string:

{/* // Input */}
text = "That is awesome!"

{/* // Output */}
sentiment_label = 'positive'
sentiment_score = 1.998
sentiment_attention_tokens = ['That', "'", 's', 'awesome', '!']
sentiment_attention_weights = [0.154, 0.078, 0.069, 0.444, 0.255]

Deploy the sentiment-reviews Model

This topic explains how to deploy the sentiment-reviews pre-trained sentiment prediction model. This model is trained on a variety of customer reviews and optimized for longer texts. It also supports attention weights output that can be used for highlighting the tokens that provide stronger sentiment; see Model output below for an example.

Install the model in Managed Fusion

Navigate to Collections > Jobs.
Select New > Create Seldon Core Model Deployment.
Configure the job as follows:
- Job ID. The ID for this job, such as deploy-sentiment-reviews
- Model Name. The model name of the Seldon Core deployment that will be referenced in the Machine Learning pipeline stage configurations, such as sentiment-reviews.
- Docker Repository. The value is lucidworks.
- Image Name. The value is sentiment-reviews:v1.0.
- Kubernetes Secret Name for Model Repo. The value is left empty.
- Output Column Names for Model. The value is [label, score, tokens, attention_weights].
Click Save.
Click Run > Start.

Configure the Machine Learning pipeline stages

You can put your sentiment prediction model to work using the Machine Learning index stage or Machine Learning query stage. You will specify the same Model Name that you used when you installed the model above.Generally, you only need to apply the model in the index pipeline to perform sentiment prediction on your content. Optionally, you can configure the query pipeline in a similar way, to perform sentiment prediction on incoming queries and outgoing responses and apply special treatment depending on the prediction.

Configure the Machine Learning index stage

In your index pipeline, click Add a Stage > Machine Learning.
In the Model ID field, enter the model name you configured above, such as sentiment-reviews.

In the Model input transformation script field, enter one of the following, depending on whether you want to output attention weights:

Without attention weights:

var modelInput = new java.util.HashMap()
modelInput.put("text", doc.getFirstFieldValue("text"))
modelInput

With attention weights:

var modelInput = new java.util.HashMap()
modelInput.put("text", doc.getFirstFieldValue("text"))
modelInput.put("attention_output", "true")
modelInput

In the Model output transformation script field, enter the following:

Without attention weights:

doc.addField("sentiment_label_s", modelOutput.get("label")[0])
doc.addField("sentiment_score_d", modelOutput.get("score")[0])

With attention weights:

doc.addField("sentiment_label_s", modelOutput.get("label")[0])
doc.addField("sentiment_score_d", modelOutput.get("score")[0])
doc.addField("sentiment_attention_tokens_ss", modelOutput.get("tokens"))
doc.addField("sentiment_attention_weights_ds", modelOutput.get("attention_weights"))

Save the pipeline.

Optional: Configure the Machine Learning query stage

In your query pipeline, click Add a Stage > Machine Learning.
In the Model ID field, enter the model name you configured above, such as sentiment-reviews.

In the Model input transformation script field, enter the following:

Without attention weights:

var modelInput = new java.util.HashMap()
modelInput.put("text", request.getFirstParam("q"))
modelInput

With attention weights:

var modelInput = new java.util.HashMap()
modelInput.put("text", request.getFirstParam("q"))
modelInput.put("attention_output", "true")
modelInput

In the Model output transformation script field, enter the following, noting the sections that need to be uncommented if you are using attention weights:

{/* // To put into request */}
request.putSingleParam("sentiment_label", modelOutput.get("label")[0])
request.putSingleParam("sentiment_score", modelOutput.get("score")[0])

{/* // With attention output also uncomment this */}
{/* // request.putSingleParam("sentiment_attention_tokens", modelOutput.get("tokens")) */}
{/* // request.putSingleParam("sentiment_attention_weights", modelOutput.get("attention_weights")) */}

{/* // To put into query context */}
context.put("sentiment_label", modelOutput.get("label")[0])
context.put("sentiment_score", modelOutput.get("score")[0])

{/* // With attention output also uncomment this */}
{/* // context.put("sentiment_attention_tokens", modelOutput.get("tokens")) */}
{/* // context.put("sentiment_attention_weights", modelOutput.get("attention_weights")) */}

{/* // To put into response documents (can be done only after Solr Query stage) */}
var docs = response.get().getInnerResponse().getDocuments();
var ndocs = new java.util.ArrayList();

var attention_tokens = modelOutput.get("tokens")
var attention_weights = modelOutput.get("attention_weights")
var attention_tokens_arr = new java.util.ArrayList(attention_tokens.size());
var attention_weights_arr = new java.util.ArrayList(attention_weights.size());
for ( i = 0; i < attention_tokens.size(); i++) {
 attention_tokens_arr.add(attention_tokens[i])
 attention_weights_arr.add(attention_weights[i])
}

for (var i=0; i<docs.length;i++){
 var doc = docs[i];
 doc.putField("query_sentiment_label", modelOutput.get("label")[0])
 doc.putField("query_sentiment_score", modelOutput.get("score")[0])
{/*   // With attention output also uncomment this */}
{/*   // doc.putField("query_sentiment_attention_tokens", attention_tokens_arr) */}
{/*   // doc.putField("query_sentiment_attention_weights", attention_weights_arr) */}
 ndocs.add(doc);
}

response.get().getInnerResponse().updateDocuments(ndocs);

Save the pipeline.

Model output

Both of the pre-trained models output the following:

a label, negative or positive
a score from -2 to 2

{/* // Input */}
text = "That is awesome!"

{/* // Output */}
sentiment_label = ‘positive’
sentiment_score = 1.998
sentiment_attention_tokens = ['That', "'", 's', 'awesome', '!']
sentiment_attention_weights = [0.154, 0.078, 0.069, 0.444, 0.255]

Deploy a pre-trained scispaCy model

This article uses a pre-packaged model, which you do not need to download to deploy. To use the pre-packaged model, skip to Deploy model to Fusion. The section Create the model describes how to complete this process on your own.

Create the model (OPTIONAL)

Skip this section to use the pre-packaged model.

Download the scispacy.ipynb file and open it in Jupyter Notebook (or a similar alternative).
Follow the steps in the notebook, substituting your custom values as needed.

Deploy model to Fusion

Navigate to Collections > Jobs.
Click the Add button.
Select the Create Seldon Core Model Deployment under Model Deployment Jobs.
Enter the values for your model deployment. If you are using the pre-packaged model, use the following values:
Parameter Value
Jobs ID scispacymodel-seldon-deployment
Model Name scispacymodel
Docker Repository shahanesanket
Image Name scispacymodel-grpc:1.0
Output Column Names for Model [entities]
Run the job by clicking Run and selecting Start.

Parameter	Value
Jobs ID	`scispacymodel-seldon-deployment`
Model Name	`scispacymodel`
Docker Repository	`shahanesanket`
Image Name	`scispacymodel-grpc:1.0`
Output Column Names for Model	`[entities]`

When the job completes successfully, the model is deployed. Check the list of microservices to verify:

Import sample data

Download and save the sample data file sampleJSON_body_content.csv.
Navigate to Indexing > Datasources.
Click the Add button and select File Upload V2.
Click Browse, select the sampleJSON_body_content.csv file, and click Open. Click the Upload File button to complete the upload process.
Assign a value to the Datasource ID parameter. This article uses the ID sample-data.
Click the Save button.

Create a Machine Learning stage in the Index Workbench

Navigate to Indexing > Index Workbench.
Click the Load button.
Choose the datasource you created.
Click the Add a Stage button.
Choose the Machine Learning stage.
In the Model ID field, enter scispacymodel.

In the Model input transformation script field, enter the following script:

var modelInput = new java.util.HashMap()
var list = new java.util.ArrayList()
list.add(doc.getFirstFieldValue("body_t"))
modelInput.put("text", list)
modelInput

In the Model output transformation script field, enter the following script:
```
doc.addField("entities_ss", modelOutput.get("entities"))
```
Click the Apply button.

Verify results

Click the Start Job button, and allow the job to finish.
Check the simulated results. If everything was successful, the results will resemble this:

Set Up a Pre-Trained Cold Start Model for Smart Answers

UI tour

Index data

Query data

Metrics and analytics

Improve your queries

Administration

Developer documentation

Machine learning

Neural Hybrid Search

Release notes

FAQs

Install the model in Managed Fusion

Configure the Machine Learning pipeline stages

Configure the Machine Learning index stage

Configure the Machine Learning query stage

Model output

Install the model in Managed Fusion

Configure the Machine Learning pipeline stages

Configure the Machine Learning index stage

Optional: Configure the Machine Learning query stage

Model output

Create the model (OPTIONAL)

Deploy model to Fusion

Import sample data

Create a Machine Learning stage in the Index Workbench

Verify results

Deploy a pre-trained cold-start model into Managed Fusion

Next steps

Deploy a pre-trained cold-start model into Managed Fusion

Next steps

Configuration properties

UI tour

Index data

Query data

Metrics and analytics

Improve your queries

Administration

Developer documentation

Machine learning

Neural Hybrid Search

Release notes

FAQs

​Configuration properties

Configuration properties