Async Prediction APILucidworks AI

Table of Contents

Prerequisites
Common POST request parameters and fields
- modelConfig
Async prediction use case by modelid
POST response parameters and fields
- Example POST request
- Example GET request
Pass-through use case
- Unique values for the pass-through use case
Retrieval augmented generation (RAG) use case
Standalone query rewriter use case
Summarization use case
Keyword Extraction use case
Named Entity Recognition use case

The Lucidworks AI Async Prediction API is used to send asynchronous API calls that run predictions on specific models.

The Lucidworks AI Async Prediction API contains two requests:

POST request - submits a prediction task for a specific useCase and modelId. The API responds with the following information:
- predictionId. A unique UUID for the submitted prediction task that can be used later to retrieve the results.
- status. The current state of the prediction task.
GET request - uses the predictionId you submit from a previously-submitted POST request and returns the results associated with that previous request.

Lucidworks deployed the mistral-7b-instruct and llama-3-8b-instruct models. The Lucidworks AI Use Case API returns a list of all supported models. For more information about supported models, see Generative AI models.

You can enter the values returned in the Lucidworks AI Use Case API for the USE_CASE and MODEL_ID fields in the /async-prediction use case POST requests.

The generic path for the Async Prediction API is /ai/async-prediction/USE_CASE/MODEL_ID.

The use cases based on the generic path are:

Pass-through use case
Retrieval augmented generation (RAG) use case
Standalone query rewriter use case
Summarization use case
Keyword extraction use case
Named entity recognition (NER) use case

Prerequisites

To use this API, you need:

The unique APPLICATION_ID for your Lucidworks AI application. For more information, see credentials to use APIs.
A bearer token generated with a scope value of machinelearning.predict. For more information, see Authentication API.
Other required fields specified in each individual use case.

Common POST request parameters and fields

Some parameters of the /ai/async-prediction/USE_CASE/MODEL_ID POST request are common to all of the generative AI (GenAI) use cases, including the modelConfig parameter. If you do not enter values, the following defaults are used.

"modelConfig":{
  "temperature": 0.7,
  "topP": 1.0,
  "presencePenalty": 0.0,
  "frequencyPenalty": 0.0,
  "maxTokens": 256
}

Also referred to as hyperparameters, these fields set certain controls on the response of a LLM:

Field Description

Field	Description
temperature	A sampling temperature between 0 and 2. A higher sampling temperature such as 0.8, results in more random (creative) output. A lower value such as 0.2 results in more focused (conservative) output. A lower value does not guarantee the model returns the same response for the same input.
topP	A floating-point number between 0 and 1 that controls the cumulative probability of the top tokens to consider, known as the randomness of the LLM’s response. This parameter is also referred to as top probability. Set `topP` to 1 to consider all tokens. A higher value specifies a higher probability threshold and selects tokens whose cumulative probability is greater than the threshold. The higher the value, the more diverse the output.
presencePenalty	A floating-point number between -2.0 and 2.0 that penalizes new tokens based on whether they have already appeared in the text. This increases the model’s use of diverse tokens. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens.
frequencyPenalty	A floating-point number between -2.0 and 2.0 that penalizes new tokens based on their frequency in the generated text. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens.
maxTokens	The maximum number of tokens to generate per output sequence. The value is different for each model. Review individual model specifications when the value exceeds 2048.
apiKey	The optional parameter is only required when the specified model is used for prediction. This secret value is specified in the external model. For: OpenAI models, `"apiKey"` is the value in the model’s `"[OPENAI_API_KEY]"` field. For more information, see Authentication API keys. Azure OpenAI models, `"apiKey"` is the value generated by Azure in either the model’s `"[KEY1 or KEY2]"` field. For requirements to use Azure models, see Generative AI models. Google VertexAI models, `"apiKey"` is the value in the model’s `"[BASE64_ENCODED_GOOGLE_SERVICE_ACCOUNT_KEY]"` field. For more information, see Create and delete Google service account keys. The parameter (for OpenAI, Azure OpenAI, or Google VertexAI models) is only available for the following use cases: Pass-through RAG Standalone query rewriter Summarization Keyword extraction NER
azureDeployment	The optional `"azureDeployment": "[DEPLOYMENT_NAME]"` parameter is the deployment name of the Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.
azureEndpoint	The optional `"azureEndpoint": "[ENDPOINT]"` parameter is the URL endpoint of the deployed Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.
googleProjectId	The optional `"googleProjectId": "[GOOGLE_PROJECT_ID]"` parameter is only required when a Google VertexAI model is used for prediction.
googleRegion	The optional `"googleRegion": "[GOOGLE_PROJECT_REGION_OF_MODEL_ACCESS]"` parameter is only required when a Google VertexAI model is used for prediction. The possible region values are: us-central1 us-west4 northamerica-northeast1 us-east4 us-west1 asia-northeast3 asia-southeast1 asia-northeast

temperature

A sampling temperature between 0 and 2. A higher sampling temperature such as 0.8, results in more random (creative) output. A lower value such as 0.2 results in more focused (conservative) output. A lower value does not guarantee the model returns the same response for the same input.

topP

A floating-point number between 0 and 1 that controls the cumulative probability of the top tokens to consider, known as the randomness of the LLM’s response. This parameter is also referred to as top probability. Set topP to 1 to consider all tokens. A higher value specifies a higher probability threshold and selects tokens whose cumulative probability is greater than the threshold. The higher the value, the more diverse the output.

presencePenalty

A floating-point number between -2.0 and 2.0 that penalizes new tokens based on whether they have already appeared in the text. This increases the model’s use of diverse tokens. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens.

frequencyPenalty

A floating-point number between -2.0 and 2.0 that penalizes new tokens based on their frequency in the generated text. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens.

maxTokens

The maximum number of tokens to generate per output sequence. The value is different for each model. Review individual model specifications when the value exceeds 2048.

apiKey

The optional parameter is only required when the specified model is used for prediction. This secret value is specified in the external model. For:

OpenAI models, "apiKey" is the value in the model’s "[OPENAI_API_KEY]" field. For more information, see Authentication API keys.
Azure OpenAI models, "apiKey" is the value generated by Azure in either the model’s "[KEY1 or KEY2]" field. For requirements to use Azure models, see Generative AI models.
Google VertexAI models, "apiKey" is the value in the model’s

"[BASE64_ENCODED_GOOGLE_SERVICE_ACCOUNT_KEY]" field. For more information, see Create and delete Google service account keys.

The parameter (for OpenAI, Azure OpenAI, or Google VertexAI models) is only available for the following use cases:

Pass-through
RAG
Standalone query rewriter
Summarization
Keyword extraction
NER

azureDeployment

The optional "azureDeployment": "[DEPLOYMENT_NAME]" parameter is the deployment name of the Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.

azureEndpoint

The optional "azureEndpoint": "[ENDPOINT]" parameter is the URL endpoint of the deployed Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.

googleProjectId

The optional "googleProjectId": "[GOOGLE_PROJECT_ID]" parameter is only required when a Google VertexAI model is used for prediction.

googleRegion

The optional "googleRegion": "[GOOGLE_PROJECT_REGION_OF_MODEL_ACCESS]" parameter is only required when a Google VertexAI model is used for prediction. The possible region values are:

us-central1
us-west4
northamerica-northeast1
us-east4
us-west1
asia-northeast3
asia-southeast1
asia-northeast

Async prediction use case by modelid

The /ai/async-prediction/USE_CASE/MODEL_ID request submits a prediction task for a specific useCase and modelId. Upon submission, a successful response includes a unique predictionId and a status. The predictionId can be used later in the GET request to retrieve the results.

Unique fields and values in the request are described in each use case.

POST response parameters and fields

The response to the POST /ai/async-prediction/USE_CASE/MODEL_ID requests are as follows:

Field Description

Field	Description
predictionId	The universal unique identifier (UUID) returned in the POST request. This UUID is required in the GET request to retrieve results. For example, fd110486-f168-47c0-a419-1518a4840589.
status	The current status of the prediction. Values are: SUBMITTED - The POST request was successful and the response has returned the `predictionId` and `status`. The `predictionId` is used in the GET request. ERROR - An error was generated when the request was sent. READY - The results associated with the `predictionId` are available and ready to be retrieved.

predictionId

The universal unique identifier (UUID) returned in the POST request. This UUID is required in the GET request to retrieve results. For example, fd110486-f168-47c0-a419-1518a4840589.

status

The current status of the prediction. Values are:

SUBMITTED - The POST request was successful and the response has returned the predictionId and status. The predictionId is used in the GET request.
ERROR - An error was generated when the request was sent.
READY - The results associated with the predictionId are available and ready to be retrieved.

Example POST request

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/USE_CASE/MODEL_ID \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
  "batch": [
    {
      "text": "Content for the model to analyze."
    }
  ],
  "useCaseConfig": [
    {
      "useSystemPrompt": true
    }
  ],
  "modelConfig": [
    {
      "temperature": 0.8,
      "topP": 1,
      "presencePenalty": 2,
      "frequencyPenalty": 1,
      "maxTokens": 1
    }
  ]
}'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The response returned is unique to each use case, and is described in the section for the use case.

Pass-through use case

The passthrough use case lets you use the service as a proxy to the LLM. The service sends text (no additional prompts or other information) to the LLM and returns a response.

Unique values for the pass-through use case

The parameters available in the passthrough use case are:

"useCaseConfig": "useSystemPrompt": boolean
"useCaseConfig": "dataType": "string"

If both useSystemPrompt and dataType are present, the value in dataType is used.

Use System Prompt

"useCaseConfig": "useSystemPrompt": boolean

This parameter can be used:

If custom prompts are needed, or if the prompt response format needs to be manipulated.
But the prompt length may increase response time.

Some models, such as the mistral-7b-instruct and llama-3-8b-instruct, generate more effective results when system prompts are included in the request.

If "useSystemPrompt": true, the LLM input is automatically wrapped into a model-specific prompt format with a generic system prompt before passing it to the model or third-party API.

If "useSystemPrompt": false, the batch.text value serves as the prompt for the model. The LLM input must accommodate model-specific requirements because the input is passed as is.

Examples:

The format for the mistral-7b-instruct model must be specific to Mistral:

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2
The format for the llama-3-8b-instruct model must be specific to Llama:

https://huggingface.co/blog/llama3#how-to-prompt-llama-3
The text input for OpenAI models must be valid JSON to match the OpenAI API specification:

https://platform.openai.com/docs/api-reference/chat/create
The format for the Google VertexAI models must adhere to the guidelines at:

https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini

POST example

This useSystemPrompt POST example does not include modelConfig parameters, but you can submit requests that include parameters described in Common POST request parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/passthrough/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
      "text": "who was the first president of the USA?"
      }
    ],
  "useCaseConfig": {
    "useSystemPrompt": true
    }
  }'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

GET example

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The following is an example response:

{
  "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  "status": "READY",
  "predictions": [
  {
    "response": "The first President of the United States was George Washington.",
    "tokensUsed": {
      "promptTokens": 49,
      "completionTokens": 11,
      "totalTokens": 60
      }
    }
  ]
}

Data Type

"useCaseConfig": "dataType": "string"

This optional parameter enables model-specific handling in the Async Prediction API to help improve model accuracy. Use the most applicable fields based on available dataTypes and the dataType value that best aligns with the text sent to the Async Prediction API.

The values for dataType in the Passthrough use case are:

"dataType": "text"

This value is equivalent to "useSystemPrompt": true and is a pre-defined, generic prompt.
"dataType": "raw_prompt"

This value is equivalent to "useSystemPrompt": false and is passed directly to the model or third-party API.
"dataType": "json_prompt"

This value follows the generics that allow three roles:
- system
- user
  - Only the last user message is truncated.
  - If the API does not support system prompts, the user role is substituted for the system role.
- assistant
  - If the last message role is assistant, it is used as a pre-fill for generation and is the first generated token the model uses. The pre-fill is prepended to the model output, which makes models less verbose and helps enforce specific outputs such as YAML.
  - The Google VertexAI does not support generation pre-fills, so an exception error is generated.
    
    This follows the HuggingFace template contraints at Hugging Face chat templates.
  - Additional json_prompt information:
    
    Consecutive messages for the same role are merged.
    
    You can paste the information for a hosted model into the json_prompt value and change the model name in the stage.

POST example

This "dataType": "json_prompt"` example does not include modelConfig parameters, but you can submit requests that include parameters described in Common parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/passthrough/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
   "batch": [
     {
       "text": "[{\"role\": \"system\", \"content\": \"You are a helpful utility program instructed to accomplish a word correction task. Provide the most likely suggestion to the user without an preamble or elaboration.\"}, {\"role\": \"user\", \"content\": \"misspeled\"}, {\"role\": \"assistant\", \"content\": \"CORRECT:\"}]"
       }
   ],
   "useCaseConfig" :{
     "dataType" : "json_prompt"
   }
}'

The following is an example response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

GET example

curl --request GET \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/fd110486-f168-47c0-a419-1518a4840589 \
  --header 'Authorization: Bearer ACCESS_TOKEN'

The following is an example response:

{
    "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
    "status": "READY",
    "predictions": [
        {
            "tokensUsed": {
                "promptTokens": 51,
                "completionTokens": 4,
                "totalTokens": 55
            },
            "response": "CORRECT: misspelled"
        }
    ]
}

Retrieval augmented generation (RAG) use case

The rag use case uses candidate documents that are inserted into a LLM’s context to ground the generated response to those documents instead of generating an answer from details stored in the LLM’s trained weights. This helps prevent frequency of LLM hallucinative responses. This type of search adds guardrails so the LLM can search private data collections.

The rag search can perform queries against external documents passed in as part of the request.

This use case can be used:

To generate answers based on the context of the responses collected (corpus)
To generate a response based on the context from responses to a previous request

Example POST request using external documents

Unique values for the external documents RAG use case

The values available in this use case (that may not be available in other use cases) are:

Parameter Value

Parameter	Value
"documents"	This array is passed in the `batch` object. Allowed LLM context length limits the number of documents to 3. The parameter can be used in the query side by clicking the "Include response documents" check box. The array contains the following parameters: "body": <contents of doc> "source": <url/id of doc - used in generating SOURCES cite list> "title": <title> "date": <creation date of the document in epoch time format>
"useCaseConfig"	The supported parameters are: "memoryUuid": "string" This parameter is optional, and is used when chat history reference information from a previous GenAI interaction is available. extractRelevantContent": boolean This parameter can be used in a query and on documents. The default is false.

"documents"

This array is passed in the batch object. Allowed LLM context length limits the number of documents to 3. The parameter can be used in the query side by clicking the "Include response documents" check box. The array contains the following parameters:

"body": <contents of doc>
"source": <url/id of doc - used in generating SOURCES cite list>
"title": <title>
"date": <creation date of the document in epoch time format>

"useCaseConfig"

The supported parameters are:

"memoryUuid": "string"

This parameter is optional, and is used when chat history reference information from a previous GenAI interaction is available.
extractRelevantContent": boolean

This parameter can be used in a query and on documents. The default is false.

The following is an example request. This example does not include modelConfig parameters, but you can submit requests that include parameters described in Common POST request parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/rag/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
      "text": "Why did I go to Germany?",
      "documents": [{
        "body": "I'm off to Germany to go to the Oktoberfest!",
        "source": "http://example.com/112",
        "title": "Off to Germany!",
        "date": 1104537600
        }]
      }
    ],
  }'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The response includes the:

Generated answer
SOURCES line of text that contains the URL of the documents used to generate the answer
Metadata about the response:
- memoryUuid that can be used to retrieve the LLM’s chat history
- Count of tokens used to complete the query

If the LLM cannot determine a response, it returns a response where the first prediction field contains I don’t know.

The following is an example response:

{
  "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  "status": "READY",
  "predictions": [
     {
      "response": "ANSWER: \"I went to Germany to visit family.\"\nSOURCES: [\"http://example.com/112\"]",
      "tokensUsed": {
        "promptTokens": 202,
        "completionTokens": 17,
        "totalTokens": 219
        },
      "answer": "I went to Germany to visit family.",
      "sources": [
          "http://example.com/112"
      ],
      "memoryUuid": "27a887fe-3d7c-4ef0-9597-e2dfc054c20e"
    }
  ]
  }

Example POST request using chat history

Unique values for the chat history RAG use case

The values available in this use case (that may not be available in other use cases) are:

Parameter	Value
"documents"	The array contains the following parameters: "body": <contents of doc> "source": <url/id of doc - used in generating SOURCES cite list> "title": <title> "date": <creation date of the document in epoch time format>
"useCaseConfig"	"memoryUuid": "string" This parameter is optional, and is used when previous chat history reference information is available.

Parameter

Value

"documents"

The array contains the following parameters:

"body": <contents of doc>
"source": <url/id of doc - used in generating SOURCES cite list>
"title": <title>
"date": <creation date of the document in epoch time format>

"useCaseConfig"

"memoryUuid": "string"

This parameter is optional, and is used when previous chat history reference information is available.

When using the RAG search, the LLM service stores the query and its response in a cache. In addition to the response, it also returns a UUID value in the memoryUuid field. If the UUID is passed back in a subsequent request, the LLM uses the cached query and response as part of its context. This lets the LLM be used as a chatbot, where previous queries and responses are used to generate the next response.

curl --request POST \
  --url  https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/rag/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
    "text": "What is RAG?",
    "documents": [{
      "body":"Retrieval Augmented Generation, known as RAG, a framework promising to optimize generative AI and ensure its responses are up-to-date, relevant to the prompt, and most important",
      "source":"http://rag.com/115",
      "title":"What is Retrieval Augmented Generation",
      "date":"1104537600"
      }]
    }
  ],
    "useCaseConfig": {
      "memoryUuid": "27a887fe-3d7c-4ef0-9597-e2dfc054c20e"
    }
    }'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The following is an example response:

{
  "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  "status": "READY",
  "predictions": [
    {
      "response": "ANSWER: \"Retrieval Augmented Generation, known as RAG, is a framework promising to optimize generative AI and ensure its responses are up-to-date, relevant to the prompt, and most important.\"\nSOURCES: [\"http://rag.com/115\"]",
      "tokensUsed": {
       "promptTokens": 238,
       "completionTokens": 54,
       "totalTokens": 292
       },
      "answer": "Retrieval Augmented Generation, known as RAG, is a framework promising to optimize generative AI and ensure its responses are up-to-date, relevant to the prompt, and most important.",
      "sources": [
          "http://rag.com/115"
      ],
      "memoryUuid": "27a887fe-3d7c-4ef0-9597-e2dfc054c20e"
      }
    ]
  }

Standalone query rewriter use case

The standalone_query_rewriter use case rewrites the text in reference to the context based on the memoryUuid.

This use case can be used to rewrite the context from a previous response into a standalone query.

Unique values for the standalone query rewriter use case

The values available in this use case (that may not be available in other use cases) are:

Parameter	Value
"text"	Free-form content contained in the document.
"useCaseConfig"	"memoryUuid": "string"

Parameter

Value

"text"

Free-form content contained in the document.

"useCaseConfig"

"memoryUuid": "string"

Example POST request

This example does not include modelConfig parameters, but you can submit requests that include parameters described in Common POST request parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/standalone_query_rewriter/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
      "text": "Is it a framework?"
      },
  ],
    "useCaseConfig": {
      "memoryUuid": "27a887fe-3d7c-4ef0-9597-e2dfc054c20e"
    }
}'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The following is an example response:

{
  "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  "status": "READY",
  "predictions": [
    {
      "response": "Is RAG a framework?",
      "tokensUsed": {
       "promptTokens": 245,
       "completionTokens": 6,
       "totalTokens": 251
       },
    }
  ]
}

Summarization use case

In the summarization use case, the LLM ingests text and returns a summary of the text as a response.

The context length is 2048 tokens. No options can be configured.

This use case can be used to condense the text of a large document into a summarized response.

Unique values for the summarization use case

The values available in this use case (that may not be available in other use cases) are:

Parameter	Value
"text"	Free-form content contained in the document.
"useCaseConfig"	"maxWords": integer This parameter specifies the maximum number of words returned in the summary when generated by the model.

Parameter

Value

"text"

Free-form content contained in the document.

"useCaseConfig"

"maxWords": integer

This parameter specifies the maximum number of words returned in the summary when generated by the model.

Example POST request

This example does not include modelConfig parameters, but you can submit requests that include parameters described in Common POST request parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/summarization/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
      "text": "Nearly ten years had passed since the Dursleys had woken up to find their nephew on the front step, but Privet Drive had hardly changed at all. The sun rose on the same tidy front gardens and lit up the brass number four on the Dursleys''' front door; it crept into their living room, which was almost exactly the same as it had been on the night when Mr. Dursley had seen that fateful news report about the owls. Only the photographs on the mantelpiece really showed how much time had passed. Ten years ago, there had been lots of pictures of what looked like a large pink beach ball wearing different-colored bonnets - but Dudley Dursley was no longer a baby, and now the photographs showed a large blond boy riding his first bicycle, on a carousel at the fair, playing a computer game with his father, being hugged and kissed by his mother. The room held no sign at all that another boy lived in the house, too."
      },
  ],
    "useCaseConfig": {
      "maxWords": 100
    }
}'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The following is an example response:

{
  "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  "status": "READY",
  "predictions": [
    {
      "tokensUsed": {
       "promptTokens": 248,
       "completionTokens": 65,
       "totalTokens": 313
       },
      "response": "A decade after the Dursleys found their nephew on their doorstep, Privet Drive remains largely unchanged. The only noticeable difference is in the family photographs, which now depict Dudley Dursley as a large blond boy engaged in various activities, replacing his baby pictures. There is no indication of another boy living in the house."
      }
    ]
  }

Keyword Extraction use case

In the keyword_extraction use case, the LLM ingests text and returns a JSON response that contains a list of keywords extracted from the text. No options can be configured.

This use case can be used:

To extract keywords to create new facets
To extract keywords from each document:
- To write the keyword into a new field that subsequently searches large content documents more efficiently
- To boost specific keyword content
- For document clustering

Unique values for the keyword extraction use case

The values available in this use case (that may not be available in other use cases) are:

Parameter	Value
"text"	Free-form content contained in the document.
"useCaseConfig"	"maxKeywords": integer This parameter specifies the maximum number of keywords that can be extracted, even though the model may contain more than the maximum number specified in this field.

Parameter

Value

"text"

Free-form content contained in the document.

"useCaseConfig"

"maxKeywords": integer

This parameter specifies the maximum number of keywords that can be extracted, even though the model may contain more than the maximum number specified in this field.

Example POST request

This example does not include modelConfig parameters, but you can submit requests that include parameters described in Common POST request parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/keyword_extraction/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
      "text": "Joseph Robinette Biden Jr.is an American politician who is the 46th and current president of the United States. Ideologically a moderate member of the Democratic Party, he previously served as the 47th vice president from 2009 to 2017 under President Barack Obama and represented Delaware in the United States Senate from 1973 to 2009.Born in Scranton, Pennsylvania, Biden moved with his family to Delaware in 1953. He studied at the University of Delaware before earning his law degree from Syracuse University. He was elected to the New Castle County Council in 1970 and to the U.S. Senate in 1972. As a senator, Biden drafted and led the effort to pass the Violent Crime Control and Law Enforcement Act and the Violence Against Women Act. He also oversaw six U.S. Supreme Court confirmation hearings, including the contentious hearings for Robert Bork and Clarence Thomas. Biden ran unsuccessfully for the Democratic presidential nomination in 1988 and 2008. In 2008, Obama chose Biden as his running mate, and Biden was a close counselor to Obama during his two terms as vice president. In the 2020 presidential election, Biden and his running mate, Kamala Harris, defeated incumbents Donald Trump and Mike Pence. Biden is the second Catholic president in U.S. history (after John F. Kennedy), and his politics have been widely described as profoundly influenced by Catholic social teaching."
      }
    ],
    "useCaseConfig": {
      "maxKeywords": 100
    }
  }'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The following is an example response:

{
  "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  "status": "READY",
  "predictions": [
    {
      "response": "Joseph Robinette Biden Jr., 46th president of the United States, Democratic Party, Vice President, Barack Obama, Delaware, University of Delaware, Syracuse University, Violent Crime Control and Law Enforcement Act, Violence Against Women Act.",
      "tokensUsed": {
        "promptTokens": 148,
        "completionTokens": 27,
        "totalTokens": 175
      },
      "keywords": [
        "Joseph Robinette Biden Jr.",
        "46th president of the United States",
        "Democratic Party",
        "Vice President",
        "Barack Obama",
        "Delaware",
        "University of Delaware",
        "Syracuse University",
        "Violent Crime Control and Law Enforcement Act",
        "Violence Against Women Act"
      ]
    }
  ]
}

Named Entity Recognition use case

In the Named Entity Recognition ner use case, the LLM ingests text and entities to extract and return a JSON response that contains a list of entities extracted from the text. No options can be configured.

This use case can be used to extract nouns and proper nouns such as Brand, Date, Company, Places, and Category in order to guide and refine searches.

Unique values for the named entity recognition `ner` use case

The values available in this use case (that may not be available in other use cases) are:

Parameter Value

Parameter	Value
"useCaseConfig"	`"entityTypeMap": {"ENTITY": ["exampleA", "exampleB"], "ENTITY1": ["exampleC", "exampleD"]}` This parameter provides a map with entity type as a key with a list of example values to search. The entity type is required, but example values are optional and can be empty. Multiple entities with examples can be entered in the request. In the LWAI Prediction index stage and the LWAI Prediction query stage, the `useCaseConfig entityTypeMap` parameter only supports a string. Therefore, the string entered in Fusion is converted to a JSON string, which is required in the Lucidworks AI `entityTypeMap` variable. For example: `"entityTypeMap": "{\"ENTITY\": [\"exampleA\", \"exampleB\"], \"ENTITY1\": [\"exampleC\", \"exampleD\"]}`

"useCaseConfig"

"entityTypeMap": {"ENTITY": ["exampleA", "exampleB"], "ENTITY1": ["exampleC", "exampleD"]}

This parameter provides a map with entity type as a key with a list of example values to search. The entity type is required, but example values are optional and can be empty. Multiple entities with examples can be entered in the request.

In the LWAI Prediction index stage and the LWAI Prediction query stage, the useCaseConfig entityTypeMap parameter only supports a string. Therefore, the string entered in Fusion is converted to a JSON string, which is required in the Lucidworks AI entityTypeMap variable. For example:

"entityTypeMap": "{\"ENTITY\": [\"exampleA\", \"exampleB\"], \"ENTITY1\": [\"exampleC\", \"exampleD\"]}

Example POST request

This example does not include modelConfig parameters, but you can submit requests that include parameters described in Common POST request parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/ner/MODEL_ID \
 --header 'Authorization: Bearer AUTH_TOKEN' \
  --data '{
	"batch": [
		{
			"text": "Mahatma Gandhi, born on October 2, 1869, in Porbandar, India, led a life that profoundly shaped the course of history. Inspired by his principles of non-violence, truth, and civil disobedience, Gandhi became a pivotal figure in India'\''s struggle for independence from British rule. His journey began as a lawyer in South Africa, where he experienced racial discrimination and injustice, sparking his commitment to social justice. Returning to India, he became the face of the nonviolent resistance movement, employing methods like peaceful protests, fasting, and marches. The iconic Salt March of 1930 exemplified his philosophy as thousands followed him in the defiance of salt taxes imposed by the British. Gandhi'\''s ascetic lifestyle, clad in simple attire and practicing self-sufficiency, endeared him to the masses. Despite facing imprisonment multiple times, he remained steadfast in his pursuit of India'\''s freedom. Tragically, Gandhi was assassinated on January 30, 1948, but his legacy endures globally as a symbol of peace, tolerance, and the transformative power of nonviolent resistance"
		}
	],
	"useCaseConfig": {
		"entityTypeMap":{
					"Location":["India", "South Africa"]
		}
	}
}'

The following is an example of a successful response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "SUBMITTED"
}

The following is an example of an error response:

{
	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
	"status": "ERROR",
	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
}

Example GET request

curl --request GET
--url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
--header 'Authorization: Bearer Auth '

The following is an example response:

{
  "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  "status": "READY",
	"predictions": [
		{
			"tokensUsed": {
				"promptTokens": 387,
				"completionTokens": 23,
				"totalTokens": 410
			},
			"entities": {
				"Location": [
					"Porbandar",
					"India",
					"South Africa"
				]
			},
			"response": "{\n\"Location\": [\n\"Porbandar\",\n\"India\"\n]\n}"
		}
	]
}