Named entity recognition (NER) use caseLucidworks AI Prediction API

Table of Contents

Prerequisites
Common parameters and fields
Uses of the responses
Fusion AI NER

The Named entity recognition (NER) use case of the LWAI Prediction API uses a large language model (LLM) to ingest input text and extract a list of named entities in a structured JSON response.

This use case:

Can be used to extract nouns and proper nouns such as Brand, Date, Company, Places, and Category in order to guide and refine searches.
Can be minimally configured with specific entities, but has a higher latency of a few seconds, making it less suitable for query-time processing.

Prerequisites

To use this API, you need:

The unique APPLICATION_ID for your Lucidworks AI application. For more information, see credentials to use APIs.
A bearer token generated with a scope value of machinelearning.predict. For more information, see Authentication API.
The USE_CASE and MODEL_ID fields for the use case request. The path is: /ai/prediction/USE_CASE/MODEL_ID. A list of supported models is returned in the Lucidworks AI Use Case API. For more information about supported models, see Generative AI models.

Some parameters of the /ai/prediction/USE_CASE/MODEL_ID request are common to all of the generative AI (GenAI) use cases, including the modelConfig parameter. If you do not enter values, the following defaults are used.

"modelConfig":{
  "temperature": 0.7,
  "topP": 1.0,
  "topK": -1.0,
  "maxTokens": 256
}

Also referred to as hyperparameters, these fields set certain controls on the response of a LLM:

Field Description

Field	Description
temperature	A sampling temperature between 0 and 2. A higher sampling temperature such as 0.8, results in more random (creative) output. A lower value such as 0.2 results in more focused (conservative) output. A lower value does not guarantee the model returns the same response for the same input.
topP	A floating-point number between 0 and 1 that controls the cumulative probability of the top tokens to consider, known as the randomness of the LLM’s response. This parameter is also referred to as top probability. Set `topP` to 1 to consider all tokens. A higher value specifies a higher probability threshold and selects tokens whose cumulative probability is greater than the threshold. The higher the value, the more diverse the output.
topK	An integer that controls the number of top tokens to consider. Set `top_k` to -1 to consider all tokens.
presencePenalty	A floating-point number between -2.0 and 2.0 that penalizes new tokens based on whether they have already appeared in the text. This increases the model’s use of diverse tokens. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens. This is applicable for all OpenAI, Mistral, and Llama models.
frequencyPenalty	A floating-point number between -2.0 and 2.0 that penalizes new tokens based on their frequency in the generated text. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens. This is applicable for all OpenAI, Mistral, and Llama models.
maxTokens	The maximum number of tokens to generate per output sequence. The value is different for each model. Review individual model specifications when the value exceeds 2048.
apiKey	The optional parameter is only required when the specified model is used for prediction. This secret value is specified in the external model. For: OpenAI models, `"apiKey"` is the value in the model’s `"[OPENAI_API_KEY]"` field. For more information, see Authentication API keys. Azure OpenAI models, `"apiKey"` is the value generated by Azure in either the model’s `"[KEY1 or KEY2]"` field. For requirements to use Azure models, see Generative AI models. Anthropic models, `"apiKey"` is the value in the model’s `"[ANTHROPIC_API_KEY]"` field. For more information, see Authentication API keys. Google VertexAI models, `"apiKey"` is the value in the model’s `"[BASE64_ENCODED_GOOGLE_SERVICE_ACCOUNT_KEY]"` field. For more information, see Create and delete Google service account keys. The parameter (for OpenAI, Azure OpenAI, Anthropic, or Google VertexAI models) is only available for the following use cases: Pass-through RAG Standalone query rewriter Summarization Keyword extraction NER
azureDeployment	The optional `"azureDeployment": "[DEPLOYMENT_NAME]"` parameter is the deployment name of the Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.
azureEndpoint	The optional `"azureEndpoint": "[ENDPOINT]"` parameter is the URL endpoint of the deployed Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.
googleProjectId	The optional `"googleProjectId": "[GOOGLE_PROJECT_ID]"` parameter is only required when a Google VertexAI model is used for prediction.
googleRegion	The optional `"googleRegion": "[GOOGLE_PROJECT_REGION_OF_MODEL_ACCESS]"` parameter is only required when a Google VertexAI model is used for prediction. The possible region values are: us-central1 us-west4 northamerica-northeast1 us-east4 us-west1 asia-northeast3 asia-southeast1 asia-northeast

temperature

A sampling temperature between 0 and 2. A higher sampling temperature such as 0.8, results in more random (creative) output. A lower value such as 0.2 results in more focused (conservative) output. A lower value does not guarantee the model returns the same response for the same input.

topP

A floating-point number between 0 and 1 that controls the cumulative probability of the top tokens to consider, known as the randomness of the LLM’s response. This parameter is also referred to as top probability. Set topP to 1 to consider all tokens. A higher value specifies a higher probability threshold and selects tokens whose cumulative probability is greater than the threshold. The higher the value, the more diverse the output.

topK

An integer that controls the number of top tokens to consider. Set top_k to -1 to consider all tokens.

presencePenalty

A floating-point number between -2.0 and 2.0 that penalizes new tokens based on whether they have already appeared in the text. This increases the model’s use of diverse tokens. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens. This is applicable for all OpenAI, Mistral, and Llama models.

frequencyPenalty

A floating-point number between -2.0 and 2.0 that penalizes new tokens based on their frequency in the generated text. A value greater than zero (0) encourages the model to use new tokens. A value less than zero (0) encourages the model to repeat existing tokens. This is applicable for all OpenAI, Mistral, and Llama models.

maxTokens

The maximum number of tokens to generate per output sequence. The value is different for each model. Review individual model specifications when the value exceeds 2048.

apiKey

The optional parameter is only required when the specified model is used for prediction. This secret value is specified in the external model. For:

OpenAI models, "apiKey" is the value in the model’s "[OPENAI_API_KEY]" field. For more information, see Authentication API keys.
Azure OpenAI models, "apiKey" is the value generated by Azure in either the model’s "[KEY1 or KEY2]" field. For requirements to use Azure models, see Generative AI models.
Anthropic models, "apiKey" is the value in the model’s "[ANTHROPIC_API_KEY]" field. For more information, see Authentication API keys.
Google VertexAI models, "apiKey" is the value in the model’s

"[BASE64_ENCODED_GOOGLE_SERVICE_ACCOUNT_KEY]" field. For more information, see Create and delete Google service account keys.

The parameter (for OpenAI, Azure OpenAI, Anthropic, or Google VertexAI models) is only available for the following use cases:

Pass-through
RAG
Standalone query rewriter
Summarization
Keyword extraction
NER

azureDeployment

The optional "azureDeployment": "[DEPLOYMENT_NAME]" parameter is the deployment name of the Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.

azureEndpoint

The optional "azureEndpoint": "[ENDPOINT]" parameter is the URL endpoint of the deployed Azure OpenAI model and is only required when a deployed Azure OpenAI model is used for prediction.

googleProjectId

The optional "googleProjectId": "[GOOGLE_PROJECT_ID]" parameter is only required when a Google VertexAI model is used for prediction.

googleRegion

The optional "googleRegion": "[GOOGLE_PROJECT_REGION_OF_MODEL_ACCESS]" parameter is only required when a Google VertexAI model is used for prediction. The possible region values are:

us-central1
us-west4
northamerica-northeast1
us-east4
us-west1
asia-northeast3
asia-southeast1
asia-northeast

Unique values for the named entity recognition `ner` use case

The GenAI NER use case does not provide direct mappings, nor does it have a mechanism to recognize the full catalog entities. Therefore, the entities passed in the request may not be in the catalog.

To create NER mappings for entities, configure the LWAI Prediction indexing stage "useCaseConfig": "entityTypeMap" parameter. Once mapped, the entities are returned and the query-side pipeline can leverage those mappings.

The values available in this use case (that may not be available in other use cases) are:

Parameter Value

Parameter	Value
"useCaseConfig"	`"entityTypeMap": {"ENTITY": ["exampleA", "exampleB"], "ENTITY1": ["exampleC", "exampleD"]}` This parameter provides a map with entity type as a key with a list of example values to search. The entity type is required, but example values are optional and can be empty. Multiple entities with examples can be entered in the request. In the LWAI Prediction index stage and the LWAI Prediction query stage, the `useCaseConfig entityTypeMap` parameter only supports a string. Therefore, the string entered in Fusion is converted to a JSON string, which is required in the Lucidworks AI `entityTypeMap` variable. For example: `"entityTypeMap": "{\"ENTITY\": [\"exampleA\", \"exampleB\"], \"ENTITY1\": [\"exampleC\", \"exampleD\"]}`

"useCaseConfig"

"entityTypeMap": {"ENTITY": ["exampleA", "exampleB"], "ENTITY1": ["exampleC", "exampleD"]}

This parameter provides a map with entity type as a key with a list of example values to search. The entity type is required, but example values are optional and can be empty. Multiple entities with examples can be entered in the request.

In the LWAI Prediction index stage and the LWAI Prediction query stage, the useCaseConfig entityTypeMap parameter only supports a string. Therefore, the string entered in Fusion is converted to a JSON string, which is required in the Lucidworks AI entityTypeMap variable. For example:

"entityTypeMap": "{\"ENTITY\": [\"exampleA\", \"exampleB\"], \"ENTITY1\": [\"exampleC\", \"exampleD\"]}

Example request

This example does not include modelConfig parameters, but you can submit requests that include parameters described in Common parameters and fields.

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/prediction/ner/MODEL_ID \
 --header 'Authorization: Bearer AUTH_TOKEN' \
  --data '{
	"batch": [
		{
			"text": "Mahatma Gandhi, born on October 2, 1869, in Porbandar, India, led a life that profoundly shaped the course of history. Inspired by his principles of non-violence, truth, and civil disobedience, Gandhi became a pivotal figure in India'\''s struggle for independence from British rule. His journey began as a lawyer in South Africa, where he experienced racial discrimination and injustice, sparking his commitment to social justice. Returning to India, he became the face of the nonviolent resistance movement, employing methods like peaceful protests, fasting, and marches. The iconic Salt March of 1930 exemplified his philosophy as thousands followed him in the defiance of salt taxes imposed by the British. Gandhi'\''s ascetic lifestyle, clad in simple attire and practicing self-sufficiency, endeared him to the masses. Despite facing imprisonment multiple times, he remained steadfast in his pursuit of India'\''s freedom. Tragically, Gandhi was assassinated on January 30, 1948, but his legacy endures globally as a symbol of peace, tolerance, and the transformative power of nonviolent resistance"
		}
	],
	"useCaseConfig": {
		"entityTypeMap":{
					"Location":["India", "South Africa"]
		}
	}
}'

The following is an example response:

{
	"predictions": [
		{
			"tokensUsed": {
				"promptTokens": 387,
				"completionTokens": 23,
				"totalTokens": 410
			},
			"entities": {
				"Location": [
					"Porbandar",
					"India",
					"South Africa"
				]
			},
			"response": "{\n\"Location\": [\n\"Porbandar\",\n\"India\"\n]\n}"
		}
	]
}

Uses of the responses

Overall, the GenAI NER is flexible in recognizing entities, but lacks built-in support for boosting or filtering. The actions described in this section can help determine whether the extracted entities should be used for boosting, filtering, or query expansion.

Entities extracted by the Fusion LWAI NER stage can be used in the query pipeline in several ways:

Boosting results: The extracted entities can be matched against catalog fields and used to apply relevance boosts to results containing those entities.
Query expansion: If an entity has multiple known variations, the query can be expanded to include all relevant forms.
Filtering results: The identified entities can be used to filter out irrelevant results based on category, brand, or other metadata. You must verify the entity is in your catalog or a zero result is returned.
Canonical mapping: If different terms map to the same entity, the query can be rewritten to standardize search terms. For example, if Hewlett-Packard, Inc. and HP are mapped to the same entity, you can rewrite the query to standardize the search terms.

If precision is the goal, strict filtering can be applied. For example, only return results that match identified entities in specific fields.

If recall is the goal, boosting rather than filtering would allow broader result sets while still prioritizing relevant matches.

Fusion AI NER

Fusion AI NER differs from GenAI NER. The Fusion AI NER approach returns entity results based on a pre-defined list and is more configurable than GenAI, is rule-based, and is optimized for query-time execution. In addition, Fusion AI NER allows:

Boosting results based on entity matches. For example, you can prioritize results that match brands or colors.
Entity recognition using predefined lists that can be used in queries to refine results. For example, a facet query can extract all unique brands and colors from the catalog.
Custom rule-based mapping of entity variations to ingest information where the entity may be referred to under multiple names. For example, Hewlett-Packard, Inc. and HP can be mapped to the same entity to return results from both identifiers.
Configurable boosting and query rewriting within Fusion’s pipeline.