Skip to main content
The Pass-through use case of the LWAI Prediction API lets you use the service as a proxy to the LLM. The service sends text (no additional prompts or other information) to the LLM and returns a response.
For detailed API specifications in Swagger/OpenAPI format, see Platform APIs.

​
Prerequisites

To use this API, you need:
  • The unique APPLICATION_ID for your Lucidworks AI application. For more information, see credentials to use APIs.
  • A bearer token generated with a scope value of machinelearning.predict. For more information, see Authentication API.
  • The USE_CASE and MODEL_ID fields for the use case request. The path is: /ai/prediction/USE_CASE/MODEL_ID. A list of supported models is returned in the Lucidworks AI Use Case API. For more information about supported models, see Generative AI models.

​
Common parameters and fields

Some parameters in the /ai/async-prediction/USE_CASE/MODEL_ID request are common to all of the generative AI (Gen-AI) use cases, such as the modelConfig parameter. Also referred to as hyperparameters, these fields set certain controls on the response. Refer to the API spec for more information.

​
Unique values for the pass-through use case

Some parameter values available in the pass-through use case are unique to this use case, including values for the useCaseConfig parameter. Refer to the API spec for more information.

​
Use System Prompt

"useCaseConfig": "useSystemPrompt": boolean This parameter can be used:
  • If custom prompts are needed, or if the prompt response format needs to be manipulated.
  • But the prompt length may increase response time.
    Some models, such as the mistral-7b-instruct and llama-3-8b-instruct, generate more effective results when system prompts are included in the request.
    If "useSystemPrompt": true, the LLM input is automatically wrapped into a model-specific prompt format with a generic system prompt before passing it to the model or third-party API.
    If "useSystemPrompt": false, the batch.text value serves as the prompt for the model. The LLM input must accommodate model-specific requirements because the input is passed as is.
Examples: This useSystemPrompt example does not include modelConfig parameters, but you can submit requests that include parameters described in Common parameters and fields. 
curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/prediction/passthrough/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
  "batch": [
    {
      "text": "who was the first president of the USA?"
      }
    ],
  "useCaseConfig": {
    "useSystemPrompt": true
    }
  }'

​
Data Type

"useCaseConfig": "dataType": "string" This optional parameter enables model-specific handling in the Prediction API to help improve model accuracy. Use the most applicable fields based on available dataTypes and the dataType value that best aligns with the text sent to the Prediction API. The values for dataType in the Passthrough use case are:
  • "dataType": "text" This value is equivalent to "useSystemPrompt": true and is a pre-defined, generic prompt.
  • "dataType": "raw_prompt" This value is equivalent to "useSystemPrompt": false and is passed directly to the model or third-party API.
  • "dataType": "json_prompt" This value follows the generics that allow three roles:
    • system
    • user
      • Only the last user message is truncated.
      • If the API does not support system prompts, the user role is substituted for the system role.
    • assistant
      • If the last message role is assistant, it is used as a pre-fill for generation and is the first generated token the model uses. The pre-fill is prepended to the model output, which makes models less verbose and helps enforce specific outputs such as YAML.
      • The Google Vertex AI does not support generation pre-fills, so an exception error is generated.
      This follows the HuggingFace template constraints at Hugging Face chat templates.
      • Additional json_prompt information:
        • Consecutive messages for the same role are merged.
        • You can paste the information for a hosted model into the json_prompt value and change the model name in the stage.
This "dataType": "json_prompt"`` example does not include modelConfig` parameters, but you can submit requests that include parameters described in Common parameters and fields. 
curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/prediction/passthrough/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
   "batch": [
       {
         "text": "[{\"role\": \"system\", \"content\": \"You are a helpful utility program instructed to accomplish a word correction task. Provide the most likely suggestion to the user without a preamble or elaboration.\"}, {\"role\": \"user\", \"content\": \"misspeled\"}, {\"role\": \"assistant\", \"content\": \"CORRECT:\"}]"
       }
   ],
   "useCaseConfig" :{
     "dataType" : "json_prompt"
   }
}'

​
Verify information sent to Gen-AI models

The Lucidworks AI Generative AI Prompting Preview API returns Prediction API passthrough use case prompts before being sent to any generative AI (Gen-AI) model. You can use this information to help debug and ensure input to the generative AI (Gen-AI) model is valid, and within the model’s processing limits. For more information, see Prompting Preview API.
⌘I