> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Pass-through use case

> Lucidworks AI Async Prediction API

export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => {
  const [isLoaded, setIsLoaded] = useState(false);
  useEffect(() => {
    const timer = setTimeout(() => {
      setIsLoaded(true);
    }, 500);
    return () => clearTimeout(timer);
  }, []);
  return <div className="lw-template-container">
      <Card title={title} icon={icon}>
        {isLoaded && <span dangerouslySetInnerHTML={{
    __html: `<lw-template id="a029c1a9-28be-427e-b0e1-5d918920246a"></lw-template
            >`
  }} />}
        <Link href={linkHref} className="agent-studio-link text-left text-gray-600 gap-2 dark:text-gray-400 text-sm font-medium flex flex-row items-center hover:text-primary dark:hover:text-primary-light group-hover:text-primary group-hover:dark:text-primary-light">Powered by Lucidworks Agent Studio</Link>
      </Card>
    </div>;
};

[localhost link]: http://localhost:3000/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/passthrough-async

[mintlify link]: https://doc.lucidworks.com/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/passthrough-async

[old doc.lw link]: https://doc.lucidworks.com/lw-platform/ai/xvnhrc

The Pass-through use case of the [Lucidworks AI Async Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/overview) lets you use the service as a proxy to the LLM. The service sends text (no additional prompts or other information) to the LLM and returns a response.

The Pass-through use case contains two requests:

* POST request - submits a prediction task for a specific `useCase` and `modelId`. The API responds with the following information:

  * `predictionId`. A unique UUID for the submitted prediction task that can be used later to retrieve the results.
  * `status`. The current state of the prediction task.
* GET request - uses the `predictionId` you submit from a previously-submitted POST request and returns the results associated with that previous request.

<Note>
  For detailed API specifications in Swagger/OpenAPI format, see [Platform APIs](/api-reference/get-predictions/passthrough-use-case).
</Note>

<LwTemplate />

## Prerequisites

To use this API, you need:

* The unique `APPLICATION_ID` for your Lucidworks AI application, which is provided by Lucidworks.
* A bearer token generated with a scope value of `machinelearning.predict`. For more information, see [Authentication API](/docs/lw-platform/lw-platform/authentication-api).
* The `USE_CASE` and `MODEL_ID` fields in the `/async-prediction` for the POST request. The path is `/ai/async-prediction/USE_CASE/MODEL_ID`. A list of supported modes is returned in the [Lucidworks AI Use Case API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-use-case-api). For more information about supported models, see [Generative AI models](/docs/lw-platform/lw-ai/lw-ai-generative-ai#generative-ai-models).

## Common POST request parameters and fields

Some parameters in the `/ai/async-prediction/USE_CASE/MODEL_ID` POST request are common to all of the generative AI (Gen-AI) use cases, such as the `modelConfig` parameter.
Also referred to as hyperparameters, these fields set certain controls on the response.
Refer to the [API spec](/api-reference/get-predictions/passthrough-use-case) for more information.

## Unique values for the pass-through use case

Some parameter values available in the `pass-through` use case are unique to this use case, including values for the `useCaseConfig` parameter.
Refer to the [API spec](/api-reference/get-predictions/passthrough-use-case) for more information.

### Use System Prompt

<ResponseField name="useCaseConfig.useSystemPrompt" type="boolean">
  This parameter controls whether the LLM input is automatically wrapped with a system prompt or passed directly.

  <Expandable title="true">
    The LLM input is automatically wrapped into a model-specific prompt format with a generic system prompt before passing it to the model or third-party API.

    Some models, such as `llama-3-8b-instruct`, generate more effective results when system prompts are included in the request.
  </Expandable>

  <Expandable title="false">
    The `batch.text` value serves as the prompt for the model. The LLM input must accommodate model-specific requirements because the input is passed as is.
  </Expandable>

  Use this parameter if custom prompts are needed or if the prompt response format needs to be manipulated. Be aware that including a system prompt may increase response time.
</ResponseField>

**Examples:**

* The format for the `llama-3-8b-instruct` model must be specific to Llama:\
  [https://huggingface.co/blog/llama3#how-to-prompt-llama-3](https://huggingface.co/blog/llama3#how-to-prompt-llama-3)
* The text input for OpenAI models must be valid JSON to match the OpenAI API specification:\
  [https://platform.openai.com/docs/api-reference/chat/create](https://platform.openai.com/docs/api-reference/chat/create)
* The format for the Google Vertex AI models must adhere to the guidelines at:\
  [https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini)

**POST example**

This `useSystemPrompt` POST example does not include `modelConfig` parameters, but you can submit requests that include parameters described in [Common POST request parameters and fields](#common-post-request-parameters-and-fields).

<CodeGroup>
  ```bash wrap Request theme={"dark"}
  curl --request POST \
    --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/passthrough/MODEL_ID \
    --header 'Authorization: Bearer ACCESS_TOKEN' \
    --header 'Content-type: application/json' \
    --data '{
    "batch": [
      {
        "text": "who was the first president of the USA?"
        }
      ],
    "useCaseConfig": {
      "useSystemPrompt": true
      }
    }'
  ```

  ```json wrap Success theme={"dark"}
  {
  	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  	"status": "SUBMITTED"
  }
  ```

  ```json wrap Error theme={"dark"}
  {
  	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  	"status": "ERROR",
  	"message": "System prompt exceeded the maximum number of allowed input tokens: 81 vs -1091798"
  }
  ```
</CodeGroup>

**GET example**

<CodeGroup>
  ```bash wrap Request theme={"dark"}
  curl --request GET
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/PREDICTION_ID
  --header 'Authorization: Bearer Auth '
  ```

  The following is an example response:

  ```json wrap Response theme={"dark"}
  {
    "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
    "status": "READY",
    "predictions": [
    {
      "response": "The first President of the United States was George Washington.",
      "tokensUsed": {
        "promptTokens": 49,
        "completionTokens": 11,
        "totalTokens": 60
        }
      }
    ]
  }
  ```
</CodeGroup>

### Data Type

<ResponseField name="useCaseConfig.dataType" type="string">
  This optional parameter enables model-specific handling in the Async Prediction API to help improve model accuracy. Use the most applicable value based on available data and the data type that best aligns with the text sent to the API.

  <Expandable title="text">
    Equivalent to `"useSystemPrompt": true` and is a predefined, generic prompt.
  </Expandable>

  <Expandable title="raw_prompt">
    Equivalent to `"useSystemPrompt": false` and is passed directly to the model or third-party API.
  </Expandable>

  <Expandable title="json_prompt">
    This value follows the generics that allow three roles.

    <ResponseField name="system" type="role">
      The system message role.
    </ResponseField>

    <ResponseField name="user" type="role">
      Only the last user message is truncated.\
      If the API does not support system prompts, the user role is substituted for the system role.
    </ResponseField>

    <ResponseField name="assistant" type="role">
      If the last message role is `assistant`, it is used as a pre-fill for generation and is the first generated token the model uses.\
      The pre-fill is prepended to the model output, which makes models less verbose and helps enforce specific outputs such as YAML.\
      The Google Vertex AI does not support generation pre-fills, so an exception error is generated.
    </ResponseField>

    This follows the Hugging Face template constraints at [Hugging Face chat templates](https://huggingface.co/docs/transformers/main/en/chat_templating).\
    Additional `json_prompt` information:

    * Consecutive messages for the same role are merged
    * You can paste the information for a hosted model into the `json_prompt` value and change the model name in the stage
  </Expandable>
</ResponseField>

**POST example**

This `"dataType": "json_prompt"` example does not include `modelConfig` parameters, but you can submit requests that include parameters described in [Common parameters and fields](#common-post-request-parameters-and-fields).

<CodeGroup>
  ```json wrap Request theme={"dark"}
  curl --request POST \
    --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/passthrough/MODEL_ID \
    --header 'Authorization: Bearer ACCESS_TOKEN' \
    --header 'Content-type: application/json' \
    --data '{
     "batch": [
       {
         "text": "[{\"role\": \"system\", \"content\": \"You are a helpful utility program instructed to accomplish a word correction task. Provide the most likely suggestion to the user without a preamble or elaboration.\"}, {\"role\": \"user\", \"content\": \"misspeled\"}, {\"role\": \"assistant\", \"content\": \"CORRECT:\"}]"
         }
     ],
     "useCaseConfig" :{
       "dataType" : "json_prompt"
     }
  }'
  ```

  ```json wrap Response theme={"dark"}
  {
  	"predictionId": "fd110486-f168-47c0-a419-1518a4840589",
  	"status": "SUBMITTED"
  }
  ```
</CodeGroup>

**GET example**

<CodeGroup>
  ```bash wrap Request theme={"dark"}
  curl --request GET \
    --url https://APPLICATION_ID.applications.lucidworks.com/ai/async-prediction/fd110486-f168-47c0-a419-1518a4840589 \
    --header 'Authorization: Bearer ACCESS_TOKEN'
  ```

  ```json wrap Response theme={"dark"}
  {
      "predictionId": "fd110486-f168-47c0-a419-1518a4840589",
      "status": "READY",
      "predictions": [
          {
              "tokensUsed": {
                  "promptTokens": 51,
                  "completionTokens": 4,
                  "totalTokens": 55
              },
              "response": "CORRECT: misspelled"
          }
      ]
  }
  ```
</CodeGroup>