> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# RAG use case

> The rag use case uses candidate documents that are inserted into a LLM’s context to ground the generated response to those documents instead of generating an answer from details stored in the LLM’s trained weights. This type of search adds guardrails so the LLM can search private data collections.

The RAG search can perform queries against external documents passed in as part of the request.

The POST request obtains and indexes prediction information related to the specified use case, and returns a unique `predictionId` and `status` of the request. The `predictionId` can be used later in the GET request to retrieve the results.




## OpenAPI

````yaml /api-reference/saas/machine-learning-platform-async-prediction.json post /ai/async-prediction/rag/{MODEL_ID}
openapi: 3.0.1
info:
  title: Lucidworks AI Async Prediction API
  version: v0
  description: >-
    The Lucidworks AI Async Prediction API contains two requests:


    * POST Request: This request is used to submit a prediction task for a
    specific `useCase` and `modelId`. Upon submission, the API responds with the
    following information:

        * `predictionId` that is a unique UUID for the submitted prediction task, and can be used later to retrieve the results. 

        * `status` that indicates the current state of the prediction task. 
     
    * GET Request: This request is used to retrieve the results of a
    previously-submitted prediction request. You must provide the unique
    `predictionId` received from the POST request. The API then returns the
    results of the prediction request associated with that `predictionId`.


    The Use Case API returns a list of all supported models.


    The `async-prediction` endpoints require an authentication token with scope
    `machinelearning.predict`.
  contact:
    name: Lucidworks
    url: https://lucidworks.com/
    email: support@lucidworks.com
  termsOfService: https://lucidworks.com/legal/developer-license-agreement/
  license:
    name: Lucidworks
    url: https://lucidworks.com/legal/developer-license-agreement/
servers:
  - url: https://APPLICATION_ID.applications.lucidworks.com
    description: Production
security: []
tags:
  - name: Create predictions
    description: Submit prediction tasks to Lucidworks AI.
  - name: Fetch predictions
    description: Get the results of a prediction task from Lucidworks AI.
paths:
  /ai/async-prediction/rag/{MODEL_ID}:
    parameters:
      - schema:
          type: string
        name: MODEL_ID
        in: path
        required: true
        description: Unique identifier for the model.
        example: llama-3-8b-instruct
    post:
      tags:
        - Create predictions
      summary: RAG use case
      description: >
        The rag use case uses candidate documents that are inserted into a LLM’s
        context to ground the generated response to those documents instead of
        generating an answer from details stored in the LLM’s trained weights.
        This type of search adds guardrails so the LLM can search private data
        collections.


        The RAG search can perform queries against external documents passed in
        as part of the request.


        The POST request obtains and indexes prediction information related to
        the specified use case, and returns a unique `predictionId` and `status`
        of the request. The `predictionId` can be used later in the GET request
        to retrieve the results.
      operationId: post-ai-prediction-rag-modelId-external-documents
      parameters:
        - in: header
          name: Authorization
          schema:
            type: string
          required: true
          description: >-
            Bearer token used for authentication. Format: `Authorization: Bearer
            ACCESS_TOKEN`.
          example: Bearer abc123def456
        - schema:
            type: string
            example: application/json
          in: header
          name: Content-Type
          description: application/json
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/RagExtDocRequest'
            example:
              batch:
                - text: Why did I go to Germany?
                  documents:
                    - body: I'm off to Germany to go to the Oktoberfest!
                      source: http://example.com/112
                      title: Off to Germany!
                      date: '2022-01-31T19:31:34Z'
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/POSTresponse'
              example:
                predictionId: 266fd643-794b-4491-a0b2-cca6b2acbb7a
                status: SUBMITTED
        4XX:
          $ref: '#/components/responses/Error'
components:
  schemas:
    RagExtDocRequest:
      title: RAG Request
      type: object
      properties:
        batch:
          $ref: '#/components/schemas/BatchRag'
        useCaseConfig:
          $ref: '#/components/schemas/UseCaseConfigRagExtDoc'
        modelConfig:
          $ref: '#/components/schemas/ModelConfig'
    POSTresponse:
      title: POST response
      type: object
      description: >-
        This is the response to the POST prediction request submitted for a
        specific `useCase` and `modelId`. 
      properties:
        predictionId:
          type: string
          format: uuid
          description: >-
            The universal unique identifier (UUID) returned in the POST request.
            This UUID is required in the GET request to retrieve results.
        status:
          type: string
          example: SUBMITTED
          description: >-
            The current status of the prediction. Allowed values are:


            * SUBMITTED - The POST request was successful and the response has
            returned the `predictionId` and `status` that is used by the GET
            request.


            * ERROR - An error was generated when the GET request was sent.


            * READY - The results associated with the `predictionId` are
            available and ready to be retrieved.


            * RETRIEVED - The results associated with the `predictionId` are
            returned successfully when the GET request was sent.
    BatchRag:
      title: BatchRag
      type: array
      description: ''
      x-examples: {}
      items:
        type: object
        properties:
          text:
            type: string
            description: >-
              Content for the model to analyze. Multiple instances of text can
              be sent in the request.
            example: What is RAG?
          documents:
            type: array
            items:
              $ref: '#/components/schemas/Document'
    UseCaseConfigRagExtDoc:
      title: UseCaseConfigRagChat
      type: object
      properties:
        memoryUuid:
          type: string
          description: >-
            The universal unique identifier (UUID) stored in the trained set of
            data in the model that is used in the model request.


            This parameter is optional, and is used when previous chat history
            reference information is available.
          example: 27a887fe-3d7c-4ef0-9597-e2dfc054c20e
        answerNotFoundMessage:
          type: string
          description: >-
            This parameter is optional, and can be passed to change the response
            if the LLM cannot answer the request. The default is "Not possible
            to answer given this content."
          default: Not possible to answer given this content.
    ModelConfig:
      title: ModelConfig
      type: object
      description: >-
        Provides fields and values that specify ranges for tokens. Fields used
        for specific use cases and models are specified. The default values are
        used if other values are not specified.
      properties:
        temperature:
          type: number
          format: float
          example: 0.8
          minimum: 0
          maximum: 2
          description: >-
            A sampling temperature between 0 and 2. A higher sampling
            temperature such as 0.8, results in more random (creative) output. A
            lower value such as 0.2 results in more focused (conservative)
            output. A lower value does not guarantee the model returns the same
            response for the same input. We recommend staying at or below a
            temperature of 1.0. Values above 1.0 might return nonsense unless
            the topP value is lowered to be more deterministic.
        topP:
          type: number
          format: float
          example: 1
          minimum: 1
          maximum: 1
          description: >-
            A floating-point number that controls the cumulative probability of
            the top tokens to consider. Required range is [0, 1]. Set topP to 1
            to consider all tokens.
        topK:
          type: integer
          example: -1
          description: >-
            An integer that controls the number of top tokens to consider. Set
            topK to -1 to consider all tokens.
        presencePenalty:
          type: number
          format: float
          minimum: -2
          maximum: 2
          description: >-
            A floating-point number that penalizes new tokens based on whether
            they have already appeared in the text. Required range is [-2, 2]. A
            value greater than zero (0) encourages the model to use new tokens.
            A value less than zero (0) encourages the model to repeat existing
            tokens. This is applicable for all OpenAI and Llama models.
          example: 2
        frequencyPenalty:
          type: number
          format: float
          minimum: -2
          maximum: 2
          example: 1
          description: >-
            A floating-point number that penalizes new tokens based on their
            frequency in the generated text. Required range is [-2, 2]. A value
            greater than zero (0) encourages the model to use new tokens. A
            value less than zero (0) encourages the model to repeat existing
            tokens. This is applicable for all OpenAI and Llama models.
        maxTokens:
          type: integer
          format: int32
          example: 1
          description: The maximum number of tokens to generate per output sequence.
        apiKey:
          type: string
          description: >-
            This optional parameter is only required when using the model for
            prediction. You can find this value in your model's settings:


            * **OpenAI**: Copy and paste the API key found in your
            organization's settings. For more information, see <a
            href="https://platform.openai.com/docs/api-reference/authentication">OpenAI
            Authentication API keys</a>.


            * **Azure OpenAI**: Copy and paste the API key found in your Azure
            portal. See <a
            href="https://learn.microsoft.com/en-us/azure/api-management/api-management-authenticate-authorize-azure-openai#authenticate-with-api-key">Authenticate
            with API key</a>.


            * **Anthropic**: Copy and paste the API key found in your <a
            href="https://console.anthropic.com/settings/keys">Anthropic
            console</a> or by using the <a
            href="https://docs.anthropic.com/en/api/admin-api/apikeys/get-api-key">Anthropic
            API</a>.


            * **Google Vertex AI**: Copy and paste the base64-encoded service
            account key JSON found in your <a
            href="https://cloud.google.com/iam/docs/keys-list-get#list-keys">Google
            Cloud console</a>. This service account key must have the <a
            href="https://cloud.google.com/iam/docs/understanding-roles#aiplatform.user">Vertex
            AI user</a> role enabled. For more information, see <a
            href="https://cloud.google.com/iam/docs/keys-create-delete#creating">generate
            service account key</a>.
          example: API key specific to use case and model
        azureDeployment:
          type: string
          description: >-
            This optional parameter is the name of the deployed Azure OpenAI
            model and is only required when a deployed Azure OpenAI model is
            used for prediction.
          example: DEPLOYMENT_NAME
        azureEndpoint:
          type: string
          description: >-

            This optional parameter is the URL endpoint of the deployed Azure
            OpenAI model and is only required when a deployed Azure OpenAI model
            is used for prediction.
          example: https://azure.endpoint.com
        googleProjectId:
          type: string
          example: '[GOOGLE_PROJECT_ID]'
          description: >-
            This parameter is optional, and is only required when a Google
            Vertex AI model is used for prediction.  
        googleRegion:
          type: string
          description: >-
            This parameter is optional, and is only required when a Google
            Vertex AI model is used for prediction. A value of `global` routes
            the query to any available region. Other possible region values are:


            * us-central1

            * us-west4

            * northamerica-northeast1

            * us-east4

            * us-west1

            * asia-northeast3

            * asia-southeast1

            * asia-northeast
          example: '[GOOGLE_PROJECT_REGION_OF_MODEL_ACCESS]'
    predictionId:
      title: predictionId
      type: string
      description: >-
        The universal unique identifier (UUID) returned in the POST request.
        This UUID is required in the GET request to retrieve results.
      format: uuid
      example: fd110486-f168-47c0-a419-1518a4840589
    status:
      title: status
      type: string
      description: >-
        The current status of the prediction. Allowed values are:


        * `SUBMITTED` - The POST request was successful and the response has
        returned the `predictionId` and `status` that is used by the GET
        request.


        * `READY` - The results associated with the `predictionId` are available
        and ready to be retrieved.


        * `ERROR` - An error was generated when the GET request was sent.


        * `RETRIEVED` - The results associated with the `predictionId` are
        returned successfully when the GET request was sent.
      example: READY
    Document:
      title: Document
      type: object
      description: >-
        This array is passed in the `batch` object. Allowed LLM context length
        limits the number of documents to 3. The parameter can be used in the 
        <a
        href="https://doc.lucidworks.com/fusion/5.9/hnuyky/lwai-prediction">Fusion
        LWAI Prediction query stage</a> by selecting the **Include response
        documents** check box.
      properties:
        body:
          type: string
          description: The contents of the document.
          example: >-
            Retrieval Augmented Generation, known as RAG, a framework promising
            to optimize generative AI.
        source:
          type: string
          description: The URL that identifies the source of the document.
          example: http://rag.com/22
        title:
          type: string
          description: The title of the document.
          example: What are the benefits of RAG?
        date:
          type: string
          format: date-time
          example: '2022-01-31T19:31:34Z'
          description: >-
            The date and time the document was created, displayed in the
            required ISO-8601 format of `yyyy-mm-ddThh:mm:ssZ`.
  responses:
    Error:
      description: >-
        The error varies based on the issue encountered regarding the
        `predictionId` or related information.
      content:
        application/json:
          schema:
            type: object
            properties:
              predictionId:
                $ref: '#/components/schemas/predictionId'
              status:
                $ref: '#/components/schemas/status'
              message:
                type: string
                description: >-
                  The error generated if the `predictionId` cannot be located or
                  the related information cannot be retrieved. For example,
                  "System prompt exceeded the maximum number of allowed input
                  tokens."

````