Product Selector

Fusion 5.9
    Fusion 5.9

    Generative AILucidworks AI

    Generative AI service

    Lucidworks AI provides a generative AI (GenAI) service that lets you access and use large language models (LLM) for a variety of GenAI use cases.

    The GenAI models use one of Lucidworks' pre-defined use cases described in the following APIs:

    The APIs specify configuration for:

    • Use cases are set in the useCaseConfig parameter.

    • Models are set in the modelConfig parameter.

    For information about the stages that integrate Fusion with Lucidworks AI, see:

    Lucidworks AI is built with Meta Llama 3.

    Generative AI models

    Lucidworks-hosted GenAI models

    All of the models currently hosted by Lucidworks are open source.

    In models hosted by Lucidworks, the data is:

    • Contained within Lucidworks and is never exposed to third parties.

    • Passed to the specified model and generates responses to the instance’s requests, but is not retained to train or be used by the model after the initial request.

    For information about model training, biases, and safety features for those models, refer to the documentation provided by the model creators. Hugging Face is one information source, and links to information about Lucidworks-hosted models are provided in Available Lucidworks-hosted models.

    Available Lucidworks-hosted models

    To display the Huggingface information for each model, click the model name.

    OpenAI models

    An API key is required in each OpenAI model request. There is no default key.

    Available OpenAI models

    The supported OpenAI models are:

    • gpt-4

    • gpt-4o

    • gpt-4o-2024-05-13

    • gpt-4-0613

    • gpt-4-turbo

    • gpt-4-turbo-2024-04-09

    • gpt-4-turbo-preview

    • gpt-4-1106-preview

    • gpt-3.5-turbo

    • gpt-3.5-turbo-1106

    • gpt-3.5-turbo-0125

    Azure OpenAI models

    Deployed Azure OpenAI models are supported in the LWAI Prediction API and the Lucidworks AI Async Prediction API in the following use cases:

    • Pass-through

    • Retrieval Augmented Generation (RAG)

    • Standalone query rewriter

    • Summarization

    • Keyword extraction

    • Named Entity Recognition (NER)

    Prerequisites

    The requirements to use Azure OpenAI models on Lucidworks AI include:

    • A valid Azure subscription on Microsoft Azure.

    • Deployed Azure OpenAI models you want to use. Lucidworks does not support Azure AI Studio.

    • The Azure Deployment Name for the model you want to use. Use this as the value of the Lucidworks AI API "modelConfig": "azureDeployment" field.

    • The Azure Key1 or Key2 for the model you want to use. Use either as the value of the Lucidworks AI API "modelConfig": "apiKey" field.

    • The Azure Endpoint for the model you want to use. Use this as the value of the Lucidworks AI API "modelConfig": "azureEndpoint" field.

    • The Lucidworks AI API value of MODEL_ID for Azure OpenAI is azure-openai.

    Google Vertex AI models

    Each request requires an apiKey, googleProjectId, and googleRegion. There are no defaults for any of these fields.

    To generate the key, get a service account key with access to Google VertexAI and then base64 encode the key. For more information, see Create and delete service account keys.

    Available Google Vertex AI models

    The supported Google Vertex AI model is gemini-pro.

    Generative AI use cases

    The GenAI use cases are used to run predictions from pre-trained models.

    The Prediction API also contains the embedding use case (that is not categorized as GenAI use cases).

    For more information about models, see:

    The generic path for the Prediction API is /ai/prediction/USE_CASE/MODEL_NAME.

    The generic path for the Async Prediction API is /ai/async-prediction/USE_CASE/MODEL_NAME.

    The GenAI use cases based on the generic path are as follows:

    • Pass-through use case lets you use the Generative AI services as a proxy to the large language model (LLM). Use this use case when you want full control over the prompt sent to the GenAI model.

    • Retrieval augmented generation (RAG) use case uses candidate documents inserted into a LLM’s context to ground the generated response to those documents to prevent frequency of LLM hallucinative responses.

    • Standalone query rewriter use case rewrites the text in relation to information associated with the memoryUuid. This use case can be invoked during the RAG use case.

    • Summarization use case where the LLM ingests text and returns a summary of that text as a response.

    • Keyword extraction use case where the LLM ingests text and returns a JSON response that lists keywords extracted from that text.

    • Named Entity Recognition ner use case where the LLM ingests text and entities to extract and return a JSON response that contains a list of entities extracted from the text.

    Anthropic models

    An API key is required in each OpenAI model request. There is no default key.

    Available Anthropic models

    The supported Anthropic models are:

    • claude-3-5-sonnet-20241022

    • claude-3-5-haiku-20241022

    Most of the typical parameters are supported. However, the tooling part of the models or streaming is not currently supported.

    To set a system prompt in the pass-through, use the JSON prompt dataType example. This method automatically passes the prompt correctly.

    To set multiple system prompts, you must add all of the values in a single text string. If all of the values are not in a single string, only the last system prompt is sent in the query. This scenario supports all use cases.