Skip to main content
In the image metadata enrichment use case of the LWAI Prediction API, the LLM ingests a text containing either a base64-encoded image or a public HTTP/HTTPS or Google Cloud Storage (gs://) image link and routes the image to a multimodal model. You can also supply the image using image or imageLink keys in each batch item. A JSON response is returned that contains a imageMetadata dictionary with a list of keywords, a list of subcategories, and a list of keywords synonyms. The metadata can be generated based on the image, the image title, and the categories. This use case can be applied in the following ways:
  • Enrich keywords for an image in a document.
  • Generate SEO and alt text.
  • Help with image-based product discovery.
  • Enrich document indexing.
  • Visually tag images for search and faceting.
  • Expand synonyms for search recall.
  • Auto-categorize images.
  • Validate or correct categories.
  • Increase semantic linking and recommendations.

​
Unique values for the image metadata enrichment use case

batch

An array of request items. Each item is contained within curly brackets and accepts one of the following.
​
text
string
Free-form content contained in the document. Pass in either a base64-encoded image or an image link and it will be parsed to the correct field. This is meant to support Fusion specifically.
​
image
string
A field meant to take a string of a base64-encoded image. If included, imageLink may not be included.
​
imageLink
string
A field for HTTP/HTTPS or pulic Google Cloud Storage links (gs://). If included, image may not be included.

useCaseConfig

Optional object to control metadata generation, such as what to consider and how much to return. The image must be provided in the batch item as useCaseConfig only tunes the output.
​
title
string
This optional parameter is designated for the title of the image or other details to include for the generated items.
​
categories
List[string]
This optional parameter specifies the current categories for the item and is the basis for the subcategories generated.
​
maxKeywords
integer
This optional parameter has a default of 15 and specifies the maximum number of keywords the model should generate.
​
maxSubcategories
integer
This optional parameter has a default of 5 and specifies the maximum number of subcategories the model should generate.
​
maxSynonyms
integer
This optional parameter has a default of 5 and specifies the maximum number of synonyms the model should generate per keyword.
​
locale
string
This optional parameter has a default of en-US and specifies the locale when generating targeted vernacular.

​
Example requests

These examples show how to use the different batch parameters.

​
Example using batch with text

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/prediction/image-metadata-enrichment/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
    "batch": [
        {
            "text": "https://i.postimg.cc/T3Gggmrx/blushing-happymushroom.png"
        }
    ],
    "useCaseConfig": {
        "title": "blushing-happy-mushroom",
        "categories": [
            "fungi",
            "cute",
            "jubilant",
            "embarrassed"
        ],
        "maxKeywords": 3,
        "maxSynonyms": 2,
        "maxSubcategories": 2,
        "locale": "en-US"
    }
}'

curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/prediction/image-metadata-enrichment/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
    "batch": [
        {
            "imageLink": "https://i.postimg.cc/T3Gggmrx/blushing-happymushroom.png"
        }
    ],
    "useCaseConfig": {
        "title": "blushing-happy-mushroom",
        "categories": [
            "fungi",
            "cute",
            "jubilant",
            "embarrassed"
        ],
        "maxKeywords": 3,
        "maxSynonyms": 2,
        "maxSubcategories": 2,
        "locale": "en-US"
    }
}'

​
Example using batch with image data encoded for Macs

encoded=$(base64 -i blushing_happymushroom.png -o - | tr -d '\n');
curl --request POST \
  --url https://APPLICATION_ID.applications.lucidworks.com/ai/prediction/image-metadata-enrichment/MODEL_ID \
  --header 'Authorization: Bearer ACCESS_TOKEN' \
  --header 'Content-type: application/json' \
  --data '{
    "batch": [
        {
            "image": "$encoded"
        }
    ],
    "useCaseConfig": {
        "title": "blushing-happy-mushroom",
        "categories": [
            "fungi",
            "cute",
            "jubilant",
            "embarrassed"
        ],
        "maxKeywords": 3,
        "maxSynonyms": 2,
        "maxSubcategories": 2,
        "locale": "en-US"
    }
}'

​
Example response

The following is an example response for all of the above requests: 
{
    "predictions":[
    {
        "tokensUsed": {
            "promptTokens": 1614,
            "completionTokens": 61,
            "totalTokens": 1675
        },
        "imageMetadata": {
            "keywords": [
                "cartoon",
                "pleased",
                "toadstool"
            ],
            "subcategories": [
                "happy",
                "mushroom art"
            ],
            "synonyms": [
                "animation",
                "cheerful",
                "delighted",
                "drawing",
                "fungus",
                "sporocarp"
            ],
            "locale": "en-US"
        },
        "response": "```yaml\nkeywords:\n  - toadstool\n  - pleased\n  - cartoon\nsubcategories:\n  - mushroom art\n  - happy\nsynonyms:\n  - fungus\n  - sporocarp\n  - delighted\n  - cheerful\n  - drawing\n  - animation\n```"
        }
    ]
}
⌘I