> ## Documentation Index
> Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# JSON

> Parser stage configuration specifications

export const schema = {
  "type": "object",
  "title": "JSON",
  "description": "Parses JSON documents with optional splitting and mappings",
  "required": ["type"],
  "properties": {
    "id": {
      "type": "string",
      "title": "Parser ID",
      "default": "3d8fdc55-fd79-44a8-8021-b89fe821a07e"
    },
    "label": {
      "type": "string",
      "title": "Label",
      "description": "A label for this Parser Stage",
      "maxLength": 255
    },
    "enabled": {
      "type": "boolean",
      "title": "Enable this Parser Stage",
      "default": true
    },
    "mediaTypes": {
      "type": "array",
      "title": "Media Types to match",
      "description": "Documents with a media type on this list will be matched by this parser stage. See inheritMediaTypes / use default media types for more.",
      "items": {
        "type": "string",
        "pattern": "^[^\\/]+\\/[^\\/]+$",
        "format": "rfc2646"
      }
    },
    "inheritMediaTypes": {
      "type": "boolean",
      "title": "Match default media types in this Parser Stage",
      "description": "Each parser stage has a built-in list of media types it handles by default. If this setting is true, that list will be used along with any optional additional types provided in the mediaTypes list. If this setting is false, this stage will only be selected for media types in the mediaTypes list, and the mediaTypes list becomes a mandatory property which must have at least one valid media type.",
      "default": true
    },
    "ignoredMediaTypes": {
      "type": "array",
      "title": "Media Types to ignore",
      "description": "Documents with a media type on this list will be not be processed by this parser stage.",
      "items": {
        "type": "string",
        "pattern": "^[^\\/]+\\/[^\\/]+$",
        "format": "rfc2646"
      }
    },
    "pathPatterns": {
      "type": "array",
      "title": "File names to parse",
      "description": "Specify a file name or pattern that must be matched for this parser stage to run. Forward slashes (\"/\") are used to join names of files inside archives with the archive name.",
      "items": {
        "type": "object",
        "properties": {
          "syntax": {
            "type": "string",
            "title": "Pattern type",
            "description": "glob uses bash shell-style wildcards; regex uses Java (PCRE-style) regex",
            "enum": ["glob", "regex"],
            "default": "glob"
          },
          "pattern": {
            "type": "string",
            "title": "File name or pattern",
            "description": "e.g.: \"z.txt\" or \"*.md\" or \"/a/*/b/f.txt\" for glob; \"z.txt$\" or \".*\\.txt$\" or \"^/a/[^\\/]*/b/f.txt$\" for regex"
          }
        }
      }
    },
    "errorHandling": {
      "type": "string",
      "title": "Error Handling",
      "enum": ["ignore", "log", "fail", "mark"],
      "default": "mark"
    },
    "outputFieldPrefix": {
      "type": "string",
      "title": "Prefix parsed fields with",
      "description": "Fields extracted by this parser will be prefixed with this string. The remainder of the field name will be as detected in the stream",
      "maxLength": 20,
      "pattern": "^$|^[A-Za-z_][A-Za-z0-9_\\-\\.]+$"
    },
    "rootPath": {
      "type": "string",
      "title": "Root path",
      "description": "Use only children of this JSON pointer."
    },
    "includePath": {
      "type": "boolean",
      "title": "Include root path",
      "description": "Include parent element names when using a root path.",
      "default": false
    },
    "splitArrays": {
      "type": "boolean",
      "title": "Split arrays",
      "description": "First split top-level arrays into multiple documents, and then apply other rules.",
      "default": true
    },
    "expectJsonL": {
      "type": "boolean",
      "title": "Expect JSONL",
      "description": "Expect the input to contain multiple line separated JSON documents",
      "default": false
    },
    "maxLineSize": {
      "type": "integer",
      "title": "Max line size",
      "description": "Set maximum size of a line in bytes. This is important for processing JsonL with large documents.",
      "default": 8192
    },
    "mappings": {
      "type": "array",
      "title": "Mapping rules",
      "description": "Extract parts of the document into specified fields",
      "items": {
        "type": "object",
        "required": ["path", "target"],
        "properties": {
          "path": {
            "type": "string",
            "title": "JSONPath expression"
          },
          "target": {
            "type": "string",
            "title": "Target field"
          }
        }
      }
    },
    "listHandling": {
      "type": "string",
      "title": "JSON List handling",
      "description": "Create a single multivalued field containing all items, or a separate index-numbered field per list item?",
      "enum": ["multivalued", "index_numbered"],
      "default": "multivalued",
      "hints": ["advanced"]
    },
    "type": {
      "type": "string",
      "enum": ["json"],
      "default": "json"
    }
  },
  "additionalProperties": false,
  "category": "Other",
  "categoryPriority": 1,
  "unsafe": false
};

export const SchemaParamFields = ({schema}) => {
  const sanitize = str => {
    if (typeof str !== "string") return str;
    return str.replace(/^"(.*)"$/s, "$1").replace(/\\/g, "").replace(/"/g, "'");
  };
  const formatDescription = str => {
    const s = sanitize(str);
    return (/[.!?]\)*$/).test(s) ? s : `${s}.`;
  };
  const {description, properties = {}, required: requiredProps = []} = schema;
  const visibleProps = useMemo(() => Object.entries(properties).filter(([, prop]) => !prop.hints?.includes("hidden")), [properties]);
  return <div>
      {description && <p>{formatDescription(description)}</p>}

      {visibleProps.map(([name, prop]) => {
    const isRequired = requiredProps.includes(name);
    const hasDefault = prop.default !== undefined;
    const rawDefault = prop.default;
    const isComplexDefault = hasDefault && (typeof rawDefault === "object" || typeof rawDefault === "string" && (rawDefault.length > 20 || rawDefault.includes('"')));
    const fieldProps = {
      key: name,
      body: prop.title || name,
      type: prop.type,
      ...prop.title && ({
        post: [<><span className="text-stone-400 dark:text-stone-500">API property: </span>{name}</>]
      }),
      ...isRequired && ({
        required: true
      }),
      ...!isComplexDefault && hasDefault ? {
        default: sanitize(String(rawDefault))
      } : {}
    };
    const isObject = prop.type === "object" && prop.properties;
    const isArrayOfObjects = prop.type === "array" && prop.items?.type === "object" && prop.items.properties;
    return <ParamField {...fieldProps}>
            {prop.description && <p>{formatDescription(prop.description)}</p>}

            {isComplexDefault && <div className="flex">
                <p>
                  <strong>Default:</strong>
                </p>
                <pre className="!my-0">
                  <code>
                    {JSON.stringify(rawDefault, null, 2)}
                  </code>
                </pre>
              </div>}

            {isArrayOfObjects && <div className="flex">
              <p>
                <strong>Object attributes:</strong>
              </p>
              <pre className="!my-0">
                <code>
                  {'{\n'}
                  {Object.entries(prop.items.properties).map(([iname, iprop]) => <>
                      {`  ${iname}`}
                      {prop.items?.required?.includes(iname) && <span style={{
      color: 'red'
    }}> required</span>}
                      {`: {\n    display name: ${sanitize(iprop.title || '')}\n    type: ${iprop.type}\n  }\n`}
                    </>)}
                  {'}'}
                </code>
              </pre>
              </div>}

            {isObject && <Expandable title="properties">
                <SchemaParamFields schema={{
      properties: prop.properties,
      required: prop.required
    }} />
              </Expandable>}
          </ParamField>;
  })}
    </div>;
};

export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => {
  const [isLoaded, setIsLoaded] = useState(false);
  useEffect(() => {
    const timer = setTimeout(() => {
      setIsLoaded(true);
    }, 500);
    return () => clearTimeout(timer);
  }, []);
  return <div className="lw-template-container">
      <Card title={title} icon={icon}>
        {isLoaded && <span dangerouslySetInnerHTML={{
    __html: `<lw-template id="a029c1a9-28be-427e-b0e1-5d918920246a"></lw-template
            >`
  }} />}
        <Link href={linkHref} className="agent-studio-link text-left text-gray-600 gap-2 dark:text-gray-400 text-sm font-medium flex flex-row items-center hover:text-primary dark:hover:text-primary-light group-hover:text-primary group-hover:dark:text-primary-light">Powered by Lucidworks Agent Studio</Link>
      </Card>
    </div>;
};

[localhost link]: http://localhost:3000/docs/lucidworks-search/09-developer-documentation/config-specs/parsers/json-parser

[mintlify link]: https://doc.lucidworks.com/docs/lucidworks-search/09-developer-documentation/config-specs/parsers/json-parser

[old doc.lw link]: https://doc.lucidworks.com/managed-fusion/5.9/1cp12w

JSON parsing converts JSON content from a single document field into one or more new documents. This parser uses Solr’s
[JsonRecordReader](https://lucene.apache.org/solr/5_1_0/solr-solrj/org/apache/solr/common/util/JsonRecordReader.html) to split JSON into sub-documents.

<LwTemplate />

If your JSON file contains a column named `id`, this column is consumed to populate the document's unique identifier (Solr's `uniqueKey`) and is not available as a stored field.

This behavior occurs because the default value of the parser's **Document ID Source Field** parameter is also `id`. When a CSV column matches this parameter:

* The column's value is used to generate the document ID.
* The column does not appear in the indexed document as a field.

If you need to preserve your `id` column data as a regular field, use one of these options:

* Change the column header from `id` to another name such as `record_id` or `item_id`. This is the simplest solution.
* In the JSON parser stage configuration, use a mapping rule to map `$.id` to another name such as `record_id` or `item_id`.
* In the Index Workbench's parser configuration, set the **Document ID Source Field** to a different column name. This allows `id` to be treated as a normal field, but you must specify a different column to use as the document identifier.

See [Parsers Overview](/docs/lucidworks-search/04-move-data-in/parsers/overview) for information about configuring the **Document ID Source Field** parameter.

<Tip>
  When entering configuration values in the UI, use *unescaped* characters, such as `\t` for the tab character. When entering configuration values in the API, use *escaped* characters, such as `\\t` for the tab character.
</Tip>

<SchemaParamFields schema={schema} />