> ## Documentation Index > Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt > Use this file to discover all available pages before exploring further. # Solr Partial Update Indexer Stage export const schema = { "type": "object", "title": "Solr Partial Update Indexer", "description": "This stage indexes partial (atomic) updates to Solr documents", "required": ["solrDocIdFieldName", "solrDocIdFieldValue"], "properties": { "skip": { "type": "boolean", "title": "Skip This Stage", "description": "Set to true to skip this stage.", "default": false, "hints": ["advanced"] }, "label": { "type": "string", "title": "Label", "description": "A unique label for this stage.", "hints": ["advanced"], "maxLength": 255 }, "condition": { "type": "string", "title": "Condition", "description": "Define a conditional script that must result in true or false. This can be used to determine if the stage should process or not.", "hints": ["code", "code/javascript", "advanced"] }, "enforceSchema": { "type": "boolean", "title": "Map to Solr Schema", "default": true }, "concurrencyControlEnabled": { "type": "boolean", "title": "Enable Concurrency Control", "description": "Select to enable Optimistic Concurrency Control in Solr, guaranteeing that the document update will not be overridden by another Partial Update to the same document. If disabled, in the case of an edit collision, the last committed update to the document will win.", "default": true }, "rejectUpdatesIfDocNotPresent": { "type": "boolean", "title": "Reject Update if Solr Document is not Present", "description": "Whether to reject the update attempt if the document with given id is not present in Solr. This is not typical situation since the updates usually are performed on existing documents, however you may disable this to attempt update even if the document is not present. If the concurrency control is disabled, enabling this flag will force set the _version_ field to 1, or to 0 otherwise.", "default": true }, "updateAllDocFields": { "type": "boolean", "title": "Process All Pipeline Doc Fields", "description": "If this option is set, the Partial Update Stage will process pipeline document fields even if they are not set by Updates and Deletions instructions here. In this case those fields will be included into the partial update document and will be processed by Solr according to atomic update rules, i.e. non-map field value(s) will be treated as a 'set' update for the field, and Map field values will be processed as an atomic update defined in the Map. The Map structure should comply to Solr atomic update rules. Note that the Partial Update stage does NOT validate consistency of fields that are not Updates or Deletions configured here, it just sends them to Solr 'as is'.", "default": false }, "solrDocIdFieldName": { "type": "string", "title": "Solr Document ID Field Name", "default": "id" }, "solrDocIdFieldValue": { "type": "string", "title": "Solr Document ID Field Value", "default": "" }, "dateFormats": { "type": "array", "title": "Additional Date Formats", "hints": ["advanced"], "items": { "type": "string" } }, "params": { "type": "array", "title": "Additional Update Request Parameters", "hints": ["advanced"], "items": { "type": "object", "required": ["key"], "properties": { "key": { "type": "string", "title": "Parameter Name" }, "value": { "type": "string", "title": "Parameter Value" } } } }, "updatedFields": { "type": "array", "title": "Updates", "description": "Fields to update (set, add or remove field values) in the Solr Document.", "items": { "type": "object", "required": ["fieldName", "values"], "properties": { "updateType": { "type": "string", "title": "Update Type", "description": "The Update Type", "enum": ["set", "add", "remove", "remove_regex", "increment", "decrement"], "default": "set" }, "fieldName": { "type": "string", "title": "Field Name", "description": "The Solr Document Field to update." }, "values": { "type": "string", "title": "Value", "description": "For increment operation only one value (positive or negative int) is allowed. For add, set, remove or remove_regex a single value or list of values can be specified. The list separator is a comma (,) if the comma should be present in the field value, escape it with a backslash (\\). Quotation marks(\"\") can be used to enclose Field value to preserve the white spaces, if needed." } } } }, "deletedFields": { "type": "array", "title": "Deletions", "description": "Fields to Delete from Solr Document.", "items": { "type": "object", "required": ["fields"], "properties": { "fields": { "type": "string", "title": "Field" } } } }, "positionalUpdates": { "type": "array", "title": "Positional Updates", "description": "Update Field or Group of Fields to update (add or set) value at a specific position. See documentation for additional information.", "hints": ["advanced"], "items": { "type": "object", "required": ["position", "fieldsAndValues"], "properties": { "positionalUpdateType": { "type": "string", "title": "Update Type", "description": "The Update Type", "enum": ["set", "add"], "default": "set" }, "position": { "type": "string", "title": "Position", "description": "The position at which the new value(s) will be changed. Could be 'first', 'last' or numeric value (position index)." }, "fieldsAndValues": { "type": "string", "title": "Fields and Values", "description": "The field:value [,field:value ...] list. The values will be changed for specified fields at specified position. The list separator is comma (,) if the comma present in the field value, escape it with a backslash (\\). Quotation marks(\"\") can be used to enclose Field value to preserve the white spaces, if needed." } } } }, "positionalRemovals": { "type": "array", "title": "Positional Removals", "description": "Update Field or Group of Fields to remove value at a specific position. See documentation for additional information.", "hints": ["advanced"], "items": { "type": "object", "required": ["position", "fields"], "properties": { "position": { "type": "string", "title": "Position", "description": "The position at which the field value will be removed. Could be 'first', 'last' or numeric value (position index)." }, "fields": { "type": "string", "title": "Fields List", "description": "The field [,field ...] list of fields (Solr field names) where removal of a value at specified position should happen." } } } }, "customRouteFieldName": { "type": "string", "title": "Custom Route Field Name", "description": "This option is used when custom shard routing is configured is Solr so the document route is defined by value of Solr document's field (defined as 'router.field' when created the collection). If set here, the field with this name will be transferred to the partial update Solr document from the pipeline document." } }, "category": "Indexing", "categoryPriority": 4, "unsafe": false }; export const SchemaParamFields = ({schema}) => { const sanitize = str => { if (typeof str !== "string") return str; return str.replace(/^"(.*)"$/s, "$1").replace(/\\/g, "").replace(/"/g, "'"); }; const formatDescription = str => { const s = sanitize(str); return (/[.!?]\)*$/).test(s) ? s : `${s}.`; }; const {description, properties = {}, required: requiredProps = []} = schema; const visibleProps = useMemo(() => Object.entries(properties).filter(([, prop]) => !prop.hints?.includes("hidden")), [properties]); return
{description &&

{formatDescription(description)}

} {visibleProps.map(([name, prop]) => { const isRequired = requiredProps.includes(name); const hasDefault = prop.default !== undefined; const rawDefault = prop.default; const isComplexDefault = hasDefault && (typeof rawDefault === "object" || typeof rawDefault === "string" && (rawDefault.length > 20 || rawDefault.includes('"'))); const fieldProps = { key: name, body: prop.title || name, type: prop.type, ...prop.title && ({ post: [<>API property: {name}] }), ...isRequired && ({ required: true }), ...!isComplexDefault && hasDefault ? { default: sanitize(String(rawDefault)) } : {} }; const isObject = prop.type === "object" && prop.properties; const isArrayOfObjects = prop.type === "array" && prop.items?.type === "object" && prop.items.properties; return {prop.description &&

{formatDescription(prop.description)}

} {isComplexDefault &&

Default: 

                  
                    {JSON.stringify(rawDefault, null, 2)}
} {isArrayOfObjects &&

Object attributes: 

                
                  {'{\n'}
                  {Object.entries(prop.items.properties).map(([iname, iprop]) => <>
                      {`  ${iname}`}
                      {prop.items?.required?.includes(iname) &&  required}
                      {`: {\n    display name: ${sanitize(iprop.title || '')}\n    type: ${iprop.type}\n  }\n`}
                    )}
                  {'}'}
} {isObject && } ; })}
; }; export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => { const [isLoaded, setIsLoaded] = useState(false); useEffect(() => { const timer = setTimeout(() => { setIsLoaded(true); }, 500); return () => clearTimeout(timer); }, []); return
{isLoaded && ` }} />} Powered by Lucidworks Agent Studio
; }; [localhost link]: http://localhost:3000/docs/4/fusion-server/reference/pipeline-stages/indexing/solr-partial-update-indexer-stage [mintlify link]: https://doc.lucidworks.com/docs/4/fusion-server/reference/pipeline-stages/indexing/solr-partial-update-indexer-stage [old doc.lw link]: https://doc.lucidworks.com/fusion/5.9/233 The Solr Partial Update Indexer Stage updates of one or more fields of an existing Solr document in a collection managed by Fusion. It provides an alternative to the Solr Indexer stage. When a data feed consists of an ongoing flow of messages about known documents in a collection, such as item price, inventory counts, or weather conditions at a location, this stage provides fast indexing throughput and can be configured to enforce data atomicity to guarantee that the index always reflects the most recent update. This stage is configured with a set of update directives based on Solr’s [atomic updates](https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents). At run time, it creates a Solr update by applying these directive to the data from a Fusion PipelineDocument object and then submits this update to Solr’s update handler. Solr’s atomic update functionality requires that the schema for a collection is configured so that all fields have the attribute stored="true", excepting fields which are destinations which must be configured as stored="false". ## Example Stage Specification *Configuration for a Partial Updater Stage in JSON:* ```json wrap theme={"dark"} { "type" : "solr-partial-update-index", "enforceSchema" : false, "solrDocIdFieldName" : "id", "solrDocIdFieldValue" : "", "updatedFields" : [ { "updateType" : "set", "fieldName" : "statusValue", "values" : "" }, { "updateType" : "set", "fieldName" : "lastCommunicationTime", "values" : "" } ], "concurrencyControlEnabled" : true, "skip" : false, "label" : "solr-partial-update-index", } ``` The expression \ will evaluate to the contents of the current PipelineDocument’s field named "X". ## Types of Update Operations The set of update operations are based on operations supported by Solr. They are: * 'add' - add a new value or values to an existing Solr document field, or add a new field and value(s). * 'set' - change the value or values in an existing Solr document field. * 'remove' - remove all occurrences of the value or values from an existing Solr document field. * 'removeregex' - remove all occurrences of the values which match the regex or list of regexes from an existing Solr document field. * 'increment' - increment the numeric value of existing Solr document field by a specific amount. * 'decrement' - decrement the numeric value of existing Solr document field by a specific amount. In addition, this stage introduces experimental "Positional" operations which can be used to add, set or remove exactly one element of a field which takes a list of values (i.e, a multi-valued field). * 'positionalUpdates' - used to add or set the value at specific position. * 'positionalRemoves' - used to delete an element at a specific position. When a collection contains two or more multi-value fields which are maintained in parallel so that taken together, they act like a table stored column by column, a positional update operation updates several data cells across one row of the table. To maintain this kind of column-oriented table, the positional delete directive must specify all the fields in the document which logically comprise the table. ## Document Identifier Field A Fusion collection is a Solr collection managed by Fusion. Underlyingly, a Solr document is a list of named, typed fields. The Solr [unique key field](https://wiki.apache.org/solr/UniqueKey) stores a string which is the unique identifier for that document. There is at most one UniqueKey field per document, which is defined in the Solr schema. The UniqueKey field value is required. For collections created via Fusion, the UniqueKey field is named "id". Other document fields may also store string values which can be used as a unique identifier. Solr uses the UniqueKey field to find the document to be updated. If the data feed information contains a document identifier which is different than the identifier value stored in the UniqueKey field, then this stage must do a Solr lookup to find the UniqueKey value. ## Optimistic Concurrency Solr’s [Optimistic Concurrency](http://yonik.com/solr/optimistic-concurrency/) is a mechanism which checks whether or not a document has changed between the point at which an update request was submitted and the point at which the request is processed. Solr documents have an internal field named "*version*" which is updated whenever there is any change made to any of the other fields in that document. When optimistic concurrency control is on, update requests will be discarded if the current version of the document has changed since that request was made. This guarantees that the document will always reflect the most recent update. However, this require an additional Solr lookup to get the current document version number, which is submitted as part of the update request. ## Performance Considerations In order to send a single update request to Solr, without preliminary lookup requests: * The document identifier field should match the Solr collection’s UniqueKey identifier field. * Optimistic Concurrency should be turned off. * Positional updates are experimental and potentially expensive, since all the values for all fields being updated must be fetched into memory in order to perform positional operations. ## Solr Date Formats ``` "yyyy-MM-dd'T'HH:mm:ss'Z'", // Solr format without milliseconds "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", // standard Solr format, with literal "Z" at the end "yyyy-MM-dd'T'HH:mm:ss.SS'Z'", // standard Solr format, with literal "Z" at the end "yyyy-MM-dd'T'HH:mm:ss.S'Z'" // standard Solr format, with literal "Z" at the end ``` See [https://cwiki.apache.org/confluence/display/solr/Working+with+Dates](https://cwiki.apache.org/confluence/display/solr/Working+with+Dates) ## Configuration When entering configuration values in the UI, use *unescaped* characters, such as `\t` for the tab character. When entering configuration values in the API, use *escaped* characters, such as `\\t` for the tab character.