Skip to main content

Documentation Index

Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt

Use this file to discover all available pages before exploring further.

The Format Signals stage normalizes both the fields and field contents of a PipelineDocument to ensure that certain pre-defined fields for signals are populated.

​
Date/time parsing and formatting

Timestamp data can be obtained from the following fields, in this order of precedence:
  • timestamp
  • timestamp_tdt
  • timestamp_dt
  • epoch. value in this field is treated as a number of milliseconds since epoch, and UTC zone is assumed.
It is now possible to specify the locale to be used for parsing timestamps by setting the “timestampLocale” property in the stage configuration. If this property is null then the default platform locale will be used. Output document will carry the following two fields:
  • “timestamp” - containing the ISO8601 timestamp
  • “tz_timestamp_txt” - containing the “zoned format” of the timestamp with normalized components.
This stage does not define a list of allowed types.

​
Example Stage Specification

The Format Signals stage defined as part of the default ‘signals_ingest’ pipeline included with Fusion: 
{
    "type" : "format-signals",
    "id" : "ingest-signals",
    "flatten" : true,
    "undefinedType" : "general",
    "skip" : false,
    "label" : "format-signals",
    "type" : "format-signals"
  }

​
Configuration

When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.