Exporting and Importing Fusion Objects

Fusion allows you to export objects from one Fusion instance and import them into another. In this way, you can back up your Fusion data, copy it to new instances, or restore it after an incident of data loss.

Exporting can only be performed using the Objects API. Importing can be performed using the API or the UI.

A separate topic explains how to migrate Fusion configuration data.

Object export and import

Collections and encrypted values are treated specially; details are provided below. During import, conflicts are resolved according to the specified import policy.

For objects other than collections, no implicit filtering is performed; all objects are included by default. However, on export you can filter by type and ID.

Supported objects

Fusion lets you export and import these types of objects:

  • collections (with their dependent objects)

    See below for collection-specific details.

  • index pipelines

  • query pipelines

  • search clusters

  • schedules

  • aggregations

  • datasources

  • dashboards (banana)

  • parsing configurations

Exporting and importing collections

Collections are processed with these dependent objects:

  • features

  • index profiles

  • query profiles

Datasources, parser configurations, and pipeline configurations are not included when collections are exported or imported. These must be exported and imported explicitly.

Only user-created collections are included by default. Certain types of collections are excluded:

  • the "default" collection

  • collections whose type is not DATA

  • collections whose names start with "system_"

  • "Secondary" collections, that is, collections created by features

    Instead, create the same features on the target system; this automatically creates the corresponding secondary collections.

You can override these exclusions by specifying a collection, like this:

http://localhost:8764/api/apollo/objects/export?collection.ids=default

Encrypted passwords

Some objects, such as datasources and pipelines, include encrypted passwords for accessing remote data.

  • On export, these encrypted values are replaced with ${secret.n.nameOfProperty}.

  • On import, the original, plaintext passwords must be provided in a JSON map:

    {"secret.1.bindPassword" : "abc", "secret.2.bindPassword" : "def"}

    The file must be supplied as multipart form data.

Import policies

On import, the importPolicy parameter is required. It specifies what to do if any object in the import list already exists on the target system:

abort

If there are conflicts, then import nothing.

merge

If there are conflicts, then skip the conflicting objects.

overwrite

If there are conflicts, then overwrite or delete/create the conflicting objects on the target system.

Filtering on export

On export, there are two ways to specify the objects to include:

  • by type

    You can specify a list of object types to export all objects of those types. Valid values:

    • collection

    • index-pipeline

    • query-pipeline

    • search-cluster

    • schedule

    • aggregation

    • datasource

    • banana

    • parsing-configuration

  • by type and ID

    The type.ids parameter lets you list the IDs to match for the specified object type.

The type and type.ids parameters can be combined as needed.

Validation

Objects are validated before import. If any objects fail validation, the whole import request is rejected. A separate endpoint is available for validating objects without importing them.

Validation includes checking whether an object already exists on the target system and whether the user is authorized to create or modify the object.

For collection objects, the following special validation is performed:

  • We check the searchClusterId of each collection and verify that a cluster with this ID exists on the target system or in the import file (error).

  • We check that features, index profiles, and query profiles belong only to the collections specified in the import file (error).

  • We check that a feature exists on the target system for each feature in the import file (error).

  • We check for index profiles or query profiles that do not exist on the target system or in the import file (warning).

Status messages

Validation completed with no errors

The validation method was called and no errors found, though there may be warnings.

Validation found errors

The validation was called and errors found. Validation does not stop on the first error, so the complete list of errors is reported.

Validation was not completed because of system error

The validation was interrupted by system error.

Import was not performed because validation errors exist

The import method was called, but import didn’t start because of validation errors.

Import was not performed because of input data error

The import method was called, but import didn’t start, because Fusion could not find a substitution for one of the secret values in objects in import.

Import was not completed because of system error

The validation found no errors and import started, but it was interrupted by system error.

Import was completed

Validation found no errors and import finished successfully.

How to export Fusion objects

Exporting can only be performed using the Objects API.

You can select all objects, or limit the operation to specific object types or IDs. In addition to export endpoints, a validation endpoint is provided for troubleshooting.

Note
By default, system-created collections are not exported.

Some example requests are shown below. For complete reference information about object export endpoints, see the Objects API.

Export all objects
http://localhost:8764/api/apollo/objects/export
Export all schedules
http://localhost:8764/api/apollo/objects/export?type=schedule
Export all datasources and pipelines, plus two specific parsing configurations
http://localhost:8764/api/apollo/objects/export?type=datasource,index-pipeline,query-pipeline&parsing-configuration.ids=cinema_parser,metafiles_parser

How to import Fusion objects

Objects can be imported using the REST API or the Fusion UI.

Importing objects with the REST API

Some example requests are shown below. For complete reference information about object export endpoints, see the Objects API.

Import objects from a file and stop if there are conflicts
curl -u admin:password123 -H "Content-Type:multipart/form-data" -X POST -F 'importData=@/Users/admin/Fusion/export.json' http://localhost:8764/api/apollo/objects/import?importPolicy=abort
Import objects, substitute the password variables, and merge any conflicts
curl -u admin:password123 -H "Content-Type:multipart/form-data" -X POST -F 'importData=@/Users/admin/Fusion/export.json' -F 'variableValues=@password_file.json' http://localhost:8764/api/apollo/objects/import?importPolicy=merge
Note
password_file.json must contain plaintext passwords.

Importing objects with the Fusion UI

How to import objects using the UI
  1. launcher menu devops In the upper left, click the Launcher button and select Devops.

  2. In the Home panel, click Import Fusion Objects.

    The Import Fusion Objects window opens.

  3. Select the data file from your local filesystem.

    If you are importing passwords, also select the JSON file that maps variables to plaintext passwords.

    import objects

  4. Click Import.

    If there are conflicts, Fusion prompts you to specify an import policy:

    import objects2

    • Click Overwrite to overwrite the objects on the target system with the ones in the import file.

    • Click Merge to skip all conflicting objects and import only the non-conflicting objects.

    • Click Start Over to abort the import.

    Fusion confirms that the import was successful:

    import objects3

  5. Click Close to close the Import Fusion Objects window.