Product

Fusion 5.8

Enable cloud signal storage

This article teaches you how to enable Fusion’s cloud signal storage for Google Cloud Storage or Amazon S3.

Fusion deployment setup

To enable cloud signal storage, start with a new deployment. Cloud signal storage is enabled in your custom values YAML file.

You must create an authentication secret in your namespace, regardless of which cloud storage provider you choose.

Google Cloud Storage

  1. Add the following values to your custom values YAML file:

    cloudSignals:
  enabled: true
  enabledStorage:
    - gcs
  gcs:
    outputDir: "gs://BUCKET_NAME"
    cloudSecret: SERVICE_ACCOUNT_SECRET

  2. Change BUCKET_NAME to the name of your bucket in Google Cloud Storage.

  3. Change SERVICE_ACCOUNT_SECRET to your service account’s authentication secret. For more information on generating a service account secret, see Creating service account credentials.

  4. Deploy Fusion using these custom values.

Amazon S3

  1. Add the following values to your custom values YAML file:

    cloudSignals:
  enabled: true
  enabledStorage:
    - s3
  s3:
    outputDir: "s3a://BUCKET_NAME"
    cloudSecret: AWS_SECRET
    region: REGION

  2. Change BUCKET_NAME to the name of your bucket in Google Cloud Storage.

  3. Change AWS_SECRET to your AWS secret value. For more information on generating a service account secret, see Accessing AWS using your AWS credentials. For information on storing your AWS credentials in Kubernetes, see Store AWS Credentials in Kubernetes Secret.

  4. Change REGION to the region value for your bucket. For more information on regions, see see AWS regional endpoints. To retrieve the region for your bucket, see AWS get-bucket-location.

  5. Deploy Fusion using these custom values.

Application setup

  1. Create a new Fusion application and index some data. For more information, see Getting Started with Fusion Server.

  2. Update the click signals aggregation job.

    1. Navigate to Collections > Jobs and select the click signals aggregation job. By default, this job is named APP_NAME_click_signals_aggregation.

    2. Change the Source Collection value to the path of your cloud storage location. For example: gs://cloud-signals/collection=COLLECTION_NAME.

      You might see a message that states, "This collection does not exist." This is expected with cloud signal collections. Ensure your path is correct and disregard this message.

    3. Change the Data Format value to parquet.

    4. Click Save to save the changes to the job.

  3. Send signals to your application. For testing, you can send signals manually.

    1. To send signals using the API:

      curl -u USERNAME:PASSWORD -X POST \
    --url 'https://FUSION_HOST.com/api/apps/APP_NAME/signals/APP_NAME?async=false&commit=true' \
    --header 'Content-Type: application/json' \
    --header 'cache-control: no-cache' \
    --data '[
        {
          "type": "SIGNAL_TYPE",
          "params": {
            "docId": "DOCUMENT_ID",
            "count": "NUMBER_OF_SIGNALS",
            "collection": "APP_NAME",
            "query": "*:*",
            "filterQueries": []
          }
        }
        ]'

      Replace placeholder values, such as APP_NAME, with your environment values. The SIGNAL_TYPE value can be any signal type, such as click, response, cart, or a custom signal type.

    2. To send signals in the Fusion UI from the Query Workbench:

      1. Navigate to Querying > Query Workbench.

      2. Click Format Results.

      3. Enable the Send click signals option.

      4. Click Save.

      5. Click a result title in the query workbench to send a signal.

  4. Verify your signals were captured in your cloud storage. Signals are stored with your provider as follows:

    SERVICE_PROVIDER://BUCKET_NAME/collection=COLLECTION_NAME/type=SIGNAL_TYPE/day_of_year=DAY_OF_YEAR/FILENAME.parquet
Why is my signals collection empty?

This is expected. The signals collection contains signals located in a Solr cluster. If you’re using cloud signal storage, this collection will be empty.