Product Selector

Fusion 5.9
    Fusion 5.9

    Add Custom Headers to HTTP Requests

    The Web V1 connector retrieves data from a Web site using HTTP and starting from a specified URL.

    In Fusion 4.1 and later, you can optionally add custom headers to all http get requests from the Web connector. For example, you might want to add a header that includes Connection: keep-alive to prevent the connector from timing out while crawling your Web site.

    To add a custom header, use the configuration parameter f.addedHeaders. To send multiple headers, use the following format:

    Header1: Value1
Header2: Value2
    To add a custom header in the Fusion UI on any node:

    1. Click Indexing > Datasources.

    2. Click Add+, then Web.

    3. Enter a datasource ID and a start link.

    4. Click Link discovery.

    5. Fill in the Headers to add to HTTP requests field.

      • Add each header in the format HeaderName: HeaderValue.

      • To add multiple headers to all HTTP requests, put each header on a new line.

    6. Fill in any remaining options to configure the datasource.

    7. Click Save.