Update your Confluence recipe
If you are using theconfluence-v1.json recipe, you must migrate to the confluence.json recipe.
You have two options for updating to the confluence.json recipe:
- Clear an existing datasource and replace the recipe. This is the recommended method.
- Create a new datasource with the new recipe.
confluence.json recipe file and edited the following values in the file:
serviceURL: Your Confluence URL- Confluence username and password or API token
collection: Your Fusion collection namepipeline: Your Fusion pipeline name
Clear an existing datasource
Clear an existing datasource
To use the same REST V2 datasource for your updated Confluence recipe:
- Navigate to Indexing > Datasources and select your existing REST V2 datasource.
- Click Clear Datasource.
- Use the Connector Datasources API to replace the datasource configuration. Replace
AUTHORIZATION_CREDENTIALSwith your Lucidworks login information in Base64:
- Return to your datasource in Fusion. Make any additional edits to your configuration and click Save.
- Click Run, and then click Start to run your datasource.
Create a new datasource
Create a new datasource
To use a new datasource for your updated Confluence recipe:
- Navigate to Indexing > Datasources.
- Click Add + and select REST (v2) to create a new datasource.
- Fill out the following required fields:
- Enter a name for the connector in the Configuration ID field.
- Enter your Fusion pipeline in the Pipeline ID field.
- Enter your Confluence URL in the Service URL field.
- Upload the recipe to Fusion with the Connector Datasources API. Replace
AUTHORIZATION_CREDENTIALSwith your Lucidworks login information in Base64:
- Return to your datasource in Fusion. Make any additional edits to your configuration and click Save.
- Click Run, and then click Start to run your datasource.
Confluence REST configuration
This documentation describes the Confluence RESTconfluence.json file configuration such as the authentication methods, data crawled and retrieved, pagination information, variables used, and endpoints used. Terminology is also provided as a reference.
The list of data the REST connector crawls using the Confluence REST configuration is:
- Spaces
- Pages such as Wiki pages
- Blog posts
- Page and blog post comments
Authentication methods
The Confluence REST configuration supports:- Basic Authentication using the username and password from an Atlassian account. For more information, see Basic auth for REST APIs.
- API Token. For information about how to create a new API token, see API Tokens.
Supported crawl options
The Confluence REST configuration supports the following crawl options:- Full crawl
- Recrawl that relies on the
strayContentDeletionparameter
Pagination information
This recipe uses pagination per request. The connector requests the next page’s URL, and Confluence returns the next page’s URL in the response under the_links.next path. When _links.next is not provided in the response, the connector has found no more pages, and the pagination stops.
The following code sample shows an example response snippet with a link to a next page:
Configure the pagination by next page URL property
Next Page URL Key: _links.next, where_links.nextis the key of the response that contains the next page URL
Configure query parameters
limit=50, where 50 is the number of items per page
Variables used
The Confluence REST configuration uses one variable.${LW_PARENT_DATA_KEY} is used to access a value of the response from the main request to use in an additional request. The Confluence use case indicates this variable can be added to the URL to execute a GET request to retrieve comments for blog posts and pages.
Endpoints to configure with the Confluence REST connector
The following table describes the Confluence REST connector endpoints. Each request is configured under the property List of Requests Configuration, or underrequestConfigurations in the Confluence recipe.
Notes
-
The requests are linked hierarchically using the properties
ObjectTypeandParentObjectType.- This hierarchy maintains the parent-child relationships between different levels of objects. For instance, a Page is a Space-Child, a Comment is a Page-Child, and a Comment-Reply is a Comment-Child.
- When objects are indexed, the field
_lw_rest_parent_object_sskeeps the list of parents related to an object. For example, for a page, indexes_lw_rest_parent_object_ss: ["/spaces/TestSpaceName", "/spaces/TestSpace/pages/<pageId>/TestPageName"], where<pageId>is a numeric value.
-
With Confluence API v2 endpoints, different requests are needed to retrieve the footer comments and inline comments from pages and blog posts. To maintain the relationship between the comments, replies, and their parents (pages, blog posts, and spaces), there are eight different requests configurations.
- To retrieve page comments:
COMMENT_FOOTER_PAGEandCOMMENT_INLINE_PAGE. - To retrieve replies of comments:
COMMENT_REPLY_FOOTER_PAGE, andCOMMENT_REPLY_INLINE_PAGE. - To retrieve blog comments:
COMMENT_FOOTER_BLOGandCOMMENT_INLINE_BLOG. - To retrieve replies of blog comments:
COMMENT_REPLY_FOOTER_BLOG, andCOMMENT_REPLY_INLINE_BLOG. - When comments are indexed, the field contains:
_lw_rest_parent_object_ss: ["/spaces/TestSpaceName", "/spaces/TestSpace/pages/<pageId>/TestPageName", "<commentId>"]. - When replies are indexed, the field contains:
_lw_rest_parent_object_ss: ["/spaces/TestSpaceName", "/spaces/TestSpace/pages/<pageId>/TestPageName", "<commentId>", "<commentReplyId>"], where<pageId>,<commentId>and<commentReplyId>are numeric values.
- To retrieve page comments:
Response parsing configuration
Per request, configure the Response Handling property to specify how to parse the response. This field isresponseConfiguration in the JSON recipe.
The Confluence recipe does not use binary parsing.
Plugin parsing
- This parsing happens by default. The responses are parsed as a JSON object structure using JsonPath.
- Plugin parsing applies to all the requests listed in the Endpoints configuration table.
- The properties
Response Handling -> Data ID, Data Pathare configured to extract certain values from the objects parsed. - The properties
Response Handling -> Parent Data Keyare configured to extract the ‘id’ of the parent object.
Terminology
The following terms are provided as a reference.Recipe
Replace the following values in the recipe:pipelinewith your Fusion pipelinecollectionwith your Fusion collectionidwith the name of a Fusion datasource if you want to use a different name than the one providedserviceURLwith your Confluence URLpasswordwith your Confluence password or API tokenuserwith your Confluence email address