Skip to main content
  • Latest version: v1.4.0
  • Compatible with Fusion version: 5.9.4 and later
Connector flow The AEM V2 connector supports the following:
  • Full crawling and recrawling of pages and assets in Adobe Experience Manager
  • Basic authentication
    In Fusion 5, the username and password fields have moved under Authentication Settings > Login Settings.
  • OAuth authentication
  • Security trimming to filter results based on user permissions
    • Filter document crawling by including and excluding paths when setting up the connector
  • Specific wait time between fetch requests to throttle crawls, if necessary
  • Optional crawling of child paths
  • RSA PEM private key authentication via Adobe Identity Management Service (IMS), with automatic token renewal
The AEM V2 connector is officially tested with AEM 6.5. While this connector has not been officially tested with AEM as a Cloud Service, page and metadata crawling over OAuth 2.0 (Adobe IMS) generally works there. Binary and attachment crawling aren’t supported for AEM Cloud Service because the /crx/de download path the connector relies on isn’t available outside the local AEM SDK. Security trimming has not been validated against Cloud Service’s IMS-backed user/group model.
This document explains how to configure an AEM V2 connector to crawl data in Adobe Experience Manager. This connector is compatible with Fusion 5.5.1 and later.

Configure AEM Datasource

  1. In Fusion, under Indexing > Datasources, click Add, then select AEM
  2. Enter a Configuration ID
  3. Enter the AEM URL (the URL used to access the AEM Admin UI) as well as the AEM username and password used to authenticate access to the QueryBuilder JSON Servlet.
AEM authentication
  1. Go to AEM to log in and access CRXDE Lite. In the CRXDE Lite UI, select a path to crawl. Enter this path into Fusion. Click Add to crawl multiple paths.
CRXDE pathAEM paths to crawl
  1. Optional: To exclude paths from the crawl, in Fusion, enter a Java Regular Expression (regex) that represents paths to exclude in the indexed content.
paths to exclude
  1. In Fusion, enter the AEM type to crawl. In the CRXDE Lite UI, this is the jcr:primaryType. In this example, the AEM connector is configured to crawl the AEM Type cq:Page, which represents web content pages.
CRXDE typeAEM type
  1. To index assets with a particular file extension, locate a file type in CRXDE Lite and enter the value of the jcr:primaryType into Fusion. In this example, the value of NY_FairHealth.pdf is dam:Asset.
AEM attachments
  1. You can choose which content properties to include and exclude from the index. These parameter values are represented by Java regex. For example, to only include properties that start with “jcr” enter jcr:(.*).
  2. In Fusion, click Save when you’re done configuring the AEM datasource.

Configuration settings

Field data population

There are multiple sources where AEM data is indexed. The /bin/querybuilder.json endpoint data is mandatory and must exist in order for a document to be indexed.Note the list of fields that can appear in an indexed document:
Check for duplicate data when crawling child paths. For example, if the connector indexes both cq:Page and cq:PageContent then the results could include duplicated data.
The v1.3.0 version of this connector is only compatible with Fusion 5.9.4 and later when using security trimming. The v1.3.0 connector version uses Graph Security Trimming and not regular security trimming. It is imperative to treat this as a new connector, as configurations do not transfer over due to disparities between newer versions and previous ones. A full crawl is mandatory.

Prerequisites

Perform these prerequisites to ensure the connector can reliably access, crawl, and index your data. Proper setup helps avoid configuration or permission errors, so use the following guidelines to keep your content available for discovery and search in Fusion.

AEM instance

  • AEM Author instance must be reachable over HTTP/HTTPS.
  • The QueryBuilder JSON API must be enabled. It’s typically located at /bin/querybuilder.json.
  • The JCR Download endpoints must be accessible, such as /crx/de/download.jsp or direct node paths.

Configure an AEM service account

Create a service account with the following:
  • Read access to content paths being crawled
  • Access to user and group APIs if using security trimming
  • Permissions to access the following:
    • Page metadata, such as cq:Page or jcr:content
    • Binary attachments such as PDFs and DOCX files
    • JCR nodes and properties
    • If using group mapping, access to /libs/cq/security/userinfo.json or equivalent endpoints

Content Paths

  • You must define one or more JCR root paths to crawl, such as /content/SITE_NAME/en.
  • Optionally provide:
    • Exclude path regexes to filter out subtrees
    • Attachment extension types
    • JCR property include/exclude filters

Authentication

Setting up the correct authentication according to your organization’s data governance policies helps keep sensitive data secure while allowing authorized indexing. The AEM V2 connector supports two modes of authenticating to your AEM instance: basic HTTP and OAuth. Fusion handles session management as needed, including cookie handling and token renewal. Basic HTTP authentication:
  • Provide a standard AEM username and password with read access to JCR paths.
  • If using security trimming, the AEM account also requires read access to the user and groups APIs.
OAuth 2.0 authentication:
  • Paste in an Access Token and optional Refresh Token.
  • If you do not have a pre-obtained token, the connector can fetch a token using JWT authentication.
  • For enterprise setups where tokens expire every 24 hours, use private key OAuth authentication via Adobe IMS. The connector automatically signs and renews tokens using a configured RSA private key, with no manual intervention required.
The AEM V2 Connector supports OAuth 2.0 authorization with JWT token along with private key OAuth 2.0 authorization.

Supported authorization options

Requests are authorized by including an Access Token in the Authorization header.Example:
There are three ways the connector can get Access Token:
  • From the datasource configuration
  • From AEM server using Refresh Token
  • From AEM server using JWT token
Request Access Token, Refresh Token, and JWT token manually and set them in the datasource configuration:AEM token requestOther settings: Client Id, Client Secret, and Redirect Uri can be found in the AEM admin page under Security > Oauth Clients:AEM client settings

Getting Access Token

  1. Open this URL in a browser:
  2. You are redirected to login page (if you are not logged in): AEM login screen
  3. Logging in redirects you to confirm the authorization. Click Yes, I authorize this request. AME login confirmation
  4. You are redirected to the URL provided in redirect_uri with parameter code: REDIRECT_URI?code=AUTHORIZATION_CODE.
  5. Copy the value of the code parameter. This is your Authorization Code.
  6. Execute the request to get Access Token:
    Example:
    Response:

Getting Refresh Token

To get Refresh Token, follow the same process for Access Token, but:
  1. You must include offline_access in the scope list.
  2. You must revoke all the previously obtained token. It can be done by clicking Revoke All Tokens.

Getting JWT Bearer Token

  1. Download Private Key from the AEM Oauth client section. You should have downloaded file store.p12.
  2. Run:
    When asked about password type notasecret. You should have generated file named store.crt.pem.
  3. Run:
    You should have generated file named store.private.key.txt.
  4. Create JWT token with the below payload and encrypt it with the private key using RS256:
    For example, install pyjwt with pip install pyjwt to encrypt with this python script:

Private key authentication (Adobe IMS)

Use private key authentication when your AEM setup uses Adobe Identity Management System (IMS) and tokens expire every 24 hours. The connector automatically signs its own JWT from a configured RSA private key and exchanges it for an access token at the Adobe IMS endpoint, refreshing automatically when tokens expire.Configure the following fields in your Fusion datasource:The connector:
  1. Builds and signs a JWT (RS256) using the private key and configured claims.
  2. Exchanges the JWT for an access token at https://ims-na1.adobelogin.com/ims/exchange/jwt/.
  3. Automatically refreshes the access token on expiry with no manual intervention.
Prefer storing the RSA private key in Fusion’s encrypted secrets or a Vault integration rather than pasting it directly into the datasource.

Security trimming

The AEM V2 connector supports graph security trimming. If you are using Fusion 5.18.0 or later, you can use separate collections to store your content and your ACLs. Use a separate sidecar collection if you need to manage ACL data independently of the rest of your content. To use a sidecar collection for your ACLs, you must create the ACL collection before configuring your datasource to read from this collection. To enable graph security trimming for your datasource:
  1. Navigate to Indexing > Datasources.
  2. Open your datasource.
  3. Under Graph Security Filtering Configuration, select Enable security trimming.
  4. If you are using a sidecar collection in Fusion 5.18.0 or later, set ACL Collection ID to your ACL collection name (for example, acl_collection).
  5. Save your changes.
After configuring your datasource, you must reference the ACL collection in the Graph Security Trimming stage.
If you are transitioning from using the same collection to creating a sidecar collection for ACLs, you must run a full recrawl of your datasource so the ACLs route to the sidecar collection. ACL documents stored in the content collection from before the migration remain in the content collection unless the collection is reindexed or manually cleaned.

Remote connectors

V2 connectors support running remotely in Fusion versions 5.7.1 and later. For more information, see Configure remote V2 connectors. Below is an example configuration showing how to specify the file system to index under the connector-plugins entry in your values.yaml file:
You may also need to specify the user that is authorized to access the file system, as in this example:

Configuration