AEM V2

​

Configure AEM Datasource

1. In Fusion, under Indexing > Datasources, click Add, then select AEM
2. Enter a Configuration ID
3. Enter the AEM URL (the URL used to access the AEM Admin UI) as well as the AEM username and password used to authenticate access to the QueryBuilder JSON Servlet.

4. Go to AEM to log in and access CRXDE Lite. In the CRXDE Lite UI, select a path to crawl. Enter this path into Fusion. Click Add to crawl multiple paths.

5. Optional: To exclude paths from the crawl, in Fusion, enter a Java Regular Expression (regex) that represents paths to exclude in the indexed content.

6. In Fusion, enter the AEM type to crawl. In the CRXDE Lite UI, this is the jcr:primaryType. In this example, the AEM connector is configured to crawl the AEM Type cq:Page, which represents web content pages.

7. To index assets with a particular file extension, locate a file type in CRXDE Lite and enter the value of the jcr:primaryType into Fusion. In this example, the value of NY_FairHealth.pdf is dam:Asset.

8. You can choose which content properties to include and exclude from the index. These parameter values are represented by Java regex. For example, to only include properties that start with “jcr” enter jcr:(.*).
9. In Fusion, click Save when you’re done configuring the AEM datasource.

​

Configuration settings

​

Field data population

​

Supported authorization options

curl -H 'Authorization: Bearer ACCESS_TOKEN' http://HOST:PORT/content/COMPANY/us/en/community/messaging.html

• From the datasource configuration
• From AEM server using Refresh Token
• From AEM server using JWT token

​

Getting Access Token

1. Open this URL in a browser:
   Copy

   http://HOST:PORT/oauth/authorize?response_type=code&client_id=placeholderClientId&client_secret=CLIENT_SECRET&username=admin&password=PASSWORD&scope=offline_access&redirect_uri=REDIRECT_URI
   

2. You are redirected to login page (if you are not logged in):
3. Logging in redirects you to confirm the authorization. Click Yes, I authorize this request.
4. You are redirected to the URL provided in redirect_uri with parameter code: REDIRECT_URI?code=AUTHORIZATION_CODE.
5. Copy the value of the code parameter. This is your Authorization Code.
6. Execute the request to get Access Token:
   Copy

   curl --location --request POST 'http://HOST:PORT/oauth/token?code=AUTHORIZATION_CODE&client_id=CLIENT_ID&client_secret=CLIENT_SECRET&grant_type=authorization_code&redirect_uri=REDIRECT_URI' --header 'Content-Type: application/x-www-form-urlencoded' --header 'Accept: application/json'
   

   Example:
   Copy

   curl --location --request POST 'http://34.71.168.50:4502/oauth/token?code=AUTHORIZATION_CODE&client_id=CLIENT_ID&client_secret=CLIENT_SECRET&grant_type=authorization_code&redirect_uri=http://localhost:8080/test' --header 'Content-Type: application/x-www-form-urlencoded' --header 'Accept: application/json'
   

   Response:
   Copy

   "access_token":"{placeholderAccessToken}","refresh_token":"{placeholderRefreshToken}","expires_in":3600
   

​

Getting Refresh Token

1. You must include offline_access in the scope list.
2. You must revoke all the previously obtained token. It can be done by clicking Revoke All Tokens.

​

Getting JWT Bearer Token

1. Download Private Key from the AEM Oauth client section. You should have downloaded file store.p12.
2. Run:
   Copy

   openssl pkcs12 -in store.p12 -out store.crt.pem -clcerts -nokeys
   

   When asked about password type notasecret. You should have generated file named store.crt.pem.
3. Run:
   Copy

   openssl pkcs12 -in store.p12 -passin pass:notasecret -nocerts -nodes -out store.private.key.txt
   

   You should have generated file named store.private.key.txt.
4. Create JWT token with the below payload and encrypt it with the private key using RS256:
   Copy

   {
   "aud": "http://HOST:PORT/oauth/token",
   "iss": "CLIENT_ID",
   "sub": "USERNAME",
   "exp": <Current time in milliseconds+expiry>,
   "iat": <Current time in milliseconds>,
   "scope": "SCOPE",
   "cty": "code"
   }
   

   For example, install pyjwt with pip install pyjwt to encrypt with this python script:
   Copy

   import jwt
   payload_data = {
   "aud": "http://34.71.168.50:4502/oauth/token",
   "iss": "dp0dtqd9lqpcntvb6t12hrscpa-z1hqkpdg",
   "sub": "admin",
   "exp": 1697840880541,
   "iat": 1697740880541,
   "scope": "offline_access",
   "cty": "code"
   }
   private_key = open('store.private.key.txt', 'r').read()
   token = jwt.encode(
       payload=payload_data,
       key=private_key,
       algorithm='RS256'
   )
   print(token)
   

​

Prerequisites

• The plugin zip file and the connector-plugin-standalone JAR.
• A configured connector backend gRPC endpoint.
• Username and password of a user with a remote-connectors or admin role.
• If the host where the remote connector is running is not configured to trust the server’s TLS certificate, you must configure the file path of the trust certificate collection.

​

Connector compatibility

​

System requirements

• (Fusion 5.9.0-5.9.10) JVM version 11
• (Fusion 5.9.11) JVM version 17
• Minimum of 2 CPUs
• 4GB Memory

​

Enable backend ingress

ingress:
  enabled: false
  pathtype: "Prefix"
  path: "/"
  #host: "ingress.example.com"
  ingressClassName: "nginx"   # Fusion 5.9.6 only
  tls:
    enabled: false
    certificateArn: ""
    # Enable the annotations field to override the default annotations
    #annotations: ""

• Set enabled to true to enable the backend ingress.
• Set pathtype to Prefix or Exact.
• Set path to the path where the backend will be available.
• Set host to the host where the backend will be available.
• In Fusion 5.9.6 only, you can set ingressClassName to one of the following:
  • nginx for Nginx Ingress Controller
  • alb for AWS Application Load Balancer (ALB)
• Configure TLS and certificates according to your CA’s procedures and policies.
  TLS must be enabled in order to use AWS ALB for ingress.

​

Connector configuration example

kafka-bridge:
  target: mynamespace-connectors-backend.lucidworkstest.com:443 # mandatory
  plain-text: false # optional, false by default.  
    proxy-server: # optional - needed when a forward proxy server is used to provide outbound access to the standalone connector
    host: host
    port: some-port
    user: user # optional
    password: password # optional
  trust: # optional - needed when the client's system doesn't trust the server's certificate
    cert-collection-filepath: path1

proxy: # mandatory fusion-proxy
  user: admin
  password: password123
  url: https://fusiontest.com/ # needed only when the connector plugin requires blob store access

plugin: # mandatory
  path: ./fs.zip
  type: #optional - the suffix is added to the connector id
    suffix: remote

​

Minimal example

kafka-bridge:
  target: mynamespace-connectors-backend.lucidworkstest.com:443

proxy:
  user: admin
  password: "password123"

plugin:
  path: ./testplugin.zip

​

Logback XML configuration file example

<configuration>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="com.lucidworks.logging.logback.classic.LucidworksPatternLayoutEncoder">
            <pattern>%d - %-5p [%t:%C{3.}@%L] - %m{nolookups}%n</pattern>
            <charset>utf8</charset>
        </encoder>
    </appender>

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOGDIR:-.}/connector.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <!-- rollover daily -->
            <fileNamePattern>${LOGDIR:-.}/connector-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
            <maxFileSize>50MB</maxFileSize>
            <totalSizeCap>10GB</totalSizeCap>
        </rollingPolicy>
        <encoder class="com.lucidworks.logging.logback.classic.LucidworksPatternLayoutEncoder">
            <pattern>%d - %-5p [%t:%C{3.}@%L] - %m{nolookups}%n</pattern>
            <charset>utf8</charset>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="FILE"/>
    </root>
</configuration>

​

Run the remote connector

java [-Dlogging.config=[LOGBACK_XML_FILE]] \
  -jar connector-plugin-client-standalone.jar [YAML_CONFIG_FILE]

​

Test communication

java -Dstandalone.connector.connectivity.test=true -jar connector-plugin-client-standalone.jar [YAML_CONFIG_FILE]

​

Encryption

​

Egress and proxy server configuration

• Host. The hosts where the proxy server is running.
• Port. The port the proxy server is listening to for communication requests.
• Credentials. Optional proxy server user and password.

​

Password encryption

1. Enter a user name and password in the connector configuration YAML.
2. Run the standalone JAR with this property:
   Copy

   -Dstandalone.connector.encrypt.password=true
   

3. Retrieve the encrypted passwords from the log that is created.
4. Replace the clear password in the configuration YAML with the encrypted password.

​

Connector restart (5.7 and earlier)

​

Recoverable bridge (5.8 and later)

​

Job expiration duration (5.9.5 only)

​

Use the remote connector

​

Enable asynchronous parsing (5.9 and later)

​

Supported authorization options

curl -H 'Authorization: Bearer {placeholderAccessToken}' http://HOST:PORT/content/COMPANY/us/en/community/messaging.html

• From the datasource configuration
• From AEM server using Refresh Token
• From AEM server using JWT token

​

Getting Access Token

1. Open this URL in a browser:
   Copy

   http://HOST:PORT/oauth/authorize?response_type=code&client_id=placeholderClientId&client_secret=placeholderClientSecret&username=admin&password=PASSWORD&scope=offline_access&redirect_uri=REDIRECT_URI
   

2. You are redirected to login page (if you are not logged in):
3. Logging in redirects you to confirm the authorization. Click Yes, I authorize this request.
4. You are redirected to the URL provided in redirect_uri with parameter code: <REDIRECT_URI>?code=<AUTHORIZATION_CODE>.
5. Copy the value of the code parameter. This is your Authorization Code.
6. Execute the request to get Access Token:
   Copy

   curl --location --request POST 'http://HOST:PORT/oauth/token?code={placeholderAuthorizationCode}&client_id={placeholderClientId}&client_secret={placeholderClientSecret}&grant_type=authorization_code&redirect_uri=<REDIRECT_URI>' --header 'Content-Type: application/x-www-form-urlencoded' --header 'Accept: application/json'
   

   Example:
   Copy

   curl --location --request POST 'http://34.71.168.50:4502/oauth/token?code={placeholderAuthorizationCode}&client_id={placeholderClientId}&client_secret={placeholderClientSecret}&grant_type=authorization_code&redirect_uri=http://localhost:8080/test' --header 'Content-Type: application/x-www-form-urlencoded' --header 'Accept: application/json'
   

   Copy

   "access_token":"{placeholderAccessToken}","refresh_token":"{placeholderRefreshToken}","expires_in":3600
   

​

Getting Refresh Token

1. You must include offline_access in the scope list.
2. You must revoke all the previously obtained token. It can be done by clicking Revoke All Tokens.

​

Getting JWT Bearer Token

1. Download Private Key from the AEM Oauth client section. You should have downloaded file store.p12.
2. Run:
   Copy

   openssl pkcs12 -in store.p12 -out store.crt.pem -clcerts -nokeys
   

   When asked about password type notasecret. You should have generated file named store.crt.pem.
3. Run
   Copy

   openssl pkcs12 -in store.p12 -passin pass:notasecret -nocerts -nodes -out store.private.key.txt
   

   You should have generated file named store.private.key.txt.
4. Create JWT token with the below payload and encrypt it with the private key using RS256:
   Copy

   {
    "aud": "http://<HOST>:<PORT>/oauth/token",
    "iss": "<client id>",
    "sub": "<user name>",
    "exp": <Current time in milliseconds+expiry>,
    "iat": <Current time in milliseconds>,
    "scope": "<scope>",
    "cty": "code"
   }
   

   For example, install pyjwt to use this python script:
   Copy

   pip install pyjwt
   

   Copy

   import jwt  
   
   payload_data = {
    "aud": "http://34.71.168.50:4502/oauth/token",
    "iss": "dp0dtqd9lqpcntvb6t12hrscpa-z1hqkpdg",
    "sub": "admin",
    "exp": 1697840880541,
    "iat": 1697740880541,
    "scope": "offline_access",
    "cty": "code"
   }  
   
   private_key = open('store.private.key.txt', 'r').read()  
   
   token = jwt.encode(
      payload=payload_data,
      key=private_key,
      algorithm='RS256'
   )  
   
   print(token)