Product Selector

Fusion 5.9
    Fusion 5.9

    Read from the MongoDB Oplog

    Retrieve data from a MongoDB instance.

    You can configure the Fusion MongoDB connector to read from the MongoDB oplog rather than from the entire MongoDB collection. In this mode, the connector crawls the full MongoDB collection, saves a checkpoint in ZooKeeper, then continues running indefinitely, grabbing updates from the oplog as they happen in real time. This way, the connector can delete documents that are deleted from MongoDB.

    If the connector stops for any reason, it stores a timestamp in ZooKeeper that shows what the latest update was. When the connector restarts, it continues reading from that checkpoint onward.

    To read from the MongoDB oplog:

    1. Make sure your connector authenticates to Mongo as a user with read oplog permissions. See Role-based access control in the MongoDB documentation.

    2. Make sure Process Oplog is selected in the Fusion MongoDB connector UI.