Kafka Connect MongoSourceConnector

CDC Çıktısı

- fullDocument

- updateDescription

- fullDocumentBeforeChange

alanlarının hepsini CDC alınca alınca çıktı 16 MB'den büyük olabiliyor

CDC Çıktısı - updateDescription Alanı

Açıklaması şöyle

“updateDescription” can be excluded by using “$unset”, 

Bağlantı Bilgileri

Örnek

Şöyle yaparız

{
  "name": "mongo-simple-source",
  "config": {
    "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector",
    "connection.uri": "mongodb://mongo1",
    "database": "Tutorial1",
    "collection": "orders"
  }
}

change.stream Alanı

Açıklaması şöyle

The change.stream.full.document.before.change configuration property in the mongo-simple-source connector controls whether the connector will include the full document as it existed before the change in the change stream event. The possible values for this property are:

- required: The connector will always include the full document before the change.

- whenAvailable: The connector will only include the full document before the change if it is available.

- ``: The connector will not include the full document before the change.

The default value for this property is whenAvailable. This means that the connector will only include the full document before the change if it is available. If the full document is not available, the connector will only include the changes that were made to the document

Örnek

Şöyle yaparız

{
  "name": "mongo-simple-source",
  "config": {
    "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector",
    "connection.uri": "mongodb://mongo1",
    "database": "Tutorial1",
    "change.stream.full.document.before.change": "whenAvailable",
    "name": "mongo-simple-source"
  },
  "tasks": [],
  "type": "source"
}

pipeline Tag Alanı

Örnek

Şöyle yaparız

{
  "name": "mongo-simple-source",
  "config": {
    "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector",
    "connection.uri": "mongodb://mongo1",
    "database": "Tutorial1",
    "change.stream.full.document.before.change": "whenAvailable",
    "pipeline": "[ {\"\$project\": {\"_id\":1,\"operationType\":1,\"fullDocument\":1,\"ns\":1,\"documentKey\":1,\"clusterTime\":1,\"fullDocumentBeforeChange.org_id\":1} } ]"
  }
}