Kafka Connect JdbcSourceConnector mode=timestamp+incrementing

Giriş

Açıklaması şöyle

 - timestamp+incrementing – considered to be the most robust and accurate mode, since it combines 2 modes mentioned above, and having both the timestamp column and id column in place allows us to identify both the new and updated rows uniquely

Açıklaması şöyle

use two columns, a timestamp column that detects new and modified rows and a strictly incrementing column which provides a globally unique ID for updates so each row can be assigned a unique stream offset.

üretilen SQL şuna benzer

In timestamp+incrementing mode, the following suffix is appended by Kafka-jdbc-connect with four parametrized values

    WHERE [timstamp-column] < ?

    AND (

    ([timstamp-column] = ? AND [increment-column] > ?) 

    OR [timstamp-column] > '?

    ) 

    ORDER BY "last_updated","increment-column" ASC

Örnek

Şöyle yaparız

mode":"timestamp+incrementing",
"query":"SELECT * FROM (
  SELECT ID_COL, TIMESTAMP_COL, COL1, COL2 
  FROM TABLE_A 
  INNER JOIN TABLE_B ON PK=FK 
  WHERE COL1='FOO')",
"timestamp.column.name":"TIMESTAMP_COL",
"incrementing.column.name":"ID_COL",