Object Storage

Giriş

Açıklaması şöyle

The year 2023 witnessed the emergence of building Kafka on object storage. At least five vendors have introduced a solution like that since 2023. We had WarpStream and AutoMQ in 2023, Confluent Freight Clusters, Bufstream, or Redpanda Cloud Topics in 2024.

Açıklaması şöyle

Each vendor did this with their approach. At the high level, these systems try to speak the Kafka protocol and store complete data in the object storage. 

- Bufstream and Warpstream rewrite the Kafka protocol from scratch. 

- AutoMQ takes a very different approach, leveraging Kafka’s code for the protocol layer to ensure 100% Kafka compatibility while re-implementing the storage layer so the broker can write data to the object storage without sacrificing the latency due to the introduction of the write-ahead log.