-
Late-arriving data: when the event shows up after the report closed
Event time is not arrival time. How late data breaks closed reports and incremental loads, and the patterns that make it harmless: watermarking, lookback reprocessing, idempotent partitions, and versioned event logs.
streamingevent-timewatermarksidempotencydata-quality -
Effectively-once: making a Kafka → ClickHouse sink idempotent
Kafka only promises at-least-once. Here is how write-then-commit, source-timestamp versions, and ReplacingMergeTree make duplicates harmless — without claiming exactly-once.
clickhousekafkaidempotencyexactly-oncestreaming