Partition (Kafka)
Also called: Consumer group, Offset
One ordered, append-only log inside a Kafka topic. Messages with the same key go to the same partition, and each partition is read by one consumer per group.
Definition
A topic is split into partitions so it can be stored on several brokers and read in parallel. The producer hashes the message key to pick a partition, so all events for one key stay in order. Within a consumer group, each partition is assigned to exactly one consumer, which makes the partition count the ceiling on parallelism. Adding partitions later changes which partition a key maps to.
In practice
An orders topic with 12 partitions, keyed by order id, can be consumed by up to 12 warehouse instances in parallel.