Apache Kafka Tutorial for Beginners | Part 2

Key Concepts in Kafka

What happens in Kafka is,

1. Topic is container where the messages are published.
2. Kafka needs small Zookeeper cluster to operate, i.e. coordination between various processes.
3. Message in kafka is a key, value pair with timestamp, value in key, value pair is what we are interested, value is data/message.
4. Kafka is usually deployed in cluster setup, which is collection of nodes running kafka, each node in the kafka cluster is called kafka server or kafka broker.
5. Kafka cluster is responsible for bringing-in the messages from producer to the consumer reliably.

1. A topic consists of partitions.
2. Partition: ordered + immutable sequence of messages that is continually appended to.
3. Offset: A sequence id given to messages as they arrive in a partition.
4. To locate a message, what you need is
   Topic name -> Partition Number -> Offset

1. Kafka consumers are typically part of a consumer group.
2. When multiple consumers are subscribed to a topic and belong to the same consumer group, each consumer in the group will receive messages from a different subset of the partitions in the topic.

Happy Learning !!!