heartin's blog

On a high level, Kafka provides following guarantees:

1. Messages sent by a producer to a particular topic partition will be appended in the order they are sent.

   • If a message M1 is sent by the same producer as a message M2, and M1 is sent first, then M1 will have a lower offset than M2 and appear earlier in the log.

We will write a simple consumer to receive messages from a Kafka broker using Java. We will be using the High level APIs here. We will see another example for low level APIs later.

Summary of Steps

1. Setting up the environment

2. Defining Properties

Consumers consume the messages published by Kafka producers.

Consumer API Basics

• The consumer subscribes to the message consumption from a specific topic on the Kafka broker. 

We will write a simple producer to send messages to a Kafka broker using Java.

This is a simple program without specifying any custom partitioning. We will also create another Java producer with custom partitioning in later notes.

Summary of Steps

1. Setting up the environment

Producers are processes that publish messages to Kafka.

Writing Producers basic concepts

• Producers can be implemented in different languages such as Java, C, and Python; we will however only look into Java for now. 

We have seen the components and types of Kafka clusters already. We will now quickly see how Kafka components work together in a single node single broker cluster.

Even though in production scenarios, it will be almost always installed in linux/unix based machines, we will install and test it in Windows for simplicity.

Apache Kafka is distributed. Kafka supports message partitioning over Kafka servers and distributing consumption over a cluster of consumer machines while maintaining per-partition ordering semantics. Kafka cluster can grow elastically and transparently without any downtime.

Apache Kafka is a publish-subscribe based distributed messaging system.

Kafka Basic Terminology

Following is a list of important terminology related to Apache Kafka.

• Broker

Apache Kafka is a publish-subscribe based distributed messaging system. 

Kafka is open source, distributed, partitioned, and replicated.

From the architecture perspective, Kafka is closer to traditional messaging systems such as ActiveMQ or RabitMQ.