Welcome to Spring Boot + Apache Kafka Tutorial series. In this lecture, we will take a look at the Apache Kafka overview.
Lecture #1 - Apache Kafka Overview
Transcript:
Hey there everyone Ramesh here, back again with a new brand series or course on Spring boot and Apache Kafka.
Well, this course is going to be a practical course which means I'm not going to cover a lot of theory parts in this course. Though I'm going to go cover the core concepts and the terminologies of Apache Kafka but I'm not going to, you know, deep dive into the theoretical part of Apache Kafka so you can go ahead and read the official documentation of Apache Kafka.
So I am going to only focus on focus on the practical part of this course. All right. So in this course, you will learn how to use Apache Kafka in a Spring Boot application. Well, this course is also useful to you if you are using event architecture in your microservices. Let's say you are using Apache Kafka for the communication between multiple microservices as a messaging system then you can go ahead and take this course so that you will understand the fundamentals of Apache Kafka, and how you can use Apache Kafka in a Spring Boot project. All right. So if you look at this architecture here, microservices one basically produce the messages and it will send that message to the Kafka broker and microservices two basically consume that message from the Kafka broker.
Well, there can be a lot of microservices that can consume a message from the Kafka broker. All right.
Well, in this course, you will learn how to create a Kafka Producer to produce a message and send it to the Kafka broker and how to create a Kafka consumer to consume the message from the Kafka broker. All right.
You will learn a lot of stuff in this course. Well, let's quickly take a look into what is Apache Kafka?
Well, go to the browser and just type Apache Kafka and go to the official website of Apache Kafka. Apache Kafka is nothing but an open-source distributed, even stream platform used by thousands of companies for high-performance data, pipeline streaming analytics, data integration, and mission-critical applications.
Well, Apache Kafka was initially built by, you know, LinkedIn, but later on it wasn't managed by the Apache Foundation. Well, in order to understand more about Apache Kafka, let's take a look at how apache Kafka looks in a nutshell.
As we can see this is the Apache Kafka ecosystem. Well, as we know, that Apache Kafka distributed system so it acts as a Kafka cluster. Well, a Kafka cluster is anything but a cluster it has one or more Kafka blockers. You can see in the diagram we have broker one, broker two, and broker three. Well, whenever you use Kafka in a production, make sure that you at least use three brokers in a Kafka cluster. All right. So here you can see the producer and consumer. Well, Producer is nothing but an application that produces the messages and sends that message to the Kafka broker.
Well, the producer can produce any kind of message like a message can be an event, the message can be a stream of records, stream of data, the event can be Avro, plaintext, stream, JSON Okay. So Producer basically produces the message and sends that message to the Kafka cluster. All right. And then here you can see the consumer. Well, consumers will basically consume those messages from the Kafka broker. All right. So there is one more Kafka term that is topic. So we basically create a topic in a Kafka cluster so that a consumer can able to subscribe to that particular topic.
Well, here you can see one more important service which is Zookeeper. Well, the zookeeper plays a very important role here. So the Zookeeper basically manages the state of all these Kafka brokers Okay. So Cluster can contain a lot of Kafka brokers. Right. So there should be some way to manage the state of all these brokers in a cluster, right? so that is what the zookeeper does. So Zookeeper basically maintains the state of all the Kafka brokers in the Kafka cluster, as well as Zookeeper, maintains the configuration of all the topics, producers, and consumers. All right.
So Zookeeper basically plays a very important role in the Kafka cluster. Well, this is how the Kafka ecosystem looks at a high level. All right. So Apache Kafka is a distributed streaming platform. It means Apache Kafka can act as a cluster so that it can know contains one or more capture brokers and Apache Kafka is fault tolerance. Fault tolerance means if one node or broker goes down, then it can have the capability to manage the other nodes as well because the data is distributed among different Kafka brokers. Right. If one node or Apache Kafka goes down, then it has the capability to manage the entire cluster. Right.
Well, the data can be stored in Apache Kafka broker at any duration so that the consumers can be able to consume the history data from the Kafka broker. Okay, so these are the few important features of Apache Kafka. Well, let's take a look into some of the use cases of Apache Kafka. So here you can see these are the few popular use cases for Apache Kafka. Well, messaging.
Well, as we know that there are some traditional messaging systems like ActiveMQ or RabbitMQ so this Apache Kafka works well as a replacement for traditional message brokers. All right. And website activity, tracking metrics, log aggregation, stream processing, event sourcing, and commit logs so are the few popular use cases for Apache Kafka. Well, if you want to know more about Apache Kafka, then you can go ahead and read this introduction page. So this page will give you a brief idea about what is Apache Kafka and where you can use Apache Kafka. All right.
In the next lecture, I'm going to cover the important key technologies and the core concepts of Apache Kafka so that it will be easy for you to understand whenever we use Apache Kafka in a Spring Boot application. All right, I will see you in the next lecture.
❮ Previous Lecture Next Lecture ❯
Comments
Post a Comment
Leave Comment