Spring Boot and Kafka Integration: Building a Real-Time Messaging Application

📘 Premium Read: Access my best content on Medium member-only articles — deep dives into Java, Spring Boot, Microservices, backend architecture, interview preparation, career advice, and industry-standard best practices.

🎓 Top 15 Udemy Courses (80-90% Discount): My Udemy Courses - Ramesh Fadatare — All my Udemy courses are real-time and project oriented courses.

▶️ Subscribe to My YouTube Channel (176K+ subscribers): Java Guides on YouTube

▶️ For AI, ChatGPT, Web, Tech, and Generative AI, subscribe to another channel: Ramesh Fadatare on YouTube

Integrating Apache Kafka with Spring Boot offers a powerful combination for creating efficient, scalable real-time messaging applications. In this blog post, we'll walk through the steps to set up a simple messaging system using Spring Boot and Kafka, highlighting the usage of a Kafka producer and consumer service.

Prerequisites 

Before we start, ensure that you have the following: 

• Java and Spring Boot set up in your development environment. 
• Apache Kafka is installed and running on your machine or a server. 
• Basic understanding of Spring Boot and RESTful services.

Step 1: Setting Up Dependencies 

Include the following Maven dependencies in your pom.xml:

<dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
</dependency>
   
 <dependency>
      <groupId>org.springframework.kafka</groupId>
      <artifactId>spring-kafka</artifactId>
</dependency>

Step 2: Kafka Configuration 

Configure the Kafka properties in your application.properties file:

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=myGroup
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer

Step 3: Creating Kafka Producer Service

Now, let's set up the KafkaProducerService:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

@Service
public class KafkaProducerService {

    private final KafkaTemplate<String, String> kafkaTemplate;

    @Autowired
    public KafkaProducerService(KafkaTemplate<String, String> kafkaTemplate) {
        this.kafkaTemplate = kafkaTemplate;
    }

    public void sendMessage(String topic, String message) {
        kafkaTemplate.send(topic, message);
    }
}

Step 4: Implementing a REST Controller

To send messages, implement a REST controller that uses KafkaProducerService:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class MessageController {

    private final KafkaProducerService producerService;

    @Autowired
    public MessageController(KafkaProducerService producerService) {
        this.producerService = producerService;
    }

    @PostMapping("/send")
    public String sendMessage(@RequestParam String message) {
        producerService.sendMessage("myTopic", message);
        return "Message sent to Kafka topic";
    }
}

Step 5: Creating Kafka Consumer Service

Next, create a Kafka consumer service to receive messages:

@Service
public class KafkaConsumerService {

    @KafkaListener(topics = "myTopic", groupId = "myGroup")
    public void listen(String message) {
        System.out.println("Received message in group 'myGroup': " + message);
    }
}

Step 6: Testing the Setup

To test, send a message using a tool like Postman or a curl command:

curl -X POST "http://localhost:8080/send?message=Hello Kafka!"

The consumer service should log:

Received message in group 'myGroup': Hello Kafka!

Conclusion

This basic setup illustrates how to create a real-time messaging application using Spring Boot and Kafka. This integration allows you to leverage Kafka’s capabilities for handling large-scale, real-time data while benefiting from Spring Boot's ease of development and configuration. Remember to adapt and expand upon this foundation to suit your application's specific needs and Kafka configurations.