Skip to content

demo application for testing a Kafka consumer with Avro message schema with Testcontainers using Kafka with embedded Zookeeper and no Schema Registry container

Notifications You must be signed in to change notification settings

kreuzwerker/kafka-consumer-testing

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

kafka-consumer-testing

Let's assume you have a Spring Boot microservice that listens to incoming events from Kafka, transforms them into your own business objects, writes them into a PostgreSQL database and provides them via REST interface to your frontend. The overall infrastructure provides Avro messages and the Confluent schema registry.

This sample project uses Apache Maven, with the avro-maven-plugin to download the schema files and generate the sources, but of course there are plugins for Gradle too.

Now you want to test that your Kafka consumer reads the events, transforms them into your database entities, and saves them.

When you check the internet for testing in the context of Spring Boot Kafka consumer and Avro schema, you find quite a few variants. From using the MockSchemaRegistryClient to writing your own custom Avro de-/serializers, to setting up a Testcontainers ecosystem with a Kafka, a Zookeeper and a Confluent Schema Registry, or to using the EmbeddedKafka provided by Spring in the spring-kafka-test dependency.

In this case, we decided against the use of the EmbeddedKafka and against the full Testcontainers setup and went for a setup with a smaller number of Testcontainers.

More details can be found in this blogpost: https://kreuzwerker.de/post/testing-a-kafka-consumer-with-avro-schema-messages-in-your-spring-boot

About

demo application for testing a Kafka consumer with Avro message schema with Testcontainers using Kafka with embedded Zookeeper and no Schema Registry container

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages