Spring Kafka (or Spring for Apache Kafka) is quite cool library built on top of Apache’s kafka-clients library. For example if you want to setup a producer it requires you to add very little code and create proper configuration. All the magic is happening under the hood.

However documentation isn’t the best one. You cannot find much about configuration properties in Spring Kafka Docs. Fortunately there are “Common application properties” docs, when in “Integration properties” section you can find extensive information about what properties to use to configure Spring Kafka. However it is not perfect either.

I have been struggling recently to make Spring Kafka work well with SSL certificates. Here are some details, what to avoid, and what to use to make producer code work.

Here is simple class which responsibility is to produce messages to Kafka (from Spring Kafka examples). It has all the code that is needed for producer to work:

 

@RestController
public class Controller {

	@Autowired
	private KafkaTemplate<Object, Object> template;

	@PostMapping(path = "/send/foo/{what}")
	public void sendFoo(@PathVariable String what) {
		this.template.send("topic1", new Foo1(what));
	}

}

When you want to create producer which uses more than default values you need to use following settings in applicaiton.yml:

spring:
  kafka:
    producer:
      bootstrap-servers: localhost:9092
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer

When it comes to work with secured cluster things get more tricky. Especially that configuration documentation seems to be wrong with security.protocol entry (spring.kafka.producer.security.protocol) which simply doesn’t work and doesn’t set security.protocol=SSL in producer configuration and spring.kafka.security.protocol doesn’t work either.

The solution was to use spring.kafka.properties.secrity.protocol (properties.* is described as “additional configuration”).

spring:
  kafka:
    properties:
      security.protocol: SSL
    ssl:
      key-password: pass
      keystore-location: classpath:resources/kafka.client.keystore.jks
      keystore-password: pass
      truststore-location: classpath:resources/kafka.client.truststore.jks
      truststore-password: pass
    producer:
      bootstrap-servers: localhost:9092
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer
      client-id: your-client-name

Hope that helped!