Creating SSL certificates to test inter-broker communication locallly with Kafka

0

I am trying to set up SSL communication with interbroker and broker-client comms in kafka, hosted on windows. In production we have a number of already signed certificates for each server, however to get the POC up, I want to simulate this by creating my own certificates and signing them locally with a CA I create myself.

It seems the best way to test it is get it working with running two brokers locally, each with a valid signed certificate (will be the same cert as same machine) and confirm they talk ok. Repeat for client-server comms. Then try replacing one of the signed certs with an unsigned one - both on broker 2 and on the client and confirm it denies access.

Does this sound like a reasonable approach?

If so, I have a few questions

I am using the keytool command to create the initial certs. In fact my entire process is as follows

keytool -keystore server.keystore.jks -alias localhost -validity 10000 -genkey -keyalg RSA
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 10000 -CAcreateserial -passin pass:Kafkaflow1
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed

My questions are

1) After the first step to create the keystore it asks for a password which is fine. It then asks for first and last name, organizational unit, organization, locality, state and country code. This translates to

CN={}, OU={}, O={} 

etc. How important is this info, is it information only or does it have to exactly match something on my machine to work? With prod certificates they're already set up with various values but locally to just get it working can I enter anything here? The only certificate I can find in MMC is my local user one that has been set up, which has CN, OU and three DC's populated. But I can't see how the local user certificate would have anything to do with my generated one - so can I literally put anything in these values I want - just for test purposes?

2) Since I will be running both brokers, the producer and the consumer all on my machine, do I use the same key and trust store for all four processes?

3) Does setting the keystore for the consumer and the truststore for the broker suffice for authentication? Does my machine name enter into it anywhere? I saw there was mention of a param to keytool called SAN=DNS:???? but not sure if this is necessary for me

4) When configuring the broker, my server.properties are

listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093
ssl.client.auth=required
ssl.keystore.type=JKS
ssl.truststore.type=JKS
ssl.truststore.location=c:/kafka/kafka_2.12-2.1.1/kafka.server.truststore.jks
ssl.truststore.password=Kafkaflow1
ssl.keystore.location=c:/kafka/kafka_2.12-2.1.1/kafka.server.keystore.jks
ssl.keystore.password=Kafkaflow1
ssl.key.password=Kafkaflow1

The Apache docs at

https://kafka.apache.org/documentation/#security_ssl

Say to test the key and trust store set with

openssl s_client -debug -connect localhost:9093 -tls1

It says you should see the certificate output from this command, however when I do it I see

Loading 'screen' into random state - done CONNECTED(000001F4) write to 0x784740 [0x78eb30] (92 bytes => 92 (0x5C)) 0000 - 16 03 01 00 57 01 00 00-53 03 01 5c 87 7e 7d d2 ....W...S...~}. 0010 - b4 e4 43 e0 fd 8d a3 96-4b 15 75 7d b8 5c 6c 6a ..C.....K.u}.\lj 0020 - be b0 f8 f6 20 bf 52 d1-eb 3f 3c 00 00 26 00 39 .... .R..?<..&.9 0030 - 00 38 00 35 00 16 00 13-00 0a 00 33 00 32 00 2f .8.5.......3.2./ 0040 - 00 05 00 04 00 15 00 12-00 09 00 14 00 11 00 08 ................ 0050 - 00 06 00 03 01 00 00 04-00 23 .........# 005c - read from 0x784740 [0x78a320] (5 bytes => -1 (0xFFFFFFFF)) write:errno=10053

So not sure if it is a protocol issue or something in my setup I have done wrong.

Also when I try and fire up the server just locally without any consumer or producers, just starting the broker, it loads fine. But when I connect with my consumer/producer - it doesn't deny authorization even though I have set up no SSL on the consumer/producer. I had thought this would deny?

Any help would be greatly appreciated

java
ssl
apache-kafka
asked on Stack Overflow Mar 12, 2019 by NZJames • edited Mar 12, 2019 by NZJames

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0