Multiple CA Chains in Custom Client CA Cert Secret

[urton]

I've got a "fun" one. My client just transitioned to a complete new CA (from root on down) and now I have to support tls-external Kafka Users from 2 different CAs for the next year or so.

I tried concatenating both chains in the same file and creating a new secret with that file for ca.crt (I was 99.99% sure this would fail, but had to try...). Brokers came back up, but I got SSL transport errors from my clients.

I tried creating the CLUSTER-NAME-clients-ca-cert with ca.p12 and ca.password, per the documentaton , with all the CAs from both chains in the keystore, but the brokers failed to come up with the following error:
Preparing truststore for client authentication
Adding /opt/kafka/client-ca-certs/.crt to truststore /tmp/kafka/clients.truststore.p12 with alias *
keytool error: java.io.FileNotFoundException: /opt/kafka/client-ca-certs/.crt (No such file or directory)

I have 5 CA public keys in my keystore:

$ keytool -v -list -keystore ca.p12 -storepass ****
Keystore type: PKCS12
Keystore provider: SUN

Your keystore contains 5 entries

Alias name: 2021ca
Creation date: Mar 23, 2022
Entry type: trustedCertEntry
...

Hopefully I am just missing something, but has anyone tried to support multiple custom client CAs?

[scholzj]

The operator does not use the PKCS12 file. So it should not matter what you have there. It creates its own PKCS12 store inside the container from the CRT files. In general, we do not support multiple trust chains. You can try to add them to the ...-clients-ca-cert secret as CRT files with different suffixes. E.g. ca.crt, ca-2.crt. That might work. But it is not something we test for this use-case. This is what is normally used when the CA private key is replaced.

[scholzj]

I'm not sure what exactly would you want to add to the docs. The approach with adding multiple CAs into the Strimzi CA Secret was always a hack which worked only by chance. While it probably still works, it might stop working in any Strimzi release. So unless you want to add a warning to not use it, I do not think there is much to be documented.

[marceloavan]

Right, got it, that was a confusion from my side, I was considering it is a valid but undocumented approach. I just tested it with v0.46.0 and it is still working.

All considered, I will note here to try the custom authentication method.

Thanks!

[scholzj]

@marceloavan Please note, however, that the custom authentication with a custom truststore will not work with User Operator (not sure if you planned to use it or not). User Operator will not be able to issue TLS certificates for users using the certificates from the custom truststore.

[marceloavan]

@scholzj thanks for the heads up! I am creating users with tls-external authentication type in this case just to manage ACLs.

[scholzj]

Right, that part should work as long as your certificates use the matching subjects (or you also configure the SSL subject mapping for the listener).