Mutual TLS in Elixir Part 2: Testing and Intermediate CAs

Part 2 will cover testing client authentication and setting up intermediate CAs.

In Part 1, we got HTTPoison working to simply and securely send a client certificate as part of Mutual TLS. This covered the basic use case of sending a client certificate to a server.

To test our code, we used server.cryptomix.com which is a test site that requests a client certificate but it doesn't actually verify the client certificate. This isn't realistic since in practice, if the client certificate is expired or if it isn't signed by a trusted CA, the server will return an error.

Additionally, our dummy certificate from Part 1 was not very realistic either. It was signed directly by our dummy root CA. This is a common practice when you’re using a homegrown CA internal to your company (Ex. to do mTLS between your micro-services). But if you’re doing mTLS externally with your customers, it’s more common for your certificate to be issued from a public CA (Ex. DigiCert).

Part 2 of this series of blog posts will go over those advanced use cases of properly testing client authentication and how to set up client certificates that are signed by an intermediate CA.

We will use the same dummy client certificate from Part 1. You can clone https://github.com/MichaelViveros/blog_mtls_elixir to get a template project with that certificate and the necessary dependencies.

Also feel free to use your own client certificate if you have one.

Testing Client Authentication

There is no existing test site that verifies client certificates so I made an Apache docker image to do this locally. That image will run an Apache server that does two levels of client certificate verification:

• standard verification — is the certificate expired? signed by a trusted CA?
• custom verification — is the certificate for a specific, expected client?

For the custom verification, we can set the ALLOWED_CLIENT_S_DN environment variable to our client certificate's domain name ( dummy-mtls-client.com). The server will check the client certificate's domain name against this environment variable.

In a new terminal window, you can pull the image and start the server:

Note that we connect a volume from our ./certs directory (containing our dummy certificates) to the Apache server where:

• server.crt contains the server certificate the Apache server will present
• ca.crt contains the list of trusted CAs that the Apache server uses for client verification

ca.crt just contains one certificate for our root CA which the client certificate is signed by. Our server only expects one client to connect to it so it only has one trusted CA but it's also common for servers to accept multiple clients. In that case, you could put Mozilla's canonical list of trusted CAs into ca.crt (like we did in Part 1 with CAStore).

Note that server.crt's common name is dummy-mtls-server.com. Clients will expect the server's hostname to match the certificate's common name. Since we're running the server locally, we can update /etc/hosts to map dummy-mtls-server.com to 127.0.0.1. Then we can make requests to dummy-mtls-server.com and the hostname will match the certificate's common name.

Here’s the code from Part 1 that makes requests with a client certificate:

If you haven’t completed Part 1, you’ll have to add the CAStore dependency to mix.exs.

Let’s make a request to the local test server:

Returns:

Uh-oh. We get an :unknown_ca error. The client doesn't recognize the server certificate's CA. This is because the client is using CAStore's list of canonically trusted public CAs but the server certificate is signed by our dummy root CA.

We can fix this by pointing cacertfile to ca.crt.

Now the request to the test server works.

Returns:

Now we know that client authentication works! The test server was able to verify our client certificate and return a 200.

Intermediate CAs

Our client certificate is directly signed by our root CA. The certificate chain looks like:

But it’s common for certificates issued by public CAs to not be signed directly by the trusted root CA. They are signed by an intermediate CA, who in turn has a certificate signed by the root CA. This longer certificate chain is an industry standard which limits the number of root CAs in circulation. The less root CAs there are, the less CAs there are which could potentially be hacked.

To mimic this, we can use client2.crt which is signed by an intermediate CA. The certificate chain looks like:

Let’s update our code to use client2.crt.

We’ll have to restart the test server so it can expect client2.crt's domain name, dummy-mtls-client-2.com. You can either stop it with ctrl+c or run docker stop.

Run the test server again with ALLOWED_CLIENT_S_DN pointing to dummy-mtls-client-2.com:

Make a request to the test server:

Returns:

Dang it, we got an :unknown_ca error again. This time the server doesn't recognize the client certificate's CA. The server is only allowing certificates signed by the dummy root CA but our new client certificate is signed by the dummy intermediate CA.

To fix this, we can update the code to send the intermediate CA certificate alongside the client certificate. Then the server will be able to follow the chain from the client, to the intermediate CA, to the trusted root CA.

We can actually use one of our existing config options to do this: cacertfile. This option gets used for two things:

• specifying the trusted CAs that the client will check the server certificate against (this is how we’re currently using cacertfile)
• specifying intermediate certificates in the client certificate chain that the client will send to the server

See the Erlang ssl docs for more info.

We’ll have to include both ca.crt and intermediate_ca.crt in cacertfile:

• ca.crt contains the server certificate’s CA which will allow us to trust the server
• intermediate_ca.crt contains the intermediate CA which will allow the server to trust us

root_and_intermediate.crt contains both these certificates.

Here’s the updated code:

Sending a request to the test server:

Returns:

Nice! The test server was able to successfully verify our client certificate, which was signed by an intermediate CA.

Conclusion

Part 1 went over basic use cases to get mTLS working simply and securely. Part 2 went over more advanced use cases to test out client authentication and intermediate CAs.

By using an Apache docker image, we were able to verify that client authentication worked. Then we setup a client certificate signed by an intermediate CA to test out a longer certificate chain (which is common when using certificates issued by public CAs).

Here’s the code that uses the first client certificate (which is directly signed by the root CA) to test client authentication with the Apache test server:

Here’s the code to use the second client certificate (which is signed by an intermediate CA):

Now you should be able to get mTLS working whatever way you need it, best of luck!