Dynamically Reloading TLS Trust and Identity Material

Wouldn't it be nice if you didn't have to restart your server to dynamically update keys, certificates or trust stores?  I've spend a good bit of time on this across both client and server implementations and so I have a few pointers.  If you've read the last two posts, you know I've been working through requirements and implementation.  Now I'm going to add this auto-renewal of trust and key material to that effort.

Most folks will just need to deal with setting up trust and key managers for their web application.  That's fairly straightforward.  The challenge that I face with this particular application is that there are at least three different ways that trust and key material is provided to the underlying application, depending on how the connection is handled.

The basic idea is to set up a polling thread that periodically checks for changes in trust material, and then when that happens, go off and single anyone that has registered to those change events to update trust material in whatever way they need.

For my uses, inbound connections go through the server, which is what most will have to deal with.  But I also have two different types of outbound connections which are configured in different ways.  Some are SOAP using Apache CXF, others are RESTful API calls made through the HttpsURLConnection class (those APIs aren't that difficult to work with, and so don't need much more).  But each requires a different way to communicate trust and identity material to the system.

Let's start with the first, and most common:

Since Apache Tomcat 8.5 there is an API that enables you to reload key and trust material through the protocol handler for the connection.  During Embedded Tomcat setup (if you do it programatically), you create a Connector and add it to the service.  This connector is where you will add the SSLHostConfig and setup the protocol parameters (e.g., connection timeout, max connections), et cetera through a protocol handler derived from AbstractHttp11Protocol.

Somewhere in this process you will eventually wind up with three things:

1. The Connector connector.
2. The SSLHostConfig configuration.
3. The protocol handler nioProtocol.

    // Configure SSL

    connector.addSslHostConfig(configuration);

    // Get the protocol handler

    Http11NioProtocol nioProtocol = (Http11NioProtocol) connector.getProtocolHandler();

    // Do any configuration to it to the protocol handler.

        ...

After all of this is where you add the magic.  What you are doing here is calling a method to add a runnable to a list of methods to call when trust or key material needs to be updated.  I use this model because three different components need to do something to update trust and key material in the system I'm working with.

    // set up to reload configuration.

    addSslTrustChangedListener(() -> nioProtocol.reloadSslHostConfigs());

My actual implementation of the runnable is a little more complex, because I reuse portions of code that access key and trust stores, but generally, the main idea is to call reloadSslHostConfigs() to force a refresh of key and trust material.

CXF is a bit easier.  I'm still using XML configuration for the HTTPConduit that is used, but the for the bean containing the TLSClientParameters on that conduit, I set up a runnable to refresh the socket factory thus:

    @Bean(name="tlsParamsClientWs")
    public TLSClientParameters getTLSClientParameters() {
        TLSClientParameters p = new TLSClientParameters();
        // Force reload of Socket Factory
        p.setSSLSocketFactory(getSocketFactory());
        // Add listener to update the factory.
        addSslTrustChangedListener(() -> p.setSSLSocketFactory(getSocketFactory(true)));
        return p;
    }

This method constructs the bean that contains the client parameters, and the adds a listener that forces an update of the SSLSocketFactory.  You may be able to just update the parameters and let the factory be created for you, I need a bit more control for my application.  Note: getSocketFactory() and getSocketFactory(boolean forceReload) methods aren't shown here.

For my outbound restful connections which for now use HttpUrlConnection since they aren't that complicated, I have one last method which relies on bean that that eventually calls the getSocketFactory() method referenced above.

This enables all of my inbound and outbound connections to dynamically response to updates in trust material with the addition of a scheduled executor that checks for changes to files every 10 seconds (configurable), and then calls each trust changed listener (catching exceptions inside the loop so that an exception thrown by any single listener doesn't break the next one.

I'm not going to reproduce all of the code, it's fairly straightforward.  You can use something like the Java WatchService (see https://dzone.com/articles/how-watch-file-system-changes) or working with commons.io.monitor classes.

This is the basic idea though:

public void startMonitoring() {

            ScheduledExecutorService s = Executors.newSingleThreadScheduledExecutor();

            s.scheduleAtFixedRate(this::updateTrust, 10, 10, TimeUnit.SECONDS);

        }

public void updateTrust() {

try {

if (checkForUpdates()) {

for (Runnable trustChangedListener  : trustChangedListeners) {

try {

trustChangedListener.run();

} catch (Exception e) {

LOGGER.error("Failed to update trust material", e);

}

}

reloadCount = getReloadCount() + 1;

clientStoreOutOfDate = serverStoreOutOfDate = false;

LOGGER.info("Key and Trust stores updated.");

}

} catch (IOException e) {

LOGGER.error("Could not determine trust material update status", e);

}

}

 

You will probably have to do a bit of work to make this operate in your own environment, but now you can see how to integrate it with both server and client endpoints in several different ways.