Modify portal component to reload cached certificates when renewed on disk
The Coordinating Node's SSL certificates get renewed approximately every three months through Let's Encrypt. This presents a problem for Metacat-based Member Nodes and the Metacat component of the CN because they both rely on the
TokenGenerator. This class is a singleton pattern, and the
privateKey class variables are essentially cached in memory due to the singleton instance. When the Let's Encrypt certificates renew on disk, the in-memory versions remain, and authentication fails when the CN Tomcat restarts (loading the new certs), but the various MN Tomcat instances don't restart, and continue using the old certificate. This is true for the reverse scenario too, when the MNs restart, but the CN hasn't yet.
To avoid this fragile scenario where we must coordinate restarts (certainly not scalable), modify the
TokenGenerator to reload the SSL certificates when they change on disk so the in-memory version is up-to-date. Also, when a
TokenGenerator.getSession(token) fails, reload the certificates from disk to ensure we are using up-to-date certificates.
Test this with Metacat installations that have also been updated with the new
d1_portal jar file.