Building the Keystore Architecture in thunderID

As thunderID matured, we needed a stronger and cleaner foundation for cryptographic key management. We were no longer dealing with one isolated signing use case. The platform had to support token signing, key discovery through JWKS, runtime encryption and decryption flows, and configuration-level secret protection.

That pushed key management beyond the scope of a small utility. It became a platform concern.

The main goal was to avoid hard-coding cryptographic behavior into each feature area. Instead, we wanted a provider-based architecture that could centralize key access, algorithm handling, and lifecycle behavior while keeping product modules independent from keystore internals.

At a high level, the keystore effort was about two things: consistency and evolution. We wanted one contract that all cryptographic consumers could rely on today, and an architecture that could evolve toward external KMS or HSM integration in the future.

The Problem We Wanted to Solve

Before introducing a proper keystore architecture, cryptographic responsibilities were spread across different parts of the system. Token signing, public key exposure, encryption, decryption, and configuration secret protection each had their own concerns.

That kind of separation can work at the beginning, but it becomes harder to maintain as the product grows.

If each module loads keys in its own way, validates algorithms differently, or handles failures inconsistently, the system becomes fragile. A change in key strategy would require changes across multiple protocol and runtime components. Testing also becomes more difficult because each consumer becomes tied to low-level key parsing and cryptographic setup.

We wanted to prevent that.

The keystore architecture was designed to create a single, dependable layer between cryptographic consumers and key material.

What We Planned For

During planning, we agreed that the keystore system needed to support two different cryptographic domains without mixing their responsibilities.

The first domain was runtime cryptography. This includes asymmetric and public-key operations used by OAuth, OIDC, JWKS, JWE, JWT signing, and flow execution paths.

The second domain was configuration cryptography. This includes symmetric encryption for protected configuration material and other secrets that need to be stored securely.

These two domains are related, but they are not the same. Runtime cryptography and configuration cryptography have different lifecycles, different key types, different algorithm requirements, and different operational risks.

We also wanted clear startup failure behavior. If key material is missing, invalid, or unsupported, the system should fail during initialization. It should not fail unpredictably later during a token issuance, encryption operation, or discovery request.

That fail-fast behavior became an important part of the design.

Alternatives We Considered

We considered several approaches before finalizing the provider-based design.

Option 1: Build One Monolithic Crypto Service

First option was to create a single crypto service that handled everything: signing, verification, encryption, decryption, public key discovery, configuration encryption, TLS material, and future provider integration.

This would centralize code, but not necessarily responsibilities.

A single broad crypto service can quickly become vague. It may start as a convenience layer, but over time it becomes a large service with too many unrelated responsibilities. Runtime cryptography and configuration cryptography would be forced into the same contract, even though they have different usage patterns and lifecycle concerns.

We wanted centralization, but not at the cost of unclear ownership.

Option 2: Go Directly to a KMS-First Model

We also considered a cloud KMS-first model. This is a valid long-term direction, especially for production environments that require centralized governance, auditability, and stronger key isolation.

However, going directly to a KMS-first implementation would have introduced a high integration cost before stabilizing thunderID’s internal cryptographic contracts.

The risk was that we would couple product modules to a specific external provider too early. Before choosing or deeply integrating a KMS or HSM backend, we needed to define the internal boundary that thunderID itself should depend on.

So we decided not to start with KMS as the first implementation. Instead, we designed the architecture so that KMS or HSM support could be introduced later behind the same provider contract.

Option 3: Provider-Based Keystore Architecture

The provider abstraction gave us the best balance.

It allowed us to centralize key management without locking the system into one backend. It also gave product modules a stable contract while leaving room for future provider implementations.

This became the selected approach.

Why We Built a Provider-Based Keystore

We introduced a key-manager provider layer with two distinct provider contracts.

The first contract is for runtime cryptography. It handles operations such as encryption, decryption, signing, and public key retrieval.

The second contract is for configuration cryptography. It handles symmetric encryption and decryption for protected configuration values.

This split was intentional.

A common mistake in keystore design is to create one generic interface for every cryptographic operation. That usually looks flexible at first, but it often becomes vague and leaky. Runtime crypto and config crypto have different consumers, different algorithms, and different failure surfaces.

By separating the contracts, we made the design easier to understand, test, and extend.

OAuth discovery, JWKS, JWT signing, JWE operations, and flow-related encryption can consume the runtime provider contract without knowing how keys are loaded internally. Configuration protection can use the config crypto provider without depending on runtime signing or public key discovery behavior.

That separation gave us cleaner ownership.

Keystore Design in Practice

The current default implementation is PKI-backed and initialized during server startup.

During initialization, key entries are loaded from configuration. Key and certificate pairs are validated. Thumbprints are computed. Supported algorithms are inferred from key types. If the configured key material is invalid or incomplete, initialization fails early.

Once initialized, runtime services operate by key reference instead of direct file path or direct key-loading logic.

That means business and protocol modules do not need to know where the key came from. They ask the provider to perform an operation using a key ID and algorithm, and the provider resolves the key material internally.

For configuration encryption, a symmetric key is loaded from configuration and used with AES-GCM. Encrypted payloads carry algorithm and key identifier metadata, which makes decryption explicit and deterministic.

This design gives us a uniform invocation model while still supporting different cryptographic primitives under the hood.

How Encryption, Signing, and Discovery Work

At runtime, the provider supports several operation families.

For encryption and decryption, it supports symmetric encryption paths such as AES-GCM for configuration-style cryptography. It also supports asymmetric and key-agreement flows such as RSA-OAEP and ECDH-ES families for runtime use cases.

For signing, the provider performs private key lookup by key ID and applies algorithm-aware signing behavior. This keeps signing consumers focused on the protocol flow rather than private key handling.

For public key discovery, the provider enumerates available certificates and maps them to compatible algorithms. This is important for JWKS and discovery endpoints, where consumers need public key metadata but should not know anything about private key storage.

Together, these operations allow protocol modules to work with a consistent keystore abstraction instead of duplicating key-management logic.

Failure Handling and Safety Characteristics

We intentionally designed the keystore flow to fail early and loudly on bad setup.

If configured keys are missing, invalid, unsupported, or incompatible with expected algorithms, the system surfaces that during initialization or operation validation. This reduces hidden runtime risk and makes deployment issues easier to diagnose.

At operation time, the provider validates key references and algorithm compatibility. Unsupported combinations are rejected explicitly instead of being silently downgraded or guessed.

This matters because cryptographic systems should be predictable. A key-management layer should not make surprising decisions about algorithms or fallback behavior. It should either perform the requested operation safely or return a clear failure.

What This Architecture Improved

The provider-based keystore improved thunderID in several ways.

First, protocol modules became simpler. JWT, JWE, JWKS, discovery, and flow-related components no longer need to own low-level key loading or parsing behavior.

Second, cryptographic behavior became more consistent. Key references, algorithm validation, public key discovery, signing, encryption, and decryption now flow through a common abstraction.

Third, testing became easier. Consumers can mock provider contracts instead of dealing with low-level key material setup in every test.

Fourth, the architecture now has a stable extension point. Future providers can be introduced without rewriting the modules that depend on cryptographic operations.

That was one of the biggest wins. We did not just solve the immediate local keystore problem. We created a path toward stronger key-management backends.

Why This Was the Right First Step

A good keystore architecture should not force every product module to understand cryptographic storage details. It should give those modules a clear and safe way to request cryptographic operations.

That is what the provider-based model gives thunderID.

It avoids scattered crypto logic. It avoids an overly broad monolithic service. It avoids prematurely binding the product to one external KMS model. And it gives us a stable internal contract that can support stronger backends later.

This is the kind of architecture that lets the platform evolve without forcing every feature area to change at the same time.

What Comes Next

With this foundation in place, the next improvements are straightforward.

• We can add pluggable provider selection driven by server configuration.
• We can introduce stronger key-rotation workflows for configuration encryption.
• We can expand runtime provider capabilities where the contract already anticipates future use cases.
• We can add deeper operational telemetry around key usage, provider behavior, and cryptographic failure classes.
• We can also introduce external KMS or HSM-backed providers while keeping product modules dependent on the same internal keystore contracts.

That is the value of the provider boundary. It gives us freedom to evolve implementation without destabilizing consumers.