Observability for Hazelcast clients: Micrometer, near-cache, health checks

Hazelcast Toolkit series: Part 5.

Hazelcast client integration is not finished when the client connects.

In production, the more interesting questions come later:

• Are near-caches actually being used?
• Are they being invalidated?
• Is Hibernate L2 cache hitting or missing?
• Did a runtime-created map get metrics?
• Which distributed objects exist in this client?
• Is the cache layer helping, or just making the system harder to reason about?

Hazelcast Toolkit treats observability as part of the integration, not as something to bolt on after the first incident.

It exposes three complementary surfaces:

1. Micrometer meters for production dashboards and alerts.
2. Diagnostic HTTP endpoints for manual inspection.
3. An Actuator near-cache probe for active verification.

The three surfaces

These surfaces are intentionally separate. A diagnostic endpoint is not a metrics pipeline. A metric is not an active correctness probe. A health check should not replace dashboards.

Enable Micrometer metrics

Configuration:

hazelcast:
  toolkit:
    metrics:
      enabled: true

With a MeterRegistry on the classpath, the toolkit exposes Micrometer MeterBinder beans for Spring Boot to bind to the available registry. Since 0.3.1, the toolkit no longer manually calls bindTo(…) during bean creation, which avoids circular dependency scenarios involving simple registries and metrics binders.

The binders cover:

• Hazelcast IMap near-cache metrics;
• JCache / ICache near-cache metrics;
• Hibernate L2 cache statistics when Hibernate L2 is enabled.

In stricter enterprise builds, you may need to declare Micrometer explicitly:

implementation 'io.micrometer:micrometer-core'

If you use Spring Boot Actuator with a registry exporter, the meters flow into your usual monitoring stack.

Near-cache metrics

The near-cache binder publishes metrics for maps and caches that already exist at startup, and also for distributed objects created later at runtime.

Important meters:

Useful tags:

That gives dashboards enough shape to answer questions like:

• Which maps are actually getting near-cache hits?
• Which maps have high miss counts?
• Are invalidations happening after writes?
• Is a cache growing in memory?

Hibernate L2 metrics

When Hibernate L2 is enabled, the toolkit also publishes Hibernate statistics:

hazelcast:
  toolkit:
    metrics:
      enabled: true
    hibernate:
      l2:
        enabled: true
        extended-config: true
        use-statistics: true

Meters:

The meters include a regionFactory tag, for example:

regionFactory=JCACHE

This is useful when comparing JCACHE, HAZELCAST_LOCAL, and HAZELCAST modes across services or environments.

Diagnostic endpoints

Micrometer is for production monitoring. Sometimes you still want a quick HTTP view while debugging.

Enable the diagnostic controller separately:

hazelcast:
  toolkit:
    metrics:
      diagnostic-endpoint:
        enabled: true

This registers:

GET /hz-toolkit/hz/objects
GET /hz-toolkit/hz/maps
GET /hz-toolkit/hz/map/near-stats/{mapName}
GET /hz-toolkit/hz/jcache/near-stats/{cacheName}

Use this layer for manual inspection:

• which distributed objects exist;
• which maps are visible to this client;
• whether a specific map has near-cache stats;
• whether a JCache cache is exposing near-cache stats.

Do not treat these endpoints as your main monitoring API. They are for humans during troubleshooting.

Since 0.3.1, the JCache near-stats endpoint also handles caches without Near Cache more gracefully. Instead of surfacing Hazelcast’s UnsupportedOperationException as HTTP 500, it returns a normal diagnostic response:

{
  "status": "OK",
  "name": "test-entity-region",
  "local": {
    "available": true
  },
  "near": {
    "enabled": false,
    "reason": "Near Cache is not enabled"
  }
}

That keeps the endpoint useful for troubleshooting both near-cache-enabled and plain JCache setups.

Active near-cache verification

Metrics tell you what happened. Sometimes you need a targeted probe that says:

For this entity, can this service load from L2/near-cache, then observe invalidation after eviction?

Enable the Actuator endpoint:

hazelcast:
  toolkit:
    actuator:
      near-cache-check:
        enabled: true
        entity-class: com.mycompany.entity.User
        entity-id: "42"
    hibernate:
      l2:
        enabled: true
        extended-config: true
        use-statistics: true

Request:

GET /actuator/hazelcastNearCache
GET /actuator/hazelcastNearCache?entity=com.mycompany.entity.Product&id=99

Example response:

{
  "status": "OK",
  "entity": "com.mycompany.entity.User",
  "id": "42",
  "idType": "java.lang.Long",
  "resolvedId": 42,
  "nearCache": {
    "hitVerified": true,
    "invalidationVerified": true
  },
  "timings": {
    "cachedLoadMs": 0,
    "postEvictionLoadMs": 41
  },
  "hibernateStats": {
    "l2HitsDeltaOnCachedLoad": 1,
    "l2MissesDeltaAfterEviction": 1,
    "l2HitsDeltaAfterEviction": 0
  }
}

The endpoint first resolves the entity id type through the JPA metamodel, then performs a real sequence:

1. Load the entity in a fresh EntityManager.
2. Load it again in another fresh context.
3. Verify the second load is served from L2/near-cache.
4. Evict the entity via JPA cache.
5. Load again and verify the cache is cold.

This is not just a “client is connected” check. It verifies the behavior you actually care about. In 0.3.1, id conversion works for common id types such as Integer, Long, String, Short, Byte, primitive equivalents, and value-object ids with a static valueOf(String).

Suggested dashboard panels

A useful first dashboard does not need twenty graphs.

Start with:

Near-cache hit rate by cache
Near-cache invalidations by cache
Near-cache owned entries by cache
Hibernate L2 hit/miss/put counts
Hibernate statistics enabled gauge
JVM memory next to near-cache memory

Then add alerts carefully:

Avoid alerting on raw misses alone. Misses can be normal after deploys, evictions, and cold starts.

Minimal production setup

management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus,hazelcastNearCache

hazelcast:
  toolkit:
    metrics:
      enabled: true
      diagnostic-endpoint:
        enabled: false
    hibernate:
      l2:
        enabled: true
        extended-config: true
        use-statistics: true
    actuator:
      near-cache-check:
        enabled: true
        entity-class: com.example.catalog.Product
        entity-id: "42"

For local troubleshooting, turn on diagnostic endpoints:

hazelcast:
  toolkit:
    metrics:
      diagnostic-endpoint:
        enabled: true

Secure both /hz-toolkit/… and /actuator/hazelcastNearCache with Spring Security if they are reachable outside a trusted network.

Resources

• GitHub repository: https://github.com/javaquasar/hazelcast-spring-toolkit
• Maven Central: https://central.sonatype.com/artifact/io.github.javaquasar/hazelcast-toolkit-spring-boot3/0.3.1
• Observability notes: docs/observability.md
• Runnable Spring Boot 3 example: example-spring-boot3

Gradle:

implementation 'io.github.javaquasar:hazelcast-toolkit-spring-boot3:0.3.1'

Maven:

<dependency>
    <groupId>io.github.javaquasar</groupId>
    <artifactId>hazelcast-toolkit-spring-boot3</artifactId>
    <version>0.3.1</version>
</dependency>

Closing thought

Cache integrations need visibility because they change the shape of your system.

The goal is not just to connect Spring Boot to Hazelcast. The goal is to know whether the client, near-cache, and Hibernate L2 layer are doing useful work after the application reaches production.

Hazelcast Toolkit series

This article is part of a practical series about making Hazelcast client integration feel natural in Spring Boot.

You are reading: Part 5.

• Part 1: Zero-Boilerplate Hazelcast Client for Spring Boot

• Part 2: Auto-registering Hazelcast Compact serialization in Spring Boot

• Part 3: IMap listeners as Spring beans

• Part 4: Hazelcast Hibernate L2 cache in real Spring Boot apps

GitHub:

https://github.com/javaquasar/hazelcast-spring-toolkit