Techniques: How to implement circuit breaker pattern without frameworks?

Source: Techniques: How to implement circuit breaker pattern without frameworks?

Producing reliable services often means making hard decisions about how to react when dependencies fail. Many teams grab a well-crafted library or framework and move on, but there are many situations — embedded systems, constrained containers, or policies that forbid new dependencies — where you must build a circuit breaker yourself. The goal of this article is to present concrete strategies you can implement in plain Java, why you would choose one over another, and the performance and correctness trade-offs you’ll face.

1. What a circuit breaker must do (without being pedantic)

A circuit breaker needs to (a) detect failure patterns, (b) stop letting requests hit an unhealthy component, © test the component periodically to see if it’s recovered, and (d) report state to observability. Those sound simple; real-world correctness, concurrency, and performance make the implementation non-trivial. Below we explore concrete strategies starting from minimal implementations to more robust windowed approaches and concurrency controls.

1.1 The basic state machine

1.1.1 States and transitions

At a minimum implement three states:

• Closed: requests flow normally; failures are counted.
• Open: requests fail fast without contacting the dependency.
• Half-open: allow limited probes to check recovery.

Transitions are triggered by failure counts, timeouts, and successful probes. Implementing transitions reliably under concurrency is the trickiest part — race conditions can produce both false reopenings and excessive blocking.

2. Strategy A — Very simple counter-based breaker

This is the minimal, easy-to-reason approach. It’s acceptable for low-traffic services. It uses an AtomicInteger failure counter and a timestamp for when the circuit was opened.

public class SimpleCircuitBreaker {    private final int failureThreshold;    private final long openMillis;    private final AtomicInteger failures = new AtomicInteger(0);    private volatile long openUntil = 0L; // epoch millis    public SimpleCircuitBreaker(int failureThreshold, Duration openFor) {        this.failureThreshold = failureThreshold;        this.openMillis = openFor.toMillis();    }    public boolean allowRequest() {        long until = openUntil;        return System.currentTimeMillis() >= until;    }    public void onSuccess() {        failures.set(0);    }    public void onFailure() {        int f = failures.incrementAndGet();        if (f >= failureThreshold) {            openUntil = System.currentTimeMillis() + openMillis;        }    }}

Explanation and trade-offs:

• How it works: Each failure increments the AtomicInteger. Once threshold is reached, set openUntil to now+openMillis. allowRequest checks openUntil.
• Concurrency: AtomicInteger ensures concurrent failure counts are correct. openUntil is volatile to ensure visibility across threads.
• Problems: The implementation counts failures across an unbounded time window — spikes far apart still trigger open. It may also re-open prematurely after openMillis because successes during open are ignored.
• Performance: Low overhead: single atomic increments and volatile read per request. Good for low-to-medium QPS.

This implementation is simplest when you only need fast failure isolation and you can accept coarse semantics.

3. Strategy B — Rolling window of outcomes (bucketed)

A common improvement is to compute failure rate over a sliding time window rather than counting all-time failures. Implementing an accurate sliding window requires either a ring buffer of buckets or a streaming summary. Below is a bucketed approach using time slices. This reduces the chance of opening the circuit due to historical failures.

public class BucketedCircuitBreaker {    private final int buckets;    private final long bucketMillis;    private final int failureThresholdPercent;    private final int minRequestThreshold;    private final AtomicInteger[] successBuckets;    private final AtomicInteger[] failureBuckets;    private volatile long startTime;    public BucketedCircuitBreaker(int windowMillis, int bucketCount,                                  int failureThresholdPercent, int minRequestThreshold) {        this.buckets = bucketCount;        this.bucketMillis = windowMillis / bucketCount;        this.failureThresholdPercent = failureThresholdPercent;        this.minRequestThreshold = minRequestThreshold;        this.successBuckets = new AtomicInteger[buckets];        this.failureBuckets = new AtomicInteger[buckets];        for (int i = 0; i < buckets; i++) {            successBuckets[i] = new AtomicInteger(0);            failureBuckets[i] = new AtomicInteger(0);        }        this.startTime = System.currentTimeMillis();    }    private int currentIndex() {        long now = System.currentTimeMillis();        long diff = now - startTime;        int idx = (int) ((diff / bucketMillis) % buckets);        return idx;    }    public void recordSuccess() {        successBuckets[currentIndex()].incrementAndGet();    }    public void recordFailure() {        failureBuckets[currentIndex()].incrementAndGet();    }    public boolean shouldOpen() {        int totalSuccess = 0;        int totalFailure = 0;        for (int i = 0; i < buckets; i++) {            totalSuccess += successBuckets[i].get();            totalFailure += failureBuckets[i].get();        }        int total = totalSuccess + totalFailure;        if (total < minRequestThreshold) {            return false;        }        int failurePercent = (int) ((totalFailure * 100L) / total);        return failurePercent >= failureThresholdPercent;    }}

Detailed explanation:

• Bucket semantics: Each bucket accumulates successes and failures for its time slice. currentIndex rotates through the ring buffer.
• Concurrency details: Per-bucket AtomicInteger reduces contention compared to a single counter under high QPS. You still need to handle bucket expiration: either reset old buckets when you rotate into them or keep logic that ignores stale buckets. The naive code above will keep accumulating old data until the ring wraps; in production you should zero the bucket when you start using it for a new time slice. That requires a per-bucket timestamp or CAS reset to avoid races.
• Accuracy vs performance: More buckets increases accuracy but costs memory and some overhead. This pattern avoids long-term history bias and works well at medium-to-high traffic.

3.1 Making bucket rotation safe

An improved rotation uses an array of objects holding a timestamp and counts; when a thread first touches an expired bucket it CAS-resets it. This avoids using global locks and is friendly to high concurrency. For extreme performance, consider LongAdder per bucket to reduce contention further.

4. Strategy C — Half-open probing and controlled retries

When the circuit is open you want to probe the dependency cautiously. Common practice allows N concurrent probes or a single probe at a time. Implementing this with a simple permit (Semaphore) is effective.

public class ProbingCircuitBreaker {    private volatile long openUntil = 0L;    private final int maxProbePermits;    private final Semaphore probePermits;    public ProbingCircuitBreaker(int maxProbePermits) {        this.maxProbePermits = maxProbePermits;        this.probePermits = new Semaphore(maxProbePermits);    }    public boolean tryProbe() {        if (System.currentTimeMillis() < openUntil) {            return false; // still open and not ready to probe        }        // allow only a few concurrent probes        return probePermits.tryAcquire();    }    public void probeComplete(boolean success) {        probePermits.release();        if (success) {            // here you would transition to closed and clear counters            openUntil = 0L;        } else {            // failure extends open period (backoff)            openUntil = System.currentTimeMillis() + 1000; // example backoff        }    }    public void open(long durationMillis) {        openUntil = System.currentTimeMillis() + durationMillis;    }}

Explanation:

• Why limit probes: When a downstream dependency recovers partially, flooding it with retries can re-break it. Limiting concurrent probes protects both sides.
• Backoff: On probe failure increase the open period (exponential or capped). On success, reset counters and move to Closed.
• Concurrency: Semaphore keeps state minimal and performs well. Be mindful: using tryAcquire prevents threads from blocking when probe slots are exhausted.

Combining probing with the bucketed window above gives robust, fair behavior: decide to open based on windowed metrics, and probe with limited concurrency to confirm recovery.

5. Strategy D — Bulkhead / semaphore isolation

A circuit breaker isolates failures, but a complementary strategy is to limit concurrent calls to a dependency so that failures don’t cascade. This is particularly important for thread-starved environments like servlet containers.

public class Bulkhead {    private final Semaphore concurrency;    public Bulkhead(int maxConcurrentCalls) {        this.concurrency = new Semaphore(maxConcurrentCalls);    }    public 



      Optional  







        invoke(Callable   







          task, long timeout, TimeUnit unit) throws Exception {        if (!concurrency.tryAcquire(timeout, unit)) {            return Optional.empty(); // reject due to bulkhead        }        try {            return Optional.ofNullable(task.call());        } finally {            concurrency.release();        }    }}

Trade-offs:

• Pros: Prevents thread pool exhaustion and isolates hot dependencies.
• Cons: Excessive rejections if maxConcurrentCalls is too low; tuning is necessary.

Bulkhead is not a circuit breaker replacement; combine them. Use timeouts carefully to avoid threads stuck waiting for permits.

6. Distributed breakers: when one JVM’s view isn’t enough

In service meshes or clusters, an instance-local breaker may not reflect the global health of a shared resource. Approaches:

• Per-instance breakers: Simple and fast; avoids network calls. Works well when failures are localized.
• Global breaker (centralized): Implement via Redis, etcd, or DB. Risk: adds latency and a dependency; also needs to be highly available or the breaker becomes a single point of failure.
• Hybrid: Local breakers with occasional global sync of state or thresholds.

If you use Redis to store a global failure count, you must handle race conditions, TTLs, and partial failures of the store. Use increments with EXPIRE and consider latency for each request — don’t query the central store on every request.

6.1 Example of a Redis-backed open flag (conceptual)

Use a single key per circuit keyed by resource id. On open, set key with TTL. Clients read it before issuing requests. Beware: network latency and Redis unavailability change semantics dramatically.

7. Metrics, instrumentation, and testing

A circuit without observability is dangerous. At minimum expose:

• Current state (closed/open/half-open)
• Failure rate over window
• Requests per second and success/failure counts
• Probe attempts and their outcome

Automate tests:

• Unit tests for state transitions under deterministic sequences.
• Concurrency tests using tools like JCStress or deterministic thread schedulers to catch races.
• Chaos tests in staging to validate backoff and probe behavior under real load.

7.1 Edge cases to test explicitly

• Clock skew and jumps: Use System.nanoTime() for intervals where possible; System.currentTimeMillis() with jumps can incorrectly reopen circuits or extend open windows mistakenly.
• GC pauses: Long pauses may create a burst of apparent timeouts — consider adaptive thresholds or require both high latency and failure rates to open.
• Flapping: Rapid open/close cycles harm throughput. Add hysteresis: minimum open time, increasing backoff on repeated failures, and require a minimum number of successful probes to close.
• Thread starvation: Avoid blocking in critical code paths or using synchronized blocks under heavy load; prefer non-blocking primitives.

8. Performance behavior and micro-optimizations

Performance matters: a breaker will be invoked per request in many systems.

• Volatile reads vs locks: Volatile reads are cheap; avoid full synchronized blocks on the fast path. Use atomics and CAS where possible.
• False sharing: Put frequently-updated atomics in separate cache lines if contention is high. Use padding objects on hot counters.
• Use LongAdder: For very high write-heavy counters favor LongAdder or LongAccumulator to reduce contention, at the cost of slightly more memory and eventual consistency in the counts.
• Minimize allocations: Recording per-request objects or building arrays per request adds GC pressure. Reuse buckets and pre-allocated counters.
• Batch metrics emissions: Aggregate device-level metrics and push periodically instead of synchronous metric calls on each request.

8.1 Cost examples

Atomic increment cost is roughly a few tens of nanoseconds under low contention; under heavy contention cost rises non-linearly. If your service does 100k+ calls/sec to the dependency, test with synthetic load. If a single AtomicInteger becomes hot, split into bucketed counters or LongAdder.

9. Practical recommendations and patterns to combine

Combine the following in most production systems:

1. Start with a bucketed sliding-window failure-rate detector to decide when to open.
2. On open, set an open-until timestamp with exponential backoff on repeated opens.
3. Implement half-open probing with a limited number of concurrent probes (Semaphore).
4. Add a bulkhead (semaphore) to limit concurrency to the dependency.
5. Instrument every transition and expose metrics and events to tracing systems so you can see why decisions were made.

These combined provide robustness against noisy failures, prevent cascading resource exhaustion, and let you verify behavior in production.

9.1 When to keep it simple

If you run a low-traffic service or the dependency is not shared widely, a SimpleCircuitBreaker (strategy A) may suffice. Keep the implementation small, document assumptions, and ensure you can swap in a more sophisticated breaker without a major refactor later.

10. Implementation checklist and debugging tips

• Use monotonic clocks (System.nanoTime()) for durations where possible.
• Avoid synchronized in hot paths; prefer atomics and CAS.
• Provide a health endpoint revealing circuit states for operational visibility.
• Log every state transition with context (rate, request counts, sample traces).
• Tune thresholds by observing actual failure modes — don’t pick numbers arbitrarily.
• Test under realistic failure scenarios: timeouts, high latency, partial failure, and resource exhaustion.

Final thoughts: building a circuit breaker without frameworks is entirely feasible and often preferable for bespoke constraints. Start with a simple correct approach, instrument thoroughly, and evolve to bucketed windows and controlled probes as traffic and failure complexity grows. If you have questions about adapting any of the patterns above to your specific JVM workload or concurrency model, please leave a comment and I’ll respond.

If my articles have been valuable to you, I’d be deeply grateful for your support at here . Your encouragement fuels my passion for creating even more insightful and high-quality content!