Apache Camel: Flattening the curve

Part 2: How Camel is getting faster and general purpose

In the first part we focused on development. There were however a few other area’s that people tend to critic Camel, or find it hard to use. Let’s list them:

1. Performance
2. Error Handling
3. Overview
4. Scattered Ecosystem
5. Concepts

Performance

Some people on Reddit say that Camel is slow and full of memory leaks. Others dismiss it because it’s Java-based. I know from experience that both is untrue. I’ve worked with dozens of companies that process billions of messages. In fact, Camel’s routing engine and integration patterns are so fast that it’s genuinely hard to overload it.

Take, for example, this route running with Camel 4.21.0:

- route:
    from:
      uri: jetty:http://0.0.0.0:9000/test
      steps:
        - log: Start
        - loop:
            copy: true
            expression:
              constant: 100
            steps:
            - split:
                expression:
                  jsonpath: $[*]
                aggregationStrategy: "#csvAggregationStrategy"
                steps:
                  - filter:
                      jsonpath: $[?(@.language == 'Dutch')]
                      steps:
                        - process:
                            ref: csvProcessor
        - log: End

When sending a JSON file containing 10,000 objects to the Jetty endpoint using cURL 100 times, each request loops 100 times. Therefore, the filter and aggregation steps run 100 million times. On my laptop, this process takes just one minute.

It’s true that modern Java 25 and Camel 4 are very fast. But why is there a perception that they are slow? This is because Camel frequently works with external I/O-bound endpoints. These can be HTTPS endpoints, brokers, databases, etc. If they aren’t asynchronous, a Java thread has to wait for them. This can cause the thread pool to become exhausted.

Due to the large number of I/O-bound integrations, which are the bottleneck, the introduction of virtual threads in Java seemed like a game changer. However, there were a few caveats that made it difficult to utilize virtual threads to their full potential. These were:

1. Thread pinning
2. Virtual threads support within Camel (routing engine)
3. Virtual threads support in third-party libraries

Now that thread pinning issues has been largely resolved since JDK 24, and more and more frameworks and libraries support it, there is nothing in the way to use it in Camel 4.22.0. Best to check the manual:

[embed]Virtual Threads in Apache Camel Apache Camel is an open source integration framework with 350+ connectors for databases, APIs, message brokers, and…camel.apache.org

I expect this to continue improving in the coming years, with better support and structured concurrency.

Error Handling

Error handling is another area that people find confusing. This is not because there is no good way to do it, but because there are too many ways to do it and they do not always work the way people think. This is mostly because Camel is message-oriented, so developers need to decide what to do with a message when something goes wrong. As with performance, this has to do with distributed computing and the complexity it inherits. Endpoints need retries and timeouts to be configured, as well as a way to handle exhaustion.

[embed]Error Handler Apache Camel is an open source integration framework with 350+ connectors for databases, APIs, message brokers, and…camel.apache.org

Because there are so many different ways, I would advise focusing on one try-catch and exception clause first when you only have a few routes. These are the easiest to reason about and are similar to what Java does in terms of exception handling where the error happens. A good tutorial is Baeldung’s:

[embed]Apache Camel Exception Handling | Baeldung Learn how to handle exceptions in our routes using Apache Camel.www.baeldung.com

When there are many integrations with hundreds of routes, it’s better to use a more centralized approach to handling routes that contains a dead letter channel. Although this allows for a more standardized, message-oriented approach to error handling, it can quickly become difficult to keep track of everything. To cope with this, the Error Registry was created:

[embed]Error Registry Apache Camel is an open source integration framework with 350+ connectors for databases, APIs, message brokers, and…camel.apache.org

The error registry is relatively new and was fully released in version 4.22.0. It allows you to retrieve all data about exchanges, where errors occur, why they occur, and what the data messages look like, all programmatically.

Overview

Another mistake that beginners often make is to put all the integration into one ‘God’ route. This bad practice renders many of the techniques we saw in the first part obsolete because it makes the three Ds — development, debugging and deployment — extremely difficult.

It has long been best practice to split integrations into many small routes, often connecting synchronously directly or asynchronously via SEDA or a broker. However, a new problem has arisen in the past: how can you indicate and manage the integration when everything is a separate route?

You have been able to label routes with a group label for a while, but the API has been extended in the last few releases to fully manage and monitor integrations as a group. For full details, see my previous blog post.

[embed]Building Camel integrations with route grouping It’s possible to write a complete Camel integration in just one route. But an integration is more like an application…raymondmeester.medium.com

Scattered Ecosystem

Like other large open-source projects such as Linux and Java, Camel suffers from a scattered ecosystem. Camel is an open-source project that can be enhanced, forked and customized for free. This is great, but it has resulted in multiple ways of doing things.

1. DSL’s Supports many DSL’s: Java, XML and YAML (and there even was an experimental Kotlin one).
2. Error Handling: Support many ways of error handling (see point before)
3. Languages: There are more than 25 different Languages such as scripted programming languages like Groovy, and template based languages like Velocity and Freemarker, and XML/JSON languages, and industry specifics such as Finance and Health Care, and many others.
4. Runtimes: There are more than 10 runtimes such as Main, Spring Boot, Quarkus and Kubernetes.
5. Components: More than 300 components, often overlapping in capabilities, and named of a framework, protocol or library.

While this is flexible and powerful, it can be overwhelming for beginners. Which features are relevant, and how do they work? The 4.22.0 version doesn’t address this issue, but it does emphasize the core of Camel more. I would advise beginners to stick to that for now.

1. Use the Camel CLI
2. Use the Java DSL
3. Use try-catch and exception clause
4. Use the built-in language Simple
5. Use built-in components and patterns

This approach keeps the solution small, eliminating the need to understand every detail of the framework. However, exploring the broad ecosystem based on your needs and interests is always possible.

Concepts

Saving the best for last, and a bit the elephant in the room, Camel concepts. The programming is very different than what most developers know. Or as a colleague once said:

Camel is easy to get a toy route working in an hour, but genuinely hard to become productive.

The difficulty lies not in the syntax, but in the layered concepts underlying it, such as EIPs, the exchange lifecycle, error/transaction scoping and the threading model. At its core, Camel is message-oriented rather than object-oriented and procedural. This means the message is the container of data. There are no variables, objects or primitives in the system; they are all contained within the message. This data is always in motion through the integration rather than within an application.

There are basically two approaches to how the message flows through the integration: dataflows and integration patterns.

The theory behind message-oriented programming and the various approaches to integration flow are topics for a separate blog post, but I do want to emphasize that the choice of James Strachan as the creator of Camel was not arbitrary. This programming model is simply the best way to solve integration problems. This is because the message is technology neutral way to exchange data.

Just as you can’t avoid learning object-oriented programming in Java, you can’t avoid learning message-oriented programming in Camel. A lot of the core concepts are explained at the architecture section in the user manual, but I acknowledge that a theoretical explanations how they all are tied together is still missing.

So, that’s how it is: Camel operates within a unique programming paradigm, and it takes a while for everything to become clear. On top of that, Camel message-orientated approach isn’t always strictly enforced, which has its advantages and disadvantages.

Conclusion

Apache Camel has long had a reputation for having a steep learning curve and a large, fragmented ecosystem. However, as Part 2 demonstrates, the framework is undergoing significant changes.

By combining modern Java performance features such as Virtual Threads with improved runtime observability (Error Registry and route grouping) and familiar programming abstractions (route variables and simplified CLI tooling), Camel is ironing out its rough edges.

Building robust systems no longer requires mastering 300+ components, dozens of runtimes, or complex XML DSLs. By sticking to the core patterns and embracing message-oriented thinking, Camel proves that it is not only fast and reliable, but also evolving into an accessible, general-purpose integration engine built for modern development.