The Beauty of The JVM

How the JVM Transforms, Optimizes, and Executes Your Code?

Image source: https://www.nasa.gov/image-article/spiral-galaxy-ngc-4254s-dazzling-swirls/

Intro

You’ve probably run Java code before, whether using terminal or IDE, so I assume you have a base understanding of the nature of the VM-based languages like Java and C#. Simply, they translate high-level code into an Intermediate Representation (IR), which is then executed by the JVM (for Java) or CLR (for C#). Today, I want to focus specifically on what happens inside the JVM from the moment it receives the bytecode file from the Class Loader (a component within the JVM responsible for loading **.class files from `.jar`** files or other sources into the JVM’s memory space) until your code produces its final output.

Under the hood, the JVM uses a Just-In-Time (JIT) compiler to translate bytecode into native machine code at runtime. You compile your Java source code into bytecode, and the JVM executes that bytecode. The JRE (Java Runtime Environment) contains the JVM with libraries and tools required to run Java applications. In other words: JRE = JVM + core Java libraries + runtime components. So the native machine code runs within the JVM, which is a core component of the JRE. What I want to dive into in detail is how smart the JVM handles this interpretation process. It has more than one compiler but how does each of them work? Why does it have two? How much more efficient can compilation become if we combine them effectively? Let’s find out . . .

JVM Architecture

To understand how the JVM makes execution decisions, it’s important to look at its internal architecture. The JVM is a well-orchestrated system of components working together to transform, optimize, and execute your Java code efficiently.

Among the various JVM implementations, HotSpot stands out as the default engine provided by Oracle. I think if you checked your JDK version on your machine, you’ll find that it runs on the HotSpot JVM. This is responsible for the dynamic compilation, runtime optimization, and intelligent execution strategies. And so, I’ll talk specifically about HotSpot JVM implementation, continue . . .

At the top level, when your compiled **.class files are loaded, the Class Loader is the first thing to act. It locates bytecode, verifies it for safety, and prepares it for execution. Once loaded, the bytecode moves into the Runtime Data Areas**, where the JVM maintains method stacks, heap memory for objects, and other critical runtime information. See the figure below:

Image source: https://softwareperformancenotes.github.io/jvmarch/

The Execution Engine is where the work happens. This engine is responsible for interpreting and compiling bytecode into native machine code that your processor can understand. Inside it exists the Just-In-Time (JIT) Compilers. It isn’t a single entity. The HotSpot JVM actually includes two separate compilers:

• C1 (called Client Compiler in Oracle documentation): optimized for quick startup and faster compilation. It’s specially made for desktop applications such as IDEs, where quick startup and the ability to reach reasonable performance more than long-term runtime efficiency.
• C2 (also this called Server Compiler in Oracle docs): a more advanced compiler that performs deeper analysis and applies aggressive optimizations to generate highly efficient machine code, and for sure this came at the cost of longer compilation time. But guess, it’s better in production.

For example, a Spring Boot app is typically executed using this compiler (C2) since we prioritize maximum runtime performance over startup speed. The initial warm-up happens only once, but the benefits of faster, optimized execution continue throughout the application’s lifetime.

Both of them serve different goals but share a common goal: balancing performance with compilation overhead. The JVM dynamically decides which compiler to use depending on how hot a method becomes, using runtime profiling data to guide its choices. This dynamic decision-making leads us to one of the JVM’s beautiful concept called tiered compilation, where these two compilers collaborate together and according to the profiling data collected, the JVM decide how would it execute the code.

Hot Methods This term refers to a method or block of code that has been executed more than a specific threshold. This indicates that the code is frequently used and will likely be needed again. As a result, the JVM optimizes and stores it in memory to make future executions as fast as possible.

Profiling Data You can think of profiling data as a collection of runtime metrics and analysis results that describe how a block of code behaves during execution. For sure this data helps the JVM’s runtime compiler and optimization engine in making better decisions about how to handle and optimize that code for better performance.

In earlier Java versions, you (the developer) had to choose manually between the two, C1 or C2, depending on your performance needs. Starting from Java 7, tiered compilation became the default mode in HotSpot, allowing both compilers to work together smoothly to balance startup speed and long-term performance. Let’s see how this technique actually works with more details . . .

Tiered Compilation in HotSpot Execution Engine

We can conclude the whole story by saying: everything begins by interpreting bytecode (tier-0) and only compiles hot code through the two JIT compilers (C1 & C2) as needed. Just this.

HotSpot’s execution engine is elegantly tiered to balance fast startup with efficient performance. The diagram above in the JVM architecture section illustrates the JVM’s layered structure, with the Execution Engine (interpreter + JIT) fed by the class loader and runtime data areas. HotSpot’s Execution Engine consists of the interpreter, a JIT compilers (with intermediate-code generator, optimizer, and target-code generator), and the garbage. These components work together under HotSpot’s tiered compilation model. Now, let’s talk about the levels of executions that your code may be executed by one of them . . .

Execution Levels

HotSpot’s tiered compilation defines optimization on five levels. These levels (0–4) correspond to execution paths for methods:

• Level 0 (Interpreter): the method executes in the interpreter (no JIT).
• Level 1 (C1 - Full Optimization): the method is compiled with the client compiler (C1) at maximum speed (no profiling).
• Level 2 (C1 - Limited Profiling): the method is compiled with C1 with some basic profiling analysis (invocation and *back-edge counts**).
• Level 3 (C1 - Full Profiling): the method is compiled with C1 using full profiling data (invocation counts + detailed data).
• Level 4 (C2 - Server Compiler): the method is compiled with the server compiler (C2) for maximum optimization.

This list is derived from HotSpot’s advanced threshold policy documentation. Practically, every method starts at level 0 (interpreted). As the method executes, HotSpot collects dynamic profiling information and according to the results, the compilation decisions been decided, when thresholds are exceeded, the tiered compilation policy may transition the method to a higher level.

The common path for a hotspot is 0 → 3 → 4: start in the interpreter, then compile with C1 in full-profiling mode (level 3), then recompile with C2 (level 4) for best performance. However, HotSpot’s policy adapts dynamically, which causes alternative paths based on the method size and compiler load. For example:

• Trivial Methods: if after an initial C1 compile HotSpot found out that a method is too simple, it may stop at level 1 (C1 with no profiling) since no need for optimization, it’s logical.
• Busy C1 compiler: if the C1 is overloaded, HotSpot can skip heavy profiling and either interpret longer or go straight to C2, collecting profiling in the interpreter and then generating C2 code directly.
• Busy C2 compiler: if the C2 is overloaded, HotSpot may first compile at C1 with limited profiling (level 2) so the code runs faster, then later recompile with full profiling (level 3) once C2 is free.

This summary leaves many other details, I’ll leave the resources below for reference, you must take a look if you want to know more.

Additionally, HotSpot supports on-stack replacement (OSR): if a loop becomes hot (its **back-edge counter* crosses threshold) before the whole method is hot, HotSpot can compile that loop to native code on the fly, replacing execution in the interpreter mid-loop. This allows individual hot loops to be optimized even if the surrounding method is still cold**.

Back-edge counts Simply a metric in the JVM used to determine how frequently a loop inside a method is executed. In the compilers jargon, a back edge represents a jump in the control flow that returns back to an earlier point in the code which meaning that there’s a loop in this part of code. The JVM keeps track of how many times this back edge is taken during execution. When the count exceeds a certain threshold, the JVM interprets this as a signal that this code block is hot.

From JDK 9, HotSpot also partitions its **code cache** into segments: profiled-code (for C1/JVMCI-compiled code) and non-profiled (for C2). This reduces [fragmentation](https://stackoverflow.com/a/3770593/17893979)* in the memory because short-lived profiled code (C1) is kept separate from long-lived optimized code (C2).

Code Cache Simply a dedicated memory space inside the JVM where the compiled native code produced by the JIT compiler is stored. By keeping optimized machine code in this cache, the JVM can execute frequently hot methods directly without recompiling them.

The Graal Compiler

In recent JVMs, Oracle has introduced the Graal compiler as a next-generation JIT. Graal is written totally in Java (using the JVM Compiler Interface, JVMCI) and is designed to replace the C2 server compiler. In GraalVM, HotSpot defaults to using Graal at the top tier: the JVM simply passes bytecode to Graal, which produces machine code and returns it to HotSpot’s code cache. The Graal compiler operates on a graph-based intermediate representation (IR), which is a complex topic (you’ll found it in the resources) but for short, this made the compiler language-agnostic and this is really beautiful, think of it, the same optimizer can handle Java, JavaScript, Python, Ruby, and other Truffle-based languages in the VM. See the image below from Oracle documentation:

GraalVM architecture (HotSpot + Graal + Truffle). Graal (red) is an advanced JIT on top of the HotSpot VM and Truffle (gray) provides a language-independent IR. GraalVM can run multiple languages (JS, Python, etc.) on this shared compiler infrastructure.

Key differences of Graal vs. HotSpot’s traditional C1/C2 compilers include:

• Implementation Language: Graal is implemented in Java (via JVMCI) but C1/C2 are native C++. This makes Graal easier to maintain and extend (think of it, the JVM hot code can optimize the compiler itself).
• Intermediate Representation: Graal uses a high-level, language-neutral graph IR (enabling polyglot optimizations). C2 uses its own IRs (like HIR, LIR) focused on Java bytecode.
• Advanced Optimizations: Graal includes modern phases such as aggressive inlining, partial escape analysis, and more. For example, Graal’s partial escape analysis can eliminate many short-lived allocations (common in lambdas and streams).
• Polyglot Support: thanks to Truffle and JVMCI, GraalVM can JIT-compile JVM languages and other languages on the same IR, which is outside the scope of C1/C2.
• Ahead-of-Time Mode: GraalVM also offers an AOT native-image mode (JEP 295) to compile Java code into a standalone executable. (C2 has no native-image support)

Overall, Graal serves as the new high-tier JIT in HotSpot. C1 is still used for early tiers, but Graal replaces C2 as the optimizing backend. In HotSpot/GraalVM runtime mode, Graal is simply the top-tier compiler by default. As InfoQ notes, C2 has effectively reached end-of-life and Graal is its intended successor. By leveraging JVMCI and a powerful IR, Graal brings a clean, modern architecture to the JVM’s JIT stage.

I wanna thank O’Reilly & Scott Oaks for this excellent resource. I highly recommend this book to anyone who works with Java daily or wanna understand more about what’s happening under the hood. For this article, I specifically took some insights from Chapter 4: Working with the JIT Compiler.

Book in Amazon: https://www.amazon.com/Java-Performance-Definitive-Guide-Getting/dp/1449358454

Resources

Oracle GraalVM Architecture Graal Compiler Documentation Threshold Policy for Execution Levels Truffle Language Implementation Framework Graph based Intermediate Representation by Graal

I hope I’ve added to you any new term and thanks for your time.