The Triad (Hudi, Iceberg, Delta) of Open Table Formats [3/n] in 2026

In the last 2 blogs we explored the origins, motivations, design philosophies and architecture of the triad of Open Table Formats:

• https://medium.com/@Apurvar/the-triad-of-open-table-formats-1-n-history-motivation-design-philosophy-22ccf06d9606
• https://medium.com/@Apurvar/the-triad-hudi-iceberg-delta-of-open-table-formats-2-n-architecture-deep-dive-8bd7485b5d7c

In this one we take a deeper look through the lens of performance and derive a decision framework to help decide when to choose which. Lastly we present the current state of each by exploring the latest developments across the ecosystem and where the industry is headed next.

📊 Performance Observations

One of the most common questions data practitioners ask is:

Which Open Table Format is the best?

The answer, unsurprisingly, is that it depends.

Unlike traditional databases where the storage engine and execution engine are tightly coupled, Open Table Formats sit between the compute engine and object storage. As a result, observed performance depends not only on the table format itself, but also on the query engine, table maintenance strategy, workload characteristics and even the physical layout of the data.

This makes performance benchmarking significantly harder than simply comparing database systems. Nevertheless, several studies over the last few years have attempted to evaluate the relative strengths and weaknesses of Hudi, Iceberg and Delta.

LHBench — The First Formal Benchmark

LHBench, introduced in the CIDR 2023 paper, was arguably the first comprehensive benchmark specifically designed for Open Table Formats.

The benchmark uses TPC-DS style analytical workloads and evaluates the formats across dimensions such as:

• bulk ingestion
• updates
• deletes
• analytical queries
• storage overhead
• maintenance operations

One of the key contributions of the paper was demonstrating that Open Table Formats are not merely metadata layers on top of Parquet but have meaningful performance implications arising from their architectural choices.

Key Findings From LHBench

Data Loading

• Delta Lake and Iceberg achieved nearly identical, fast bulk loading times.
• In the benchmark configuration used by LHBench, Apache Hudi was approximately 10× slower during bulk ingestion. The authors attribute this to Hudi’s focus on keyed upserts, which introduces additional preprocessing such as record indexing, uniqueness checks and data redistribution.

Query Performance (Read Workloads)

• Overall TPC-DS query workloads ran 1.4× faster on Delta Lake than on Hudi, and 1.7× faster on Delta Lake than on Iceberg.
• Delta vs. Hudi: Delta Lake targets larger file sizes (e.g., 128 MB vs. Hudi’s 8.3 MB files). Hudi’s smaller files reduce columnar compression efficiency and increase overhead for large table scans.
• Delta vs. Iceberg: Iceberg utilizes a custom Parquet reader in Spark to support column dropping and renaming. This custom reader is significantly slower than the default Spark Parquet reader used by Delta.

🔄 Data Update Strategies: CoW vs. MoR

The paper evaluates the performance trade-offs between Copy-on-Write (CoW) and Merge-on-Read (MoR) architectures:

• Copy-on-Write (CoW): High write amplification but zero read amplification. Delta Lake’s optimized MERGE command keeps it highly competitive despite only using CoW in the tested version.
• Merge-on-Read (MoR): Low write amplification (faster writes) but introduces read amplification. ◦ In Hudi, MoR updates were 1.3× faster than Hudi CoW, but subsequent queries became 3.2× slower. ◦ In Iceberg, MoR updates were 1.4× faster than CoW. However, once a table accumulated more than 10,000 updated rows, the query latency of Iceberg MoR fell behind Iceberg CoW due to record-reconciliation overhead.

📁 Metadata Access & Query Planning

The systems differ in how they process table metadata (like file names, sizes, and min-max statistics):

• Distributed Planning (Delta Lake & Hudi): Metadata is stored in a tabular format or transaction log and parsed using a parallel cluster job (e.g., via Spark).
• Single-Node Planning (Iceberg): Metadata is organized hierarchically into manifest files and parsed on a single client node using indexes.
• The Trade-off: Iceberg’s single-node query planning provides lower startup latency for small tables. However, Delta’s distributed planning scales dramatically better for large tables, outperforming Iceberg by 7–12× during the query planning phase on a table with 200,000 files.

Onehouse Challenges the Verdict

The benchmark results sparked significant discussion within the community.

Onehouse, founded by several Hudi creators and contributors, argued in https://www.onehouse.ai/blog/apache-hudi-vs-delta-lake-vs-apache-iceberg-lakehouse-feature-comparison (later updated in 2025 to include the new features and improved performance functionality) that parts of the benchmark configuration did not fully reflect Hudi’s intended operating model.

Specifically, they noted that:

• some workloads favoured analytical read performance over mutation-heavy workloads
• Hudi-specific tuning opportunities were not fully exercised
• certain comparisons effectively benchmarked implementation choices rather than architectural capabilities

This highlights an important lesson:

Benchmarking Open Table Formats is inherently difficult because many observed performance characteristics emerge from the interaction between the table format, execution engine and operational practices.

Unlike database benchmarks, there is rarely a single configuration that fairly represents every deployment model.

More Recent Academic Studies

Subsequent research has generally reinforced a more nuanced conclusion:

There is no universal winner.

Instead, the formats tend to excel in different areas that closely align with their original design goals.

A Comparative Performance & Metadata Study (2025)

https://www.researchgate.net/publication/399425664_A_Comparative_Performance_Metadata_Study_of_Open_Table_Formats_Iceberg_vs_Delta_vs_Hudi_at_Scale

This study extended the analysis beyond raw query performance and focused heavily on metadata scalability.

The findings broadly reinforced Iceberg’s strengths in:

• metadata management
• planning efficiency
• large-scale analytical workloads

while highlighting Delta’s balanced performance and Hudi’s mutation-friendly architecture. The study indicates that optimal table format selection depends on balancing metadata scaling, query planning speed, and mutation throughput.

Automotive Data Engineering Use Cases (2025)

https://arxiv.org/pdf/2508.13396

This study evaluated the formats using realistic telemetry and automotive workloads rather than synthetic benchmarks.

Interestingly, the results aligned closely with the architectural philosophies discussed in the previous article:

• Hudi performed strongly for continuous ingestion and CDC.
• Iceberg excelled in large analytical workloads.
• Delta offered a balanced operational model with strong Spark integration.

The study concludes that no single format perfectly covers the end-to-end automotive pipeline. The authors propose a hybrid Lakehouse architecture as the emerging industry norm:

• Hudi handles low-latency real-time telemetry ingestion.
• Iceberg archives historical data layers for cross-fleet batch analytics.
• Delta Parquet curates the feature stores for training machine learning models.

🧹🔧Operational Aspects: Beyond Read and Write Performance

Perhaps the most important trend since 2023 is the realisation that raw query performance is only part of the story.

As organisations move from experimentation to production, operational characteristics increasingly dominate the decision.

Questions such as:

• How often must tables be compacted?
• How quickly do metadata structures grow?
• How expensive are maintenance jobs?
• How much expertise is required to operate the platform?

often become more important than whether one benchmark query runs 10% faster.

This has led to growing interest in research around:

• automated compaction
• metadata management
• storage optimisation
• maintenance automation

For example, LinkedIn’s AutoComp project focuses on automatically managing compaction for log-structured lakehouse tables, reducing the operational burden associated with long-running production deployments.

The lesson is simple:

As Open Table Formats mature, operational simplicity increasingly becomes a competitive differentiator.

💸 Cost Considerations

As with other aspects there isn’t a clear-cut winner when it comes to cost. What is perhaps more beneficial is to develop the right framework for evaluating cost for your particular use-case. Generally cost associated with operating an OTF is the sum total of storage cost, compute cost, operational (maintenance) cost, organisation cost + vendor cost (if applicable).

Total Cost of Ownership

    = Storage Cost
    + Compute Cost
    + Maintenance Cost
    + Organizational Cost
    + Vendor Cost

Storage and Compute costs are generally in the same vicinity and these may not be the strongest levers to pull. If there is a specific use-case that one of the table formats is most optimised for — Example Hudi for streaming upserts — then the cost, even if higher, may be justified as it meets the use-case.

The more interesting differentiating factors in my opinion really are:

• the cost associated with maintaining the operational aspects such as Compaction, Cleanup, File rewrites, Optimisation, etc
• the cost associated with funnelling budget into the team for learning curves, resources, expertise, etc

Several sources cite that Delta Lake is probably the cheapest when it comes to operational and organisational cost however there is a catch!

⚠️While Delta is open source, its true cost-efficiency is unlocked on Databricks. Features like Predictive Optimisation, Photon engine, Auto Optimise, Managed Liquid Clustering severely reduce compute time and manual maintenance, offsetting vendor premiums.

💡The biggest hidden variable in most Hudi vs Iceberg vs Delta comparisons is that many benchmarks compare formats, while organisations pay for platforms. The economics of Delta on Databricks can look very different from the economics of open-source Delta, just as the economics of Iceberg depend heavily on the catalog, compute engines and operational tooling surrounding it.

💭Concluding Thoughts

The CIDR 2023 benchmark remains the most widely cited academic comparison, but subsequent analyses have shown that benchmark results are highly sensitive to engine implementations, table maintenance strategies and workload characteristics. More recent studies increasingly conclude that the formats exhibit different strengths rather than a universal winner: Hudi for streaming mutations and CDC, Iceberg for metadata scalability and analytical planning, and Delta for transactional simplicity and Spark-centric workloads.

💡Perhaps the most important observation is that the benchmark outcomes closely mirror the architectural priorities discussed in Part 2. Formats tend to perform best on the workloads they were originally designed to optimise.

🌳Decision Framework

As apparent from the preceding section, there isn’t a universal answer to “Which Open Table Format is the best?” . The feature gaps are increasingly reducing as all the formats roll out more and more features to extend functionality coverage.

How then do you choose what is best for your use-case or organisation?

In 2026, choosing between Hudi, Iceberg and Delta is rarely about missing features. All three support ACID transactions, schema evolution, time travel, CDC and streaming workloads. The decision increasingly comes down to which architectural tradeoffs best match your workload:

• Hudi optimises the write path.
• Iceberg optimises metadata planning.
• Delta optimises transactional simplicity.

The best choice is often the one whose core optimisation aligns most closely with your primary bottleneck.

• Choose Hudi when your primary bottleneck is continuous mutations and CDC.
• Choose Iceberg when your primary concern is vendor neutrality, metadata scalability and multi-engine analytics.
• Choose Delta when your primary concern is time-to-production, operational simplicity and Spark/Databricks integration.

Already using Databricks?
    └─ Yes → Delta

Need strong vendor neutrality?
    └─ Yes → Iceberg

Need fastest time-to-production?
    └─ Yes → Delta

Small data team / want minimal organisation and maintenance cost?
    └─ Yes → Delta

Heavy CDC / Upserts?
    └─ Yes → Hudi

Multi-engine analytics?
    └─ Yes → Iceberg

Petabyte-scale metadata concerns?
    └─ Yes → Iceberg

Otherwise:
    Delta or Iceberg depending on ecosystem preference

Finally, borrowing one of the most resonating lines from https://levelup.gitconnected.com/i-stress-tested-3-lakehouse-formats-and-nearly-had-a-breakdown-a40424c0e734 (Btw this is a great read! Highly recommend!)

The best format is the one your team can actually operate.

Best Format
    = f(
        Primary Bottleneck To Solve,
        Operational Complexity,
        Ecosystem & Vendor Strategy,
        Team Expertise,
        Budget For Learning Curves,
        Maintenance Cost
      )

💡And maybe it is not an either/or question at all! Maybe you need multiple data formats suited for different use-cases and an inter-operability layer. Not surprising then that the recent years have seen the rise of projects like Apache XTable and Delta Uniform .

☕What’s Brewing in 2026?

Perhaps the most interesting story in 2026 is not how the formats differ, but how they are converging. Features that were once unique to a particular format are increasingly appearing across the ecosystem. Yet each project continues to evolve in ways that reinforce its original architectural strengths.

While all three now support ACID transactions, schema evolution, CDC, time travel and streaming workloads, their roadmaps continue to reflect the priorities that shaped their original designs: Hudi remains focused on efficient mutations and indexing, Iceberg on metadata scalability and interoperability, and Delta on transactional simplicity and operational automation.

Hudi

The Hudi roadmap continues to double down on its original mission: making mutable data lakes practical at scale.

Some of the most notable developments across the Hudi 1.0, 1.1 and 1.2 releases focus on indexing, metadata acceleration and operational flexibility.

Indexing Becomes a First-Class Citizen

One of Hudi’s biggest differentiators has always been its ability to efficiently locate records in large datasets. Recent releases significantly expand these capabilities.

• Secondary Indexes bring SQL-style indexing capabilities to Hudi tables, enabling efficient lookups on non-primary-key columns.
• Expression Indexes extend this concept further by allowing indexes to be built on derived expressions rather than only raw columns or partition keys.
• Partitioned Record Indexes improve lookup scalability in extremely large partitioned datasets by distributing index management more efficiently.

Taken together, these features push Hudi closer to traditional database-style indexing while preserving the openness and scalability of data lake storage.

Metadata Table Evolution

What began as a mechanism for accelerating file listings is increasingly evolving into a generalized metadata platform.

Recent enhancements include:

• richer metadata indexing capabilities
• improved metadata scalability
• deeper integration with query planning

Particularly notable is Column Stats V2, which extends support for complex and logical data types such as timestamps and decimals with precision and scale. This enables significantly more effective data skipping and query pruning.

The Metadata Table is increasingly becoming one of Hudi’s most strategically important components.

Interoperability and Engine Support

Historically viewed as a write-optimized format, Hudi has continued investing heavily in interoperability.

Recent releases introduced:

• broader Spark and Flink compatibility
• improved Trino and Presto integrations
• pluggable table format support

The latter is especially interesting as it signals a broader trend across the ecosystem: reducing format silos and enabling coexistence between multiple table formats.

Modernised Record Merging

Record merging lies at the heart of Hudi’s mutation story.

The migration from the legacy HoodieRecordPayload model to the newer HoodieRecordMerger API introduces:

• declarative merge modes
• engine-native execution
• partial updates
• simplified customisation

This represents another example of Hudi reinforcing its original strength: efficient handling of mutable datasets and CDC workloads.

Iceberg

If Hudi’s recent evolution has focused on indexing and mutations, Iceberg’s roadmap continues to reinforce its position as the metadata and interoperability leader of the lakehouse ecosystem.

The biggest development is undoubtedly Iceberg V3.

Iceberg V3

Iceberg V3 introduces several significant capabilities:

• Deletion Vectors
• Row Lineage
• Variant Data Type
• Geospatial Types
• Nanosecond Timestamp Precision

Taken together, these features move Iceberg beyond being “just” a metadata layer and closer to a comprehensive table abstraction for modern analytical workloads.

Row Lineage

Among the V3 features, Row Lineage may ultimately prove the most impactful.

By introducing stable identifiers that track rows across modifications, Iceberg strengthens several important use cases:

• Change Data Capture (CDC)
• Auditing
• Data Governance
• Data Provenance
• Regulatory Compliance

As organisations increasingly focus on governance and observability, Row Lineage positions Iceberg well for enterprise-scale deployments.

Deletion Vectors

Historically, Iceberg relied primarily on Equality Deletes and Position Deletes to implement row-level mutations.

Deletion Vectors introduce a more efficient mechanism for representing deleted records without requiring immediate file rewrites.

This reduces rewrite amplification and narrows the gap between Iceberg and formats traditionally considered stronger for mutation-heavy workloads.

Interoperability Remains the Strategic Focus

Perhaps the most important observation is that Iceberg’s momentum increasingly comes from ecosystem adoption rather than any single feature.

Today Iceberg enjoys broad support across:

• Snowflake
• Databricks
• Athena
• Trino
• Dremio
• Flink
• Spark

This reinforces Iceberg’s position as the de facto neutral interoperability layer for the lakehouse ecosystem.

Delta Lake

Delta Lake continues to evolve around the principle that made it successful in the first place: delivering database-style reliability and simplicity on top of cloud object storage.

Recent developments focus on reducing operational burden while improving support for mutable workloads.

Deletion Vectors

Deletion Vectors are arguably the most important Delta innovation since the original transaction log.

Historically, updates and deletes required rewriting entire Parquet files.

Deletion Vectors allow Delta to logically mark rows as deleted while postponing expensive rewrites until later maintenance operations.

The result is:

• reduced write amplification
• faster updates
• improved mutation performance
• lower compute costs for update-heavy workloads

This significantly narrows the historical gap between Delta and Hudi for mutable datasets.

Liquid Clustering

Liquid Clustering may ultimately prove even more consequential.

Traditional lakehouse optimisation strategies often relied on:

• rigid partitioning schemes
• periodic Z-Ordering jobs
• expensive repartitioning exercises

Liquid Clustering introduces a more flexible model where file organization continuously evolves around specified clustering dimensions.

Benefits include:

• reduced dependence on partition design
• simplified operations
• improved pruning effectiveness
• easier adaptation to changing query patterns

In many ways, Liquid Clustering reflects the same industry trend that motivated Iceberg’s hidden partitioning: reducing the operational burden of physical layout decisions.

UniForm

Perhaps the most strategically interesting Delta development is UniForm. UniForm enables Delta tables to automatically expose Iceberg and Hudi-compatible metadata representations without duplicating the underlying data.

This allows organisations to:

• maintain a single source of truth
• support multiple query engines
• reduce format lock-in concerns

The significance of UniForm extends beyond technical interoperability. It reflects a broader realisation that ecosystem compatibility may ultimately matter more than winning individual format battles.

🗂️ Catalogs: The New Battleground

As the table formats themselves converge, competition is increasingly shifting elsewhere.

The most important battleground may no longer be the table format, but the catalog layer that manages metadata, governance and interoperability.

Examples include:

• Apache Polaris
• Project Nessie
• Apache Gravitino
• Unity Catalog
• AWS Glue Catalog

Much like Kubernetes standardised container orchestration while shifting differentiation to surrounding platforms, Open Table Formats may be following a similar trajectory.

The future competitive landscape may be less about whether data is stored as Hudi, Iceberg or Delta and more about which catalog, governance and metadata ecosystem surrounds it.

In that sense, the industry’s centre of gravity appears to be moving up the stack — from storage formats toward metadata, governance and interoperability platforms.

I will be exploring these in depth in one of the upcoming blogs.

🔮Looking Beyond the Triad

One interesting question remains: if Hudi, Iceberg and Delta have spent years converging toward a common feature set, why did a new table format like Apache Paimon emerge at all?

The answer lies in a growing realisation that while the existing formats successfully brought database-like capabilities to data lakes, they were shaped by different priorities — streaming mutations in Hudi, metadata scalability in Iceberg and transactional consistency in Delta. As real-time analytics, streaming databases and Flink-native architectures gained traction, new requirements began to emerge around changelog processing, continuous updates and low-latency data freshness.

Apache Paimon was created in response to these needs. In the next post, we’ll explore what led to its inception, how it differs from the Hudi-Iceberg-Delta triad and whether it represents a new chapter in the evolution of Open Table Formats.

📋References:

*All references have been linked inline throughout the article at the relevant sections.*

Disclaimer: While the ideas and core structure of this blog were conceptualised by the author, AI tools were used to assist in drafting and refining the content and as a sparring assistant.