Row-Based vs Columnar Execution in Spark: A Practical Case Study at Scale

Large analytical datasets often behave very differently once they cross a certain size threshold. Execution characteristics that are mostly invisible at millions of rows become measurable and observable at billions.

This article presents a practical case study comparing row-oriented execution and columnar-oriented execution on Apache Spark using the same dataset, SQL, and cluster configuration. The intent is not to evaluate or rank technologies, but to document how different execution models behave across workload shapes.

Dataset and Methodology

• Dataset: Sales fact table (~20 billion rows total)
• Format: Parquet
• Files scanned per query: 1,524
• Rows scanned per query: ~2 billion
• Cluster configuration and SQL text: unchanged across runs

Three workload shapes were evaluated:

1. Scan-only (baseline)
2. Medium aggregation
3. Heavy aggregation

Row-Based and Columnar Execution (Short Context)

Spark supports columnar storage formats such as Parquet and ORC, but execution and storage format are not the same thing.

Row-Based Execution (Scalar Processing)

• Data flows through operators one row at a time
• Operators work on InternalRow
• Shuffle serializes individual rows
• Simple mental model, widely supported

Columnar Execution (Vectorized Processing)

• Operators process batches of columnar vectors
• Expressions evaluated on batches
• Shuffle operates on columnar batches
• Different memory layout and serialization path

Both execution styles can exist within the same query plan, depending on operator support.

Workload 1: Scan-Only (Baseline)

This workload reads the full dataset and performs minimal transformation.

Query

select count(1) from (SELECT SUM(qty), SUM(price), COUNT(*) FROM spark_catalog.default.stage_sales_1 WHERE sale_date BETWEEN DATE ‘2024–01–01’ AND DATE ‘2024–12–31’);

Observed Behavior

• Scan dominates total execution time
• No predicate pruning
• Negligible aggregation and shuffle

Observed runtime pattern

• Row-oriented execution shows lower reported scan time
• Columnar execution reports higher scan + decode time

Spark scan metrics

Comet scan metrics

Why this matters This workload establishes a baseline where execution-model differences do not accumulate. Most time is spent in I/O and decoding.

Workload 2: Medium Aggregation (GROUP BY customer_id)

This workload introduces a moderate aggregation and a single shuffle.

Query

select count(1) from (SELECT customer_id,SUM(qty price) AS revenue, COUNT() AS orders, AVG(price) AS avg_price FROM spark_catalog.default.stage_sales_1 WHERE sale_date BETWEEN DATE ‘2024–01–01’ AND DATE ‘2024–12–31’ GROUP BY customer_id);

Runtime Comparison

Runtime comparison

Hash Aggregate

Comet Aggregate

Spark Exchange

Comet Exchange

Exchange Metrics (Key Observation)

Although the logical shuffle is identical, the reported physical data size differs, indicating differences in how data is represented and moved.

Workload 3: Heavy Aggregation (GROUP BY customer_id, item_id)

This workload increases aggregation cardinality and shuffle pressure.

Query

select count(1) from (SELECT customer_id,item_id,SUM(qty price) AS revenue, COUNT() AS orders FROM spark_catalog.default.stage_sales_1 WHERE sale_date BETWEEN DATE ‘2024–01–01’ AND DATE ‘2024–12–31’ GROUP BY customer_id, item_id);

Runtime Comparison

Exchange Metrics (Heavy Aggregation)

The difference in shuffle data size is more pronounced than in medium aggregation, even though both executions remain stable.

Query Runtime Comparison

Shuffle Volume Comparison

These graphs summarize how behavior changes as the workload shifts from scan-dominated to aggregation- and shuffle-dominated.

Interpreting the Results (Without Conclusions)

Across the three workloads:

• Scan-only queries are dominated by I/O and decoding
• Aggregation-heavy queries spend more time in:
• Aggregation state management
• Shuffle serialization and deserialization
• Execution-model differences become visible primarily at:
• Aggregation operators
• Exchange boundaries

The same SQL can therefore behave differently depending on where the workload spends most of its time.

Why This Is a Case Study, Not a Benchmark

• Single dataset
• Single cluster configuration
• No tuning for either execution path
• Metrics observed, not normalized

The intent is to provide concrete data points, not generalized performance claims.

Closing Notes

Row-based and columnar execution models coexist in modern analytical engines. This case study shows that their characteristics surface differently depending on workload shape and scale.

Readers are encouraged to interpret these results in the context of their own data, queries, and operational constraints.

This article was co-authored by Kuladeep Sandra