Author

Think Data

@think-data

articles: 17
total claps: 401
top claps: 215
avg read: 5.0 min
paywalled: 17
active: 2026-02-05 → 2026-07-12

Wiki topics

🔧 · Data Engineering · 14 🏛️ · Architecture · 3 ☁️ · DevOps & Cloud · 3 🧪 · Chemistry · 1 PRD · Product Design · 1 GRW · Growth & Analytics · 1 🎵 · Music & Audio · 1 💻 · Programming · 1

Top tags

#data-engineering ·13 #big-data ·6 #pyspark ·3 #spark-optimization ·3 #distributed-systems ·3 #kubernetes ·3 #docker ·3 #containers ·3 #software-engineering ·2 #sql ·2

TCH MKT MDA

TH Think Data · Towards Data Engineering

I Bombed the Spotify SQL Interview

I went into the Spotify SQL interview feeling pretty confident. But that confidence was broken in 14 minutes into the interview.

May 24, 2026 · 4 min read · 215 claps paywalled #data-engineering #sql-interview-questions #advanced-sql

TCH

TH Think Data · Towards Data Engineering

Kubernetes From First Principles

How Kubernetes actually keeps distributed systems alive

May 22, 2026 · 7 min read · 53 claps paywalled #kubernetes #kubernetes-cluster #kubectl

TCH

TH Think Data · Towards Data Engineering

A Two-Week Data Migration Investigation Ended with One Line of SQL

How a tiny timezone difference nearly derailed our Hive-to-BigQuery migration

Jun 03, 2026 · 4 min read · 53 claps paywalled #data-engineering #data-migration #bigquery

A Two-Week Data Migration Investigation Ended with One Line of SQL

ART TCH

TH Think Data · Towards Data Engineering

There Are 5 Data Ingestion Patterns — And Choosing Wrong Will Cost You Months

Most engineers don’t think about data ingestion until something breaks.

Jun 09, 2026 · 10 min read · 30 claps paywalled #data-engineering #data-ingestion #big-data

There Are 5 Data Ingestion Patterns — And Choosing Wrong Will Cost You Months

TCH

TH Think Data · Towards Data Engineering

Stop Optimizing BigQuery Until You Run These 5 INFORMATION_SCHEMA Queries

Most BigQuery optimization starts in the wrong place.

Jul 02, 2026 · 4 min read · 12 claps paywalled #bigquery #data-engineering #google-cloud-platform

Stop Optimizing BigQuery Until You Run These 5 INFORMATION_SCHEMA Queries

TCH SCI SOC

TH Think Data · Towards Data Engineering

Cost-Based Optimizer (CBO) in Spark: Statistics, Cardinality Estimation, and Join Selection

Two Spark queries can return exactly the same result.

Jun 18, 2026 · 6 min read · 9 claps paywalled #pyspark #spark-optimization #data-engineering

Cost-Based Optimizer (CBO) in Spark: Statistics, Cardinality Estimation, and Join Selection

TCH

TH Think Data · Data Engineer Things

Facts Don’t Answer Questions. Dimensions Decide Which Ones Exist!

Most confusion in data modeling starts with a simple mistake: we treat facts and dimensions as the same kind of thing.

Feb 05, 2026 · 3 min read · 9 claps paywalled #data-modeling #fact-table #data-engineering

Facts Don’t Answer Questions. Dimensions Decide Which Ones Exist!

ART ECO TCH MDA

TH Think Data · Towards Data Engineering

Don’t Become a “Data Engineer” in 2026 — At Least Not the Way, We Defined It Before

A few years ago, I used to hire data engineers primarily for one thing:

May 16, 2026 · 6 min read · 8 claps paywalled #platform-engineering #data-engineering #lakehouse-architecture

Don’t Become a “Data Engineer” in 2026 — At Least Not the Way, We Defined It Before

TCH

TH Think Data · Towards Data Engineering

The 5 Signals Every Data Engineering Interviewer Is Secretly Scoring

You think you’re being asked about SQL. Most of the time, you’re actually being measured on something else entirely.

Jul 06, 2026 · 5 min read · 5 claps paywalled #data-engineering #software-engineering #interview

The 5 Signals Every Data Engineering Interviewer Is Secretly Scoring

TCH

TH Think Data · Towards Data Engineering

Adaptive Query Execution (AQE): How Spark Re-Optimizes Queries at Runtime

Two Spark queries can return exactly the same result.

Jun 24, 2026 · 4 min read · 2 claps paywalled #data-engineering #apache-spark #spark-optimization

Adaptive Query Execution (AQE): How Spark Re-Optimizes Queries at Runtime

TCH

TH Think Data

Data Engineering’s 10 Silent Killers

In data engineering, the scariest problems are rarely caused by frameworks or languages.

May 24, 2026 · 3 min read · 2 claps paywalled #data-engineering #data-engineer-skills #data-pipeline

TCH DSN

TH Think Data · Towards Data Engineering

There Are 5 Types of CDC Techniques — But Most Engineers Use the Wrong One

Most engineers first encounter CDC (Change Data Capture) through a deceptively simple requirement:

May 28, 2026 · 6 min read · 1 claps paywalled #data-engineering #change-data-capture #design-systems

There Are 5 Types of CDC Techniques — But Most Engineers Use the Wrong One

TCH

TH Think Data · Towards Data Engineering

Docker From First Principles

How isolated Linux processes became portable software

May 20, 2026 · 5 min read · 1 claps paywalled #docker #containers #kubernetes

TCH

TH Think Data · Towards Data Engineering

Why Senior Data Engineers Ask Questions Before They Write SQL

Most candidates treat silence as failure. In a Data Engineering interview, it’s often the opposite.

Jul 12, 2026 · 4 min read · 1 claps paywalled #data-engineering #sql-interview #data-engineer-interview

Why Senior Data Engineers Ask Questions Before They Write SQL

ART DSN TCH

TH Think Data · Towards Data Engineering

Deploying Your First Real Application to Kubernetes

Understanding Deployments, Services, ConfigMaps, and Secrets without the complexity

May 23, 2026 · 5 min read · 0 claps paywalled #kubernetes #containers #microservices

Deploying Your First Real Application to Kubernetes

SCI TCH

TH Think Data · Towards Data Engineering

Catalyst Optimizer in Apache Spark: The Compiler Behind Every DataFrame Query

Most Spark developers know Catalyst as:

Jun 18, 2026 · 5 min read · 0 claps paywalled #pyspark #catalyst-optimizer #spark-optimization

Catalyst Optimizer in Apache Spark: The Compiler Behind Every DataFrame Query

TCH AI

TH Think Data

I Spent 3 Days Fighting a 30GB File… Until Memory Mapping Saved Everything

Our embedding pipeline kept crashing with out-of-memory errors. Then I tried two simple tricks that changed everything. Here’s what…

Jun 03, 2026 · 4 min read · 0 claps paywalled #data-engineering #spark #big-data