Distributed Thinking Emerges Before “Distributed Systems”

While working on Extraction 2, I wasn’t attempting to design a distributed system in the classical sense. The goal was much more pragmatic: extract reliable data from multiple Excel and CSV files whose structure could not be trusted.

The inputs were heterogeneous and semi-structured. Some sheets had empty cells between headers, others used multi-row or merged headers, and many encoded meaning through formatting rather than explicit structure. Column positions were unreliable, and identical column names often carried different semantics across files.

I built the architecture and the foundational codebase to treat each sheet as an independent unit of work. Instead of assuming a global schema, the system performs lightweight structure inference per sheet, followed by isolated extraction. Sheets are chunked and processed in parallel, with explicit boundaries between parsing, validation, and output generation.

Extraction

Parallel execution improved throughput, but more importantly, it enforced fault isolation. A malformed or ambiguous sheet no longer stalled the entire pipeline. At the same time, parallelism surfaced deeper issues: semantic inconsistencies became more visible, validation became harder than extraction, and coordination errors emerged more often than computational ones.

Reading parallel computing theory alongside this work gave language to what I was observing in practice — most failures were not due to insufficient compute, but to weak assumptions about shared structure and meaning.

This experience shifted how I approach systems design. I now treat data extraction as a problem of schema inference under uncertainty, design for partial success, and assume inputs will lie unless proven otherwise.

Distributed thinking, I learned, often begins as a survival strategy long before it becomes an architectural label.