How a Friend Cut a Production Spark Job’s Runtime by 35–40%

A few months ago, a data engineer I know was handed a problem that many in the field will recognise: a Spark pipeline that had quietly…