Skip to content
Global sound
The Databricks Data Engineer

The Databricks Data Engineer

The Databricks pipeline that ran green for a year and was wrong the whole time

· 8 min

Show notes

A revenue-attribution pipeline runs overnight, bronze to silver to gold, and lands a clean set of numbers the whole company uses to decide where the money goes. Every morning: green. Every morning, everyone moves on with their day. For a year, nobody looked any closer.

The job succeeded every single night, so everyone trusted the data. But "finished" and "correct" are two different words, and the space between them had been quietly swallowing a slice of revenue for eleven months, with no error, no warning, and not one alert.

In this episode:
- Why a Databricks job can run green every day…

Website: podcasters.spotify.com

Live, last hour: 14,754 songs on 6,179 stations in 141 countries

Discover the world through radio

More ways to discover (9)

Heard something good? Save it, or open it straight away in Spotify.

Radio RNV ClassicoContinue listening