Zum Inhalt springen
Global sound
The Databricks Data Engineer

The Databricks Data Engineer

Two Databricks jobs, one Delta table, same moment: 3 endings (and the 1 that needs a human)

· 12 Min.

Shownotes

Two in the morning. A nightly merge is folding yesterday's orders into a Delta table. A backfill someone kicked off before logging off is correcting March. Neither job knows the other exists, and nothing in Delta would tell them.

A lot of design around this runs on superstition. Either "it's Delta, it's ACID, relax." Or "never let two jobs touch that table." Both are guesses, and the tidy middle version, same data collides, different data is fine, falls apart the first time the error shows up.

In this episode:
- What Delta actually does when the second job tries to commit to a table that…

Website: podcasters.spotify.com

Live, letzte Stunde: 13.860 Songs auf 6.972 Sendern in 141 Ländern

Entdecke die Welt über das Radio

Mehr zum Entdecken (9)

Etwas Schönes gehört? Speicher es oder öffne es direkt in Spotify.

marcradioWeiterhören