Skip to content
Global sound
The Databricks Data Engineer

The Databricks Data Engineer

How Photon actually makes your Databricks queries faster (and when it silently doesn't)

· 11 min

Show notes

Two engineers run the same SQL on the same Delta table. Same data, same cluster size, copy-pasted code. Alex goes to make a coffee and comes back to a query still running. Sam's is done before they finish reading the first Slack message. The only difference is one checkbox on the cluster called Photon.

Most Databricks data engineers have that box ticked, pay a premium for it on every DBU, and could not explain in plain English what it actually does. "Photon makes it faster" isn't an answer - it's just saying the box does what the box does. And there's one thing it quietly stops doing while…

Website: podcasters.spotify.com

Live, last hour: 14,880 songs on 6,837 stations in 140 countries

Discover the world through radio

More ways to discover (9)

Heard something good? Save it, or open it straight away in Spotify.

Poder 1110Continue listening