Data Engineering Central Podcast
Spark Isn't Going Anywhere. So They Rebuilt It in Rust. — Shehab Amin, CEO of LakeSail
· 57 min
Note dell’episodio
In this episode, I sit down with Shehab Amin, co-founder and CEO of LakeSail, to talk about what happens when you decide to rebuild one of data engineering’s most important technologies, Apache Spark, in Rust … sorta.
We dig into Shehab’s journey from writing C++ as a kid to studying computer science at Berkeley, becoming a founder, and eventually building Sail. We talk about why Spark has remained so sticky, what Rust, Apache Arrow, and DataFusion are changing about data infrastructure, and why rebuilding Spark compatibility turned out to be much harder—and more interesting—than…
Sito web: dataengineeringcentral.substack.com
