Hoppa till innehållet
Global sound
DataTalks.Club

DataTalks.Club

Foundations of Analytics Engineer Role: Skills, Scope, and Modern Practices - Juan Manuel Perafan

· 1 h 23 min · Säsong 23 Avsnitt 2

Avsnittsbeskrivning

In this talk, Juan, Analytics Engineer and author of Fundamentals of Analytics Engineering share his professional journey from studying psychological research in Colombia to becoming one of the first analytics engineers in the Netherlands. We explore the evolution of the role, the shift toward engineering rigor in data modeling, and how the landscape of tools like dbt and Databricks is changing the way teams work.

You’ll learn about:

• The fundamental differences between traditional BI engineering and modern analytics engineering.
• How to bridge the gap between business stakeholders and technical data infrastructure.
• The technical "glue" that connects Python and SQL for robust data pipelines.
• The importance of automated testing (generic vs. singular tests) to prevent "silent" data failures.
• Strategies for modeling messy, fragmented source data into a unified "business reality."
• The current state of the "Lakehouse" paradigm and how it impacts storage and compute costs.
• Expert advice on navigating the dbt ecosystem and its emerging competitors.

Links:

• DE Course: https://github.com/DataTalksClub/data-engineering-zoomcamp
• Luma: https://luma.com/0uf7mmup

TIMECODES:
0:00 Juan’s psychological research and transition to data
4:36 Riding the wave: The early days of analytics engineering
7:56 Breaking down the gap between analysts and engineers
11:03 The art of turning business reality into clean data
16:25 Why data engineering is about safety, not just speed
20:53 Reimagining data modeling in the modern era
26:53 To split or not to split: Finding the right team roles
30:35 Python, SQL, and the technical toolkit for success
38:41 How to stop manually testing your data dashboards
46:34 Bringing software engineering rigor to data workflows
49:50 Must-read books and resources for mastering the craft
55:42 The future of dbt and the shifting tool landscape
1:00:29 Deciphering the lakehouse: Warehousing in the cloud
1:11:16 Pro-tips for starting your data engineering journey
1:14:40 The big debate: Databricks vs. Snowflake
1:18:28 Why every data professional needs a local community

This talk is designed for data analysts looking to level up their engineering skills, data engineers interested in the business-logic layer, and data leaders trying to structure their teams more effectively. It is particularly valuable for those preparing for the Data Engineering Zoomcamp or anyone looking to transition into an Analytics Engineering role.

Connect with Juan

• Linkedin - https://www.linkedin.com/in/jmperafan/
• Website - https://juanalytics.com/

Connect with DataTalks.Club:

• Join the community - https://datatalks.club/slack.html
• Subscribe to our Google calendar to have all our events in your calendar
• https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ- Check other upcoming events
• https://lu.ma/dtc-events- GitHub: https://github.com/DataTalksClub
• LinkedIn…

Webbplats: podcasters.spotify.com

Live, senaste timmen: 8 762 låtar på 4 745 stationer i 122 länder

Upptäck världen genom radion

Fler sätt att upptäcka (7)

Hört något bra? Spara det, eller öppna det direkt i Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Radio Wfm 105.4 - Westerlo, beFortsätt lyssna