Langsung ke konten
Global sound
learning with yacine show

learning with yacine show

How to Generate Synthetic Pretraining Data? with Joël Niklaus from Hugginface

· 1 jam 30 mnt

Catatan episode

today we are taking a deeper look at of the secret behind LLM pretraining which is synthetic data pipelines with Joel Niklaus machine learning engineer at Hugging Face!joel an his team ran 90 controlled experiments and burned over a trillion tokens to figure out what actually makes good pretraining data and in huggingface fashion provided all of their findings/artifacts openly!
lots of things to learn from experiment like this and to be honest there is still so much more to do!
couple of useful links:
👉Joel Website: https://niklaus.ai👉 Twitter: https://x.com/joelniklaus👉 Paper : arXiv…

Situs: yacinemahdid.com

Live, satu jam terakhir: 9.331 lagu di 4.470 stasiun di 143 negara

Jelajahi dunia lewat radio

Cara lain untuk menjelajah (7)

Dengar sesuatu yang bagus? Simpan, atau langsung buka di Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Time FM GevgelijaLanjut mendengarkan