Skip to content
Global sound
AI Illuminated

AI Illuminated

Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers

· 11 min · Season 1 Episode 22

Show notes

[00:00] Intro

[00:21] Key problem: Poor generalization in robotic learning

[00:51] HPT: New transformer architecture for robotics

[00:59] Core components of HPT architecture

[01:44] Scale analysis: Data and model size impacts

[02:16] Training data: Real robots, simulations, human videos

[02:54] Results: 20% improvement on new tasks

[04:04] Real-world testing limitations

[05:18] Future additions: Tactile and 3D data

[05:57] Requirements for better robotics datasets

[06:48] Weight sampling in heterogeneous data

[08:55] Benefits of modular architecture

[10:30] Scaling challenges and…

Website: podcasters.spotify.com

Live, last hour: 7,139 songs on 2,688 stations in 113 countries

Discover the world through radio

More ways to discover (9)

Heard something good? Save it, or open it straight away in Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
The Disco PhilharmonicContinue listening