Episodebeskrivelse
Send us Fan Mail In this episode we’re diving into “Transformer-Squared: Self-Adaptive LLMs” — a new framework for adapting large language models to unseen tasks on the fly by tuning only a small part of their weights. The central idea is Singular Value Fine-Tuning (SVF), a parameter-efficient fine-tuning technique that decomposes each weight matrix with Singular Value Decomposition (SVD) and then only trains a small vector that scales the singular values. These vectors become compact “expert...

