Show notes
This episode examines "All for 1-Bit," a paper claiming to compress LLM linear weights to exactly one bit each, metadata included, without retraining. It builds on double binary factorization and challenges earlier "1-bit" post-training methods, which land at roughly 2.5 to 3.5 effective bits per weight once scales, masks and outlier indices are counted. The discussion covers why bandwidth-bound decoding motivates extreme quantization, how a product of two binary matrices adds capacity without adding bits, and how null-space reasoning and Shapley-value layer allocation are meant to spend a fixed rank budget well. Along the way it separates post-training from quantization-aware training and sets up the key skeptical test: whether "genuine" and "usable" both hold, given that a score near chance (about 25 on the zero-shot scale) means a broken model. Listeners get a clear explanation of what "bits per weight" really measures and a framework for judging low-bit quantization claims.
Sources:
1. All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs — Zhixiong Zhao, Zukang Xu, Guangyu Sun, Lifeng Liu, Dawei Yang, 2026
http://arxiv.org/abs/2609.06161
2. BiLLM: Pushing the Limit of Post-Training Quantization for LLMs — Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, Xiaojuan Qi, 2024
https://scholar.google.com/scholar?q=BiLLM%3A+Pushing+the+Limit+of+Post-Training+Quantization+for+LLMs
3. ARB-LLM: Alternating Refined Binarizations for Large Language Models — Zhiteng Li, Xianglong Yan, Tianao Zhang, Haotong Qin, Dong Xie, Jiang Tian, Zhongchao Shi, Linghe Kong, Yulun Zhang, Xiaokang Yang, 2024
https://scholar.google.com/scholar?q=ARB-LLM%3A+Alternating+Refined+Binarizations+for+Large+Language+Models
4. OneBit: Towards Extremely Low-bit Large Language Models — Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, Wanxiang Che, 2024
https://scholar.google.com/scholar?q=OneBit%3A+Towards+Extremely+Low-bit+Large+Language+Models
5. BitNet: Scaling 1-bit Transformers for Large Language Models / The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, Furu Wei (BitNet); Shuming Ma et al. (b1.58), 2023 / 2024
https://scholar.google.com/scholar?q=BitNet%3A+Scaling+1-bit+Transformers+for+Large+Language+Models+%2F+The+Era+of+1-bit+LLMs%3A+All+Large+Language+Models+are+in+1.58+Bits
6. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh, 2022
https://scholar.google.com/scholar?q=GPTQ%3A+Accurate+Post-Training+Quantization+for+Generative+Pre-trained+Transformers
7. SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression — Xin Wang, Yu Zheng, Zhongwei Wan, Mi Zhang…
