Notatki do odcinka
Most language models begin with a tokenizer that cuts text into predetermined pieces. What if the model could learn those pieces for itself? Cartesia’s hierarchical networks, or H-Nets, start with raw bytes and build useful chunks inside the model. We explore how this learned hierarchy can compress familiar patterns, preserve important detail, and even change its segmentation as new words and repeated structures appear within a passage. Cartesia reports promising efficiency gains in controll...

