Beyond Text: Can AI Search Understand Images Too?

This podcast introduces MMSEARCH, a new benchmark designed to evaluate how well large multimodal models (LMMs) can function as AI-powered search engines that understand both text and images. The authors argue that existing AI search engines are limited by their focus on text-only settings, neglecting the wealth of information found in images and the way text and images are combined on websites. To address this, they created MMSEARCH, a dataset of 300 diverse search queries spanning 14 subfields, ensuring the answers cannot be found within the training data of current LMMs. They also propose…