OpenSearchCon Europe 2025

With the rapid rise of Large Vision Language Models, radically new approaches to multimodal information retrieval have been proposed. Methods like ColPali and Document Screenshot Embeddings bypass the classical document preprocessing pipeline by representing document pages only in terms of vision based image patches. This allows unprecedented accuracy for Retrieval Augmented Generation (RAG) in answering questions about figures, tables, charts and visually complex documents like Powerpoint slides, without the hassle of complex customized extraction pipelines. And unlike earlier multimodal approaches based on CLIP (Contrastive Language-Image Pre-Training) the new approach also enables retrieval of detailed text passages from normal text documents. In this talk we will present how these methods can be implemented using OpenSearch, how this compares to other implementations, and evaluate how well the resulting multimodal RAG system performs in terms of search effectiveness and efficiency.