OpenSearchCon Europe 2025

Search engines take in a query and return a set of results that best satisfy the need of the searcher. Words from a text box, user interface facets, and some user identity information have been the clues for good retrieval. Today, Large Language Models (LLMs) produce vectors with semantic information that augments or replaces lexical information for retrieving best results. OpenSearch's low-level APIs can connect with and use LLMs as part of an overall search solution. A full solution still requires some higher-level coding to take full advantage of these APIs. In this talk, we’ll cover some theory on similarity, and how and where vectors can help you retrieve better results. We’ll dive deep on opensearch-ml-quickstart (https://github.com/Jon-AtAWS/opensearch-ml-quickstart), an open-source example set for managing connections with OpenSearch, local models, and vector embeddings. We’ll walk through examples and code to help you get started quickly with OpenSearch’s vector capabilities: exact nearest-neighbor matching, dense vector embeddings, and sparse vectors. You’ll leave with a great tool in opensearch-ml-quickstart, along with the knowledge of how to apply it.

Forget the old search paradigms—AI is here to change everything! By blending sparse (traditional term-based) and dense (vectorized) search methods, OpenSearch creates a hybrid solution that optimizes search results, from precision to performance.

Learn how to implement hybrid search, combine neural embeddings from LLMs with traditional search techniques, and enhance scalability. This solution bridges the gap between legacy systems and AI-driven search, improving accuracy and relevance while scaling to enterprise-level needs. Transform how you search, analyze, and scale with OpenSearch.

In a large organisation at scale, collaborating effectively on analytics and observability is a major challenge. Sharing the right set of permissions with the right stakeholders is a critical task. OpenSearch Workspace, a powerful new feature aims to make this collaboration much easier.

This session will dive into how OpenSearch Workspace can empower your team to efficiently share insights, visualise data and help drive faster decisions. We’ll go over the basics of a workspace, practical use cases, including shared dashboards and granular access control, and also have a hands-on demo of setting up a workspace.

Attendees can expect to leave with a deep understanding of:

- How OpenSearch Workspaces function and their role in the
ecosystem
- Configuring and optimising workspaces for diverse use cases.
- Setting the right level of access for workspaces
- Getting Workspace up and running in your team.

Whether you're an engineer or a data scientist, this session will leave you equipped to enhance collaboration in your organisation and get the most out of OpenSearch Workspace.

How do you know whether your applications are running smoothly or struggling? OpenSearch can provide insights and analyze telemetry data of your applications. Modern observability revolves around three core signal types: logs, metrics, and traces. This talk demonstrates how to instrument your applications and collect these signals using OpenTelemetry, enabling comprehensive insights into application performance and health.
We explain how to map the data in a way to easily correlate logs, metrics, and traces. Building on these correlations we present how OpenSearch can be leveraged to generate advanced insights. Explore with us how far this correlated approach can exceed solutions focussing on just logs or traces.
We'll start by exploring how to process and index telemetry data with Data Prepper, making it ready for analysis, dashboarding, and alerting within OpenSearch.
You’ll learn effective strategies to map and correlate logs, metrics, and traces, unlocking deeper relationships between these signals.
Building on these correlations, we’ll showcase how OpenSearch can deliver advanced insights, enabling you to diagnose problems faster and make informed decisions.

OpenSearch Data Prepper is a pipeline tool within the OpenSearch project for data ingestion. A common question that comes up within the open-source community is how it relates to Logstash. While both tools are open-source and designed to collect, transform, and route data from various sources to destinations, they differ significantly in their features and target use cases.

Logstash, a mature platform with a vast plugin ecosystem, provides flexible and extensive customization options. Data Prepper is a newer offering initially designed for modern observability needs, such as handling OpenTelemetry (OTel) logs, traces, and metrics. Over time, it has evolved into a general-purpose tool capable of interfacing with numerous open-source and proprietary solutions.

In this presentation, we will explore the architectural differences between Data Prepper and Logstash. We’ll also conduct a comparison of their core features, emphasizing Data Prepper's unique capabilities that enable integration with tools such as OpenTelemetry, OpenSearch, Kafka, S3, DynamoDB, and Kinesis. Finally, we'll provide practical guidance and examples to facilitate a smooth migration from Logstash to Data Prepper.

In this presentation, we’ll go over a real world scenario where we used OpenTelemetry to collect data from logs, metrics and traces and OpenSearch to store and search them. We were involved in the migration to OpenSearch in a food delivery company with close to a million orders a day. To observe the performance of its microservices, a large amount of data was collected and monitored in real time on dashboards.

In the checkout flow from adding an item in the cart to placing an order, several teams were involved. They had their own microservices and dashboards to observe them. So to trace any bug, the teams were relying on an internal order id. This id was not part of the contract of all services and thus made it hard to debug.

Here’s when we introduced distributed tracing using context across all microservices. We migrated to OpenTelemetry and OpenSearch and made changes in the logs, metrics and traces of all services across teams. This made it easier to search logs and look for issues effectively. In this discussion, we’ll discuss this scenario and the challenges we faced as part of implementing it. We will also discuss the benefits of this approach we observed afterwards.

Performance testing OpenSearch clusters requires careful consideration of numerous variables and methodologies. While OpenSearch Benchmark (OSB) and its associated workloads provide powerful tools for cluster evaluation, many users struggle to implement effective testing strategies that yield meaningful, reproducible results.

In this comprehensive session, we'll dive more deeply into OSB best practices, moving beyond basic benchmarking to explore topics such as how to select and size appropriate workloads, handle common pitfalls and leverage more advanced features like custom workloads and gradual ramp-up to simulate realistic production scenarios. An understanding of critical aspects that are often overlooked such as warm-up periods, proper load generator sizing and techniques to analyze statistical significance will help you obtain actionable insights into how your OpenSearch cluster scales and performs under various operational conditions.

Whether you're a DevOps engineer, database administrator, or performance analyst, this talk will equip you with the tools and methodologies needed to unlock the full potential of your OpenSearch deployment.

In this session, Uber, a premier member of the OpenSearch Foundation, will share how the unique demands of their low-latency, high-throughput search platform have driven impactful contributions to the OpenSearch project.

The discussion will explore Uber’s collaboration with the OpenSearch community to integrate critical learnings and innovations from their own search platform. Key highlights include the development of a modular, cloud-native architecture with clear separation between readers and writers, a high-performance messaging protocol for inter-service communication, and a pull-based indexing model.

These advancements, along with contributions from the broader OpenSearch community, are shaping OpenSearch into a cutting-edge, serverless open-source search engine tailored for enterprise needs. Join this session to discover how pioneering work from Uber and OpenSearch is paving the way for the next generation of high performance and scalable search technology in open source.

Leveraging OpenSearch, we propose a Cross-Reference Code Mapping System to dynamically map medical products across terminologies and regulatory frameworks. For example, it maps Paracetamol (UK) to Acetaminophen (US), including dosage details and regulatory codes. Using nested documents, the system efficiently stores and retrieves all related data. Semantic search enables queries like "pain relief tablets," returning relevant matches across terminologies.

Additionally, by incorporating Cross-Cluster Search, the system seamlessly queries datasets stored across multiple clusters, such as those dedicated to specific regions or regulatory frameworks. This ensures unified data access while maintaining localized storage and compliance with regional regulations. Cross-Cluster Search also ensures scalability, allowing new datasets or regions to be added incrementally without disrupting operations.

Powered by OpenSearch, this system improves data interoperability, enhances medication accuracy, reduces errors, and streamlines research workflows. Its scalable design adapts to the evolving needs of the global healthcare community.

Imagine deploying OpenSearch to power search and analytics for millions of users—only to find unexpected roadblocks waiting at every turn. From managing massive data volumes to tuning performance under tight SLAs, the journey was far from smooth but rich with lessons. In this talk, we’ll take you behind the scenes of our real-world OpenSearch implementation, sharing war stories, hard-won insights, and practical strategies that helped us turn chaos into a well-oiled machine.

Through interactive storytelling, we’ll explore:
• How we scaled OpenSearch clusters to handle unpredictable growth.
• The pitfalls we encountered (and resolved) with indexing, querying, and resource optimization.
• Real examples of troubleshooting and automation techniques that saved us time and effort.

Whether you’re just starting with OpenSearch or already managing production workloads, this session is your chance to peek into our playbook, ask questions, and leave with actionable takeaways to conquer your operational challenges.

One of the really cool aspects of OpenSearch is its ability to serve as a monitoring tool for itself. Let's have a look at how to piece together a monitoring solution for OpenSearch using components from the OpenSearch ecosystem such as Data Prepper and OpenSearch alerting. We will focus on security aspects like authentication and authorization.

Any software dealing with confidential data needs to have a security solution providing authentication and authorization. And any such security solution should have proper monitoring. OpenSearch is no exception in this regard.

In this presentation, we will look how to use components like Audit Logs, Data Prepper and OpenSearch Alerting to create a small, but effective security monitoring solution.

This will give us two benefits at once:

- A security monitoring solution
- Insights into the workings of components like Alerting, etc. by looking at a real world example

In the era of vector search and semantic similarity, returning highly relevant results is only half the battle. When search results are too similar, users must wade through redundant information to find diverse perspectives. This talk introduces Maximum Marginal Relevance (MMR) implementation in OpenSearch, a powerful technique that optimally balances result relevance with diversity.

Key takeaways will include:
- Implementing MMR reranking with OpenSearch's vector search
- Optimizing performance for large-scale deployments
- Measuring and tuning diversity metrics
- Real-world applications and success patterns

In the realm of search, Machine Learning plays a pivotal role in enhancing the user experience throughout the entire lifecycle, from ingesting documents to delivering highly relevant results for user queries. This session will showcase various ML integrations of OpenSearch tailored to optimize outcomes for user queries in a retail scenario, accompanied by a live demonstration. We will explore cutting-edge techniques such as query understanding and rewriting using Large Language Models (LLMs), document enrichment, sparse, dense, and hybrid retrievers, as well as contextual re-ranking of results. As a highlight, you will learn how LLM agents and OpenSearch’s ML Search types can be integrated to build a AI shopping assistant that offers a conversational shopping experience for the users.

Scaling vector search to a billion vectors isn’t just a technical challenge—it’s a high-stakes balancing act of latency, cost, and quality. In this talk, I’ll share practical lessons learned from building a billion-scale system in OpenSearch, covering model selection, text processing, hardware essentials, and embedding strategies. But that’s not all: I’ll expose the tips, tricks, and gotchas you’ll need to navigate OpenSearch’s quirks without losing your sanity (or your wallet). Whether you’re scaling up or just getting started, this session will equip you to tame your vectors and succeed at scale.