OpenSearchCon Europe 2025

It’s safe to say Unconference is now an OpenSearchCon tradition. Each time the community gets together for this event, we learn from one another in an engaging environment with interesting talks and open discussions.

Make plans to join us again and come ready to pitch your favorite speaking topic! This is an opportunity for the community to come together and kick off OpenSearchCon Europe with an action-packed morning of sharing and discovery. With no pre-planned talks, what you want to hear about will be determined by you and your fellow conference-goers.

Attendees/Speakers
This is a first-come, first-served event; the room holds up to 100 individuals and anyone registered for OpenSearchCon is welcome to attend until the room is full. We advise you to show up a little early to make sure you get your spot.

Each speaker has 15 minutes to do with as they see fit! Want to talk for 10 minutes and have 5 minutes of questions? Great! Have a lot to say and want to talk for the whole 15? That works too! Are you a maintainer and want to hold a lightning round of questions with the audience? Fantastic! You get the picture. Just be mindful of your 15 minutes!

Voting
- At 10:20 AM, each attendee will receive a card and three gold-star stickers. Those who would like to give a talk will write their title and brief description on the card, put their name on the back, and post them to the board for voting.
- At 10:35, you’ll have 15 minutes to walk the board and place a gold star on one of the talks you would like to hear. One rule: please do not vote for yourself.
- At 10:55, our host, Kris Freedain, and a lucky volunteer will collect and sort the cards, then select the day’s talks from the cards receiving the most votes while ensuring the widest arrangement of topics are covered.

Presentations
At 11:00, we will return the cards to the wall in the order each talk will be given. Be ready to talk and participate!

Dive deep into the cutting-edge strategies for designing robust, highly available OpenSearch clusters that maintain peak performance even in the face of infrastructure challenges. This session will showcase how to leverage the latest OpenSearch features to create a resilient search & ingestion architecture that can withstand various failure scenarios.
We'll explore the synergistic benefits of recently introduced capabilities, including:

1. Weighted Round Robin: Learn how to optimize request distribution across nodes, ensuring balanced load and improved cluster stability.
2. Gray Disk Detection: Understand how to preemptively identify and mitigate potential disk failures before they impact cluster health.
3. Search & Indexing Backpressure Mechanisms: Master the art of gracefully handling query overloads to maintain consistent search & indexing performance under high demand.


Whether you're managing mission-critical search infrastructure or planning to scale your OpenSearch implementation, this session will equip you with the knowledge to architect highly available clusters that deliver reliable performance in the face of unforeseen challenges.

There are lots of statistics we can calculate for search relevance: NDCG, precision, MAP, …. OpenSearch 3.0 calculates and displays them for you.

But metrics don’t tell the whole story. In particular, NDCG can’t give you the qualitative analysis that one algorithm is great for author names, and a different one is good for shapes and sizes. For that, you need eyeballing, that is, human intuition and judgment.

That’s why we’ve built tools to help you dig deeper into your search results. The Search Comparison tool in OpenSearch 2.11 is just the starting point – in 3.0, it lets you define search configurations and test sets of multiple queries and determine which queries’ results have changed the most… and then zoom in to the individual cases. You can integrate data on judgments and query frequency from User Behavior Insights for deeper analysis.

In this talk, we’ll discuss how traditional statistical metrics are complemented by eyeballing, how the new tools in OpenSearch 3.0 support disciplined eyeballing, and how you can use them to improve the quality of your search results.

Every OpenSearch workload behaves and evolves differently. Users strive to understand their production workload’s behavior by running performance tests. But this is often done with generic data and basic queries, which can inaccurately represent their use case. The best way to determine its behavior is by benchmarking with real-life data and customer queries. To do this, users will need to recreate their actual workload, which is difficult as there are many factors to consider. Although it seems daunting, recreating production workloads and their behaviors do not require scientific methodologies and formulas but rather techniques and intuition.

In this interactive session, we’ll use a feature in OpenSearch Benchmark to convert real customer data into a sharable custom workload that recreates patterns and behaviors seen in production environments. We’ll also show how that custom workload can be used to save time, discover optimizations, and lower costs.

After the session, attendees will have developed intuition on workload dynamics, gained practical skills they can apply to their own environments, and understand that recreating workload behavior is an art, not a science.

OpenSearch Data Prepper is a pipeline tool within the OpenSearch project for data ingestion. A common question that comes up within the open-source community is how it relates to Logstash. While both tools are open-source and designed to collect, transform, and route data from various sources to destinations, they differ significantly in their features and target use cases.

Logstash, a mature platform with a vast plugin ecosystem, provides flexible and extensive customization options. Data Prepper is a newer offering initially designed for modern observability needs, such as handling OpenTelemetry (OTel) logs, traces, and metrics. Over time, it has evolved into a general-purpose tool capable of interfacing with numerous open-source and proprietary solutions.

In this presentation, we will explore the architectural differences between Data Prepper and Logstash. We’ll also conduct a comparison of their core features, emphasizing Data Prepper's unique capabilities that enable integration with tools such as OpenTelemetry, OpenSearch, Kafka, S3, DynamoDB, and Kinesis. Finally, we'll provide practical guidance and examples to facilitate a smooth migration from Logstash to Data Prepper.

In this presentation, we’ll go over a real world scenario where we used OpenTelemetry to collect data from logs, metrics and traces and OpenSearch to store and search them. We were involved in the migration to OpenSearch in a food delivery company with close to a million orders a day. To observe the performance of its microservices, a large amount of data was collected and monitored in real time on dashboards.

In the checkout flow from adding an item in the cart to placing an order, several teams were involved. They had their own microservices and dashboards to observe them. So to trace any bug, the teams were relying on an internal order id. This id was not part of the contract of all services and thus made it hard to debug.

Here’s when we introduced distributed tracing using context across all microservices. We migrated to OpenTelemetry and OpenSearch and made changes in the logs, metrics and traces of all services across teams. This made it easier to search logs and look for issues effectively. In this discussion, we’ll discuss this scenario and the challenges we faced as part of implementing it. We will also discuss the benefits of this approach we observed afterwards.

Performance testing OpenSearch clusters requires careful consideration of numerous variables and methodologies. While OpenSearch Benchmark (OSB) and its associated workloads provide powerful tools for cluster evaluation, many users struggle to implement effective testing strategies that yield meaningful, reproducible results.

In this comprehensive session, we'll dive more deeply into OSB best practices, moving beyond basic benchmarking to explore topics such as how to select and size appropriate workloads, handle common pitfalls and leverage more advanced features like custom workloads and gradual ramp-up to simulate realistic production scenarios. An understanding of critical aspects that are often overlooked such as warm-up periods, proper load generator sizing and techniques to analyze statistical significance will help you obtain actionable insights into how your OpenSearch cluster scales and performs under various operational conditions.

Whether you're a DevOps engineer, database administrator, or performance analyst, this talk will equip you with the tools and methodologies needed to unlock the full potential of your OpenSearch deployment.

Search engines guide users from their initial goals to the information that fulfills those goals. The retrieved items are deemed relevant when they align with the user's intent. Ensuring relevant results involves having the data in the catalog, analyzing it accurately, representing it through text and vectors, ranking it appropriately, and performing other actions.
In this session, discover how auction houses utilize vector search, powered by Amazon OpenSearch Service, to find similar items within their extensive database, enhancing the customer experience and preventing issues with counterfeit artworks, and see how it assists users in obtaining the most relevant results.
This session also uncovers some peculiarities in getting embeddings with AWS Bedrock and AWS SageMaker serverless inference for the abovementioned goals.

Dive deep into OpenSearch workload management and optimize your cluster's performance and resource allocation. This talk is suitable for both seasoned users and newcomers to OpenSearch.

We'll cover workload management fundamentals, exploring core concepts and principles specific to OpenSearch environments. You'll discover the benefits through real-world scenarios. We'll also guide you through implementation strategies, offering best practices and helping you avoid common pitfalls.

Gain insights into the various resource types supported by OpenSearch's workload management feature, enabling informed allocation decisions. We'll update you on current capabilities and provide a glimpse into the exciting future roadmap of this feature.

By attending, you'll gain actionable insights for immediate application, learn to address common OpenSearch challenges. This talk is ideal for OpenSearch administrators, DevOps professionals with large-scale search and analytics, IT decision-makers, and open-source enthusiasts.

Join us to master WLM in OpenSearch. Enhance your skills, optimize your clusters, and become part of the vibrant OpenSearch community driving innovation in search technology.

FinOps has revolutionized cloud cost management, with many vendors emerging over the years to offer tools to curb skyrocketing cloud expenses. However, in distributed organizations, where teams independently adopt observability & cloud-native tooling, cost control becomes even more challenging without centralized strategies.

This talk explores how policies can enable distributed teams to balance innovation with financial accountability. We'll examine the unique challenges of managing o11y costs in these environments, where usage often grows unpredictably due to independent decisions across teams. Starting with an overview of how observability tool pricing structures create cost variability, we'll discuss key policy-driven strategies for defining data retention limits, curbing over-collection, and aligning team usage with organizational budgets.

We'll dive into real-world examples of resource-intensive services that can quickly inflate observability costs, offer benchmarks to guide cost-effective observability practices and how to codify these into cross-organizational policies, and reveal whether your teams might be unintentionally "spamming" o11y systems with unnecessary data.

This session will provide an overview of key advancements in OpenSearch Dashboards delivered in 2024, including the launch of the next-generation OpenSearch UI. Highlights include multiple data source integration, OpenSearch Workspaces for tailored collaboration, and an enhanced Discover experience featuring natural language query capabilities. These updates have streamlined workflows, improved user experience, and expanded data visualization options.

Looking ahead, the session will outline the roadmap for OpenSearch Dashboards, focusing on enhancing user workflows, improving integration across observability modules, and simplifying data management. Attendees will gain insights into the platform’s direction and how it aims to address common challenges, including fragmented workflows, feature silos, and configuration complexities, to deliver a more unified and efficient analytics experience.

I'm breaking down key insights from our latest RFC that's shaping the future of core OpenSearch performance.
https://github.com/opensearch-project/OpenSearch/issues/16841

It touch on core fundemental issues that are essential to master to take the OpenSearch Core performance codebase to the next level.
In particular it surveys real live examples from OpenSearch and shows us patterns that are common in Operating systems Kernels and State of the art databases for reliable low latency high throughput performance.
The talk also provides background on core fundemental concepts for data plane engineering such as "Dynamic Memory Allocation", GC, GC tunning, Circular buffers, CPU branching, JIT, JMH profiling.

Geared towards developers looking to contribute to a new or existing OpenSearch or OpenSearch Dashboards plugin, this talk will explore the testing practices of the OpenSearch project in depth. We'll delve into the functional test repository and Cypress standards used for testing Dashboards plugins, providing insights into setting up robust and maintainable test suites. On the backend side, we'll cover unit, integration, and backward compatibility (BWC) testing strategies for OpenSearch plugins, ensuring stability and seamless upgrades. We'll also discuss the use of yamlRestTest for validating REST APIs and remote cluster testing to ensure seamless interoperability during development and release cycles. Finally, we'll examine the TestClusters framework from OpenSearch Core, showcasing how it simplifies testing in distributed environments. Whether you're a seasoned contributor or new to the ecosystem, this talk will equip you with the knowledge to implement effective testing in your plugins and contribute with confidence.

Forget the old search paradigms—AI is here to change everything! By blending sparse (traditional term-based) and dense (vectorized) search methods, OpenSearch creates a hybrid solution that optimizes search results, from precision to performance.

Learn how to implement hybrid search, combine neural embeddings from LLMs with traditional search techniques, and enhance scalability. This solution bridges the gap between legacy systems and AI-driven search, improving accuracy and relevance while scaling to enterprise-level needs. Transform how you search, analyze, and scale with OpenSearch.

This talk will dive deep into building a multi-region, highly scalable enterprise-grade logging platform on OpenSearch. The platform can scale to handle over 100 terabytes of log ingestion per day and store over a petabyte of log data. The speaker will discuss the key challenges encountered when building a logging solution at this scale, and share the solutions leveraging open-source components like OpenTelemetry agents for log ingestion and Data Prepper for aggregation, distillation, enrichment, and buffering. Attendees will learn best practices for constructing a comprehensive, resilient logging infrastructure on top of the OpenSearch ecosystem.