OpenSearchCon Europe 2025

In today's rapidly evolving AI landscape, Agentic Retrieval-Augmented Generation (RAG) systems represent a significant leap forward in intelligent search capabilities. This presentation introduces how Agentic RAG enhances traditional RAG by incorporating autonomous agents that can both retrieve and act on information. We'll demonstrate a practical implementation using OpenSearch's vector database, showcasing how this combination enables agents to understand user queries, retrieve relevant information, and execute meaningful actions based on the context.

The presentation will focus on three key areas: the fundamental architecture of Agentic RAG, its integration with OpenSearch vector search capabilities, and practical scaling considerations for enterprise applications. Through demonstration steps, we'll show how this system handles real-world scenarios, highlighting the potential impact on business operations. We'll conclude with insights into deployment strategies and best practices for maintaining system performance at scale, providing attendees with actionable knowledge they can apply in their own organizations.

OpenSearch provides several tools for search and information retrieval, important in many stages of prompt processing. However, to be used in a varied range of industries and regulations such as healthcare, telco or finance, the solution must be proven to be secure, robust and highly available.

In this talk we will discuss how to build a secure conversational chat infra from the ground up to answer your prompts, using OpenSearch. We will explore how we can use existing features of OpenSearch to build the basis of this infra, e.g. vector search and RAG. We will discuss the latest technologies such as confidential computing and hardware acceleration to get the most out of the environment. We will also explore an OpenSearch VM operator that helps with the lifecycle of the cluster. It provides the features needed to scale, upgrade, specialize nodes in different roles according to the needs and manage security such as certificate rotation and user management.

Versioning OpenSearch indexes is critical for maintaining data integrity, ensuring seamless updates, and enabling rollback capabilities in modern CICD pipelines. This presentation explores how Terraform and Python can be combined to manage the versioning of OpenSearch indexes effectively. By leveraging Terraform’s infrastructure-as-code capabilities and Python’s scripting flexibility, teams can automate the lifecycle management of indexes, ensuring consistent reintegration and streamlined operations. Key benefits include enhanced automation, improved traceability, and reduced manual overhead.

Join us for an in-depth exploration of tiered caching in OpenSearch, a powerful search and analytics engine. Caching plays a crucial role in optimizing performance for performance-intensive applications like OpenSearch. This session delves into the limitations of on-heap caching and introduces tiered caching, a multi-layered caching approach that balances performance and capacity. Learn how tiered caching leverages both on-heap and disk-based caches to store larger datasets efficiently without consuming excessive heap memory. Discover the workings of tiered caching, its benefits, and when to implement it. Uncover performance test results comparing tiered caching with the default on-heap cache, and get insights into future enhancements in tiered caching technology. Don't miss this opportunity to enhance your understanding of caching strategies in OpenSearch for improved query response times and application performance.

Performance testing OpenSearch clusters requires careful consideration of numerous variables and methodologies. While OpenSearch Benchmark (OSB) and its associated workloads provide powerful tools for cluster evaluation, many users struggle to implement effective testing strategies that yield meaningful, reproducible results.

In this comprehensive session, we'll dive more deeply into OSB best practices, moving beyond basic benchmarking to explore topics such as how to select and size appropriate workloads, handle common pitfalls and leverage more advanced features like custom workloads and gradual ramp-up to simulate realistic production scenarios. An understanding of critical aspects that are often overlooked such as warm-up periods, proper load generator sizing and techniques to analyze statistical significance will help you obtain actionable insights into how your OpenSearch cluster scales and performs under various operational conditions.

Whether you're a DevOps engineer, database administrator, or performance analyst, this talk will equip you with the tools and methodologies needed to unlock the full potential of your OpenSearch deployment.

Imagine deploying OpenSearch to power search and analytics for millions of users—only to find unexpected roadblocks waiting at every turn. From managing massive data volumes to tuning performance under tight SLAs, the journey was far from smooth but rich with lessons. In this talk, we’ll take you behind the scenes of our real-world OpenSearch implementation, sharing war stories, hard-won insights, and practical strategies that helped us turn chaos into a well-oiled machine.

Through interactive storytelling, we’ll explore:
• How we scaled OpenSearch clusters to handle unpredictable growth.
• The pitfalls we encountered (and resolved) with indexing, querying, and resource optimization.
• Real examples of troubleshooting and automation techniques that saved us time and effort.

Whether you’re just starting with OpenSearch or already managing production workloads, this session is your chance to peek into our playbook, ask questions, and leave with actionable takeaways to conquer your operational challenges.

One of the really cool aspects of OpenSearch is its ability to serve as a monitoring tool for itself. Let's have a look at how to piece together a monitoring solution for OpenSearch using components from the OpenSearch ecosystem such as Data Prepper and OpenSearch alerting. We will focus on security aspects like authentication and authorization.

Any software dealing with confidential data needs to have a security solution providing authentication and authorization. And any such security solution should have proper monitoring. OpenSearch is no exception in this regard.

In this presentation, we will look how to use components like Audit Logs, Data Prepper and OpenSearch Alerting to create a small, but effective security monitoring solution.

This will give us two benefits at once:

- A security monitoring solution
- Insights into the workings of components like Alerting, etc. by looking at a real world example

Many OS users, especially those new to Lucene-based search engines, struggle with sharding best practices. This often results in inefficient shard sizes, leading to performance issues and resource waste. Our talk introduces a novel approach: Dynamic Sharding Strategy.

We'll explore how this strategy adaptively determines the optimal primary shard count for indices, particularly beneficial for time-series data with fluctuating ingestion rates. By analyzing past ingestion rates, cluster resources, and workload patterns, our method predicts and applies suitable shard counts for each index rollover.

We'll discuss
* Common sharding pitfalls and their impact
* The idea behind our dynamic sharding approach
* Implementation as a Python script integrated with OpenSearch
* Real-world case studies and performance improvements
* Potential for native integration into OpenSearch's Index State Management

This talk aims to empower OpenSearch users with a practical solution for maintaining optimal cluster performance, reducing storage skewness, and minimizing manual interventions in shard management. Join us to learn how dynamic sharding can revolutionize your OpenSearch cluster efficiency

This talk will explore how OpenSearch can be leveraged to build a centralized and multi-tenant log analysis platform that provides logging-as-a-service to several Kubernetes enterprise environments.

We’ll address solutions for critical topics like cluster and tenant separation strategies, Kubernetes integration, automation, capacity management and monitoring best practices. Additionally, we’ll present our approach to non-technical challenges such as handling diverse tenant requirements, effective customer support and different areas of tenant collaboration.

Throughout the talk, there will be a strong emphasis on standardization as the primary solution component to achieve heterogeneity within the different configuration domains (retention policies, RBAC, etc.), while simultaneously maintaining the flexibility to be able adapt to customers' needs.

Drawing experience from a recent real-world implementation, this session will provide attendees with actionable insights and our lessons learned to overcome technical and operational challenges in building multi-tenant logging platforms using OpenSearch.