OpenSearchCon Europe 2025

The standard way for creating APIs is to first define the API contract, use that to generate classes and write your application around it. But what happens when the application is already in place and is actively being used by millions around the world and does not have an API contract?

We faced this issue when we set out to find a single source of truth for OpenSearch APIs. Finding none and realizing that we need to create one, we set out on a journey to create asuch specification.

This talk will deal with:
* Experience with Jest client and frustration with freeform request bodies as a customer
* The stages of development including ideation, PoC, RFC, choosing the technologies, gaining consensus and implementation
* setting up the project and integration framework (while ensuring frugality and complete coverage)
* Opening up the project to community contribution after initial implementation
* Adoption by the clients team
* Switch from Smithy to OpenAPI based on community discussion
and the FUTURE!

This session will provide an overview of key advancements in OpenSearch Dashboards delivered in 2024, including the launch of the next-generation OpenSearch UI. Highlights include multiple data source integration, OpenSearch Workspaces for tailored collaboration, and an enhanced Discover experience featuring natural language query capabilities. These updates have streamlined workflows, improved user experience, and expanded data visualization options.

Looking ahead, the session will outline the roadmap for OpenSearch Dashboards, focusing on enhancing user workflows, improving integration across observability modules, and simplifying data management. Attendees will gain insights into the platform’s direction and how it aims to address common challenges, including fragmented workflows, feature silos, and configuration complexities, to deliver a more unified and efficient analytics experience.

One of the really cool aspects of OpenSearch is its ability to serve as a monitoring tool for itself. Let's have a look at how to piece together a monitoring solution for OpenSearch using components from the OpenSearch ecosystem such as Data Prepper and OpenSearch alerting. We will focus on security aspects like authentication and authorization.

Any software dealing with confidential data needs to have a security solution providing authentication and authorization. And any such security solution should have proper monitoring. OpenSearch is no exception in this regard.

In this presentation, we will look how to use components like Audit Logs, Data Prepper and OpenSearch Alerting to create a small, but effective security monitoring solution.

This will give us two benefits at once:

- A security monitoring solution
- Insights into the workings of components like Alerting, etc. by looking at a real world example

Many OS users, especially those new to Lucene-based search engines, struggle with sharding best practices. This often results in inefficient shard sizes, leading to performance issues and resource waste. Our talk introduces a novel approach: Dynamic Sharding Strategy.

We'll explore how this strategy adaptively determines the optimal primary shard count for indices, particularly beneficial for time-series data with fluctuating ingestion rates. By analyzing past ingestion rates, cluster resources, and workload patterns, our method predicts and applies suitable shard counts for each index rollover.

We'll discuss
* Common sharding pitfalls and their impact
* The idea behind our dynamic sharding approach
* Implementation as a Python script integrated with OpenSearch
* Real-world case studies and performance improvements
* Potential for native integration into OpenSearch's Index State Management

This talk aims to empower OpenSearch users with a practical solution for maintaining optimal cluster performance, reducing storage skewness, and minimizing manual interventions in shard management. Join us to learn how dynamic sharding can revolutionize your OpenSearch cluster efficiency

This talk will explore how OpenSearch can be leveraged to build a centralized and multi-tenant log analysis platform that provides logging-as-a-service to several Kubernetes enterprise environments.

We’ll address solutions for critical topics like cluster and tenant separation strategies, Kubernetes integration, automation, capacity management and monitoring best practices. Additionally, we’ll present our approach to non-technical challenges such as handling diverse tenant requirements, effective customer support and different areas of tenant collaboration.

Throughout the talk, there will be a strong emphasis on standardization as the primary solution component to achieve heterogeneity within the different configuration domains (retention policies, RBAC, etc.), while simultaneously maintaining the flexibility to be able adapt to customers' needs.

Drawing experience from a recent real-world implementation, this session will provide attendees with actionable insights and our lessons learned to overcome technical and operational challenges in building multi-tenant logging platforms using OpenSearch.