Automated Realistic Benchmarks for OpenSearch Clusters

The OpenSearch service at CERN has been operating since 2016, storing and analysing 900 TBs of data to support various use cases such as log analytics and full-text search. It is currently hosted on puppet-managed servers, but since 2024 efforts are underway to migrate the deployment to a more standardized Kubernetes-based setup. As part of this transition, it’s essential to evaluate the performance of both small and large OpenSearch clusters within Kubernetes and understand how this shift impacts performance. More broadly, having a straightforward method to benchmark OpenSearch under various configurations is valuable for identifying bottlenecks and weighing the advantages and limitations of different deployment approaches. This report introduces an easy-to-use pipeline for benchmarking OpenSearch clusters in Kubernetes with customizable parameters, using real-world scenarios inspired by CERN’s primary customers use cases.