Cisco Webex cuts observability latency with Apache Pinot
Cisco Webex has adopted Apache Pinot for real-time observability of its platform, managing over 100TB of daily telemetry data at a velocity of 300k messages per second. Benchmarking showed Apache Pinot provided 5x to 150x faster P99 latencies compared to Elasticsearch, alongside a reduction in storage footprint by nearly an order of magnitude and a decrease in cluster size by over 500 nodes. Webex also developed Apache Pinot datasource plugins for Grafana and Kibana to integrate with existing SRE tools and open-sourced several contributions to the Apache Pinot ecosystem.
Key Takeaways
- Webex processes more than 100TB of telemetry per day and peaks at 300k messages per second, or more than a billion events daily.
- Apache Pinot delivered 5x to 150x better analytical-query performance than Elasticsearch, with subsecond results in most tests and lows of 36 milliseconds.
- Elasticsearch timed out in 12 of 18 test cases, with a 30-second timeout and failures in two-thirds of scenarios.
- Webex reduced data under management from 800TB of unique data to 121TB, with replicated storage of about 360TB.
- The team built Apache Pinot datasource plugins for Grafana and Kibana and has already merged several Pinot ecosystem contributions, including partitioning and Trino connector fixes.
Why It Matters
For Webex, Pinot is now the observability backend for a telemetry stream too large for slow rollups and search-engine-based analysis. The competitive point is stark: Elasticsearch could not return results in many higher-concurrency tests, while Pinot kept most queries in the subsecond range and integrated into Grafana and Kibana so SRE workflows did not have to change. The next signal to watch is whether the Grafana and Kibana plugins, currently open for community review, move into broader use beyond Webex.
Read full article at startree.ai
Enjoy our coverage?
Add StreamingMeme as a preferred source on Google to see more of our streaming news at the top of your Search results.
Add as preferred source