The Sumo Logic guide to OpenTelemetry

When you collect data without seeing all of the data points together, you won’t have an accurate big-picture view. And that’s not to mention the impact of missing important data points! Unfortunately, applications are incredibly complex systems that generate a lot of data points. The only way to have that big-picture view is to employ high-quality telemetry — collecting and analyzing all the different types of data to ensure you’re not missing anything.That sounds great, but how do you know what types of data to collect and how do you manage the data collection? Here at Sumo Logic, we’ve bet our future on OpenTelemetry as the answer. We put together this guide to help you understand the ins and outs of telemetry and why OpenTelemetry is where we hang our hat.

Before jumping into the magic of OpenTelemetry, it’s vital to understand telemetry and the types of data you’ll find. First of all, “telemetry” is a term that comes from the Greek roots tele, meaning “remote,” and metron, meaning “to measure.” Applied to observability, the idea of a “remote measure” is precisely what’s needed to understand what’s actually happening in your applications. You need to collect and analyze three main types of data: logs, metrics and traces.

• Logs

Every time an action happens within your application, a log file is created. This data captures the event, giving clues as to whether your application is functioning properly. Using logs, DevOps teams can see what events happened at the same time to find and fix errors and issues quickly.

• Metrics

Metrics are key performance indicators (KPIs) like response time or memory usage. Think about these as data points that are associated with timestamps. This data gives a greater context of system health, allowing you to quickly identify issues.

• Traces

While logs and metrics are important pieces of the puzzle, they essentially capture a snapshot of a moment in time. Traces are more like a short movie that tells a story from start to finish. The trace follows a request through the entire application system, recording every interaction along the way—and allowing you to find deviations from the expected behavior.

All of these types of data are generated through different actions and originate in various places throughout your application. It’s like a “Tower of Babel” situation—all of the different pieces speak a different “language,” making it difficult for end users to leverage the data. This issue is baked into the way observability tools were designed! After all, each observability back-end has its own proprietary way of sending and receiving data, with no standardized data format between the back-end solutions.

So, if you want to switch the observability back-end that you use, you’d have to completely reconfigure everything in order to send telemetry data to the new tool. Ultimately, this tangled-up mess boils down to two key issues: without standardization, you have no data portability and a huge burden of maintaining your telemetry to get the actionable insights you need to do your business.

Luckily, there’s an answer: OpenTelemetry.

• Infrastructure Monitoring
  Monitoring your infrastructure gives you insight into issues like:
  • Operational failures
  • Capacity problems
  • Security breaches
  • Malicious attacks

OpenTelemetry collects all the relevant event logs and system metrics that capture these problems and pushes it out to your back-end. Once integrated with Sumo Logic, you will have access to standard and customizable dashboards that visualize OTel data in a way that makes it quick and easy to identify and resolve key issues.

Using the power of OpenTelemetry and Sumo Logic, you can easily monitor environments like Kubernetes, MySQL, AWS, GCP and more. OTel brings greater efficiency to the Sumo Logic infrastructure monitoring—no need to install and manage different agents! You can easily bring together metrics, logs and traces for full stack observability.

• Application Observability (APM)
  Sumo Logic provides a unified platform to ingest, analyze and correlate OpenTelemetry data drawn from your application. You can quickly diagnose where application degradation is happening, then troubleshoot and find the root cause.

OpenTelemetry Customer Success Story: GoSpotCheck

GoSpotCheck is an app that assists field reps with collecting real-time business intelligence while on the job. For example, many brands have specific agreements for how and where their items are stocked on store shelves. Field reps used to have to check this manually, using paper and pen to make notes and report back. GoSpotCheck solved this problem with their mobile app,

While the front-end is an intuitive interface, the back-end of GoSpotCheck’s stack is extremely complicated. Their system includes Kubernetes infrastructure, Postgres, Go, Rails combined with ML and data pipelines. With a need to continuously deliver for their clients, GoSpotCheck knew that full stack observability was the only way to stay ahead of issues. The challenge was to find a solution that could provide observability for their entire system. That’s where Sumo Logic and OpenTelemetry came in. Using these tools and set of standards, GoSpotCheck was able to have every event, metric, log, trace, view and user in one place.

Read more about GoSpotCheck and OpenTelemetry.

In short, yes, OpenTelemetry has excellent security features as part of the standard. While solutions with custom code raise serious security concerns, OTel was built with this in mind. OpenTelemetry includes support for:

• End-to-end encryption and protocols like HTTPS or gRPC-TLS, preventing data from being accessed while it is transferred.

• Mutual TLS (mTLS), mutual authentication, ensuring that each end of the network connection is verified with the correct private key.

Role-Based Access Control (RBAC), also known as role-based security, allows permissions and privileges to be customized for specific authorized users to restrict system access.

OpenTelemetry and Prometheus are both open source observability projects forged in the same kiln (the Cloud Native Computing Foundation, CNCF). While OpenTelemetry represents a set of tools and standards for crafting instrumentation and collection for telemetry data, Prometheus is an observability back-end that transforms the data into a human-readable dashboard. OpenTelemetry has cemented itself as the golden standard for monitoring distributed systems and is currently the second-most popular open source project from the CNCF, behind Kubernetes.

Prometheus, on the other hand, is a metrics monitoring tool that utilizes an OpenTelemetry collection agent. Metrics in Prometheus serve as a timekeeper for logs, allowing for basic data visualization with the use of math libraries and a unique query language (PromQL). The overall purpose, vision, and scalability of these projects differ greatly with Prometheus solely focused on collection for metrics and OpenTelemetry working as the set of standards with which Prometheus can collect data. Given the esoteric nature of Prometheus, users may want to consider using an enterprise data visualization tool for all datatypes like Sumo logic.

Sumo Logic not only supports OpenTelemetry, but we also contribute to making this important tool even better. For example, we’ve created the Sumo OTel Collector, built on OpenTelemetry, to provide a single unified agent to send logs, metrics, traces and metadata for observability to Sumo Logic. Our collector provides simplified and streamlined performance and behavior monitoring for complex distributed systems. You’ll be able to easily identify issues, ultimately leading to improvement in overall system reliability and efficiency.

As data complexity grows, the need for clear observability becomes more crucial. Two key components define the future: a commitment to open standards and the power of advanced analytics. Here at Sumo Logic, we're deeply invested in this evolution, particularly through our support for OpenTelemetry. It's not just about observing the change; we're actively driving it. Let's explore what this means.

Commitment to open standards: For observability to be holistic and efficient, future platforms must be deeply ingrained with open standards. The OpenTelemetry standard for data collection is posed to dominate the observability landscape. These open standards promote interoperability and ensure that every byte of data, whether it's a log, metric, or trace, delivers value regardless of its origin.

Mastering advanced analytics in diverse environments: The sheer volume of data generated in today's multi-cloud and hybrid environments necessitates advanced analytics capabilities. It's not just about data collection; it's about transforming that data into actionable insights. The future observability tools will empower organizations to distill vast data streams into discernable patterns, identify anomalies, and bring to light correlations that could go unnoticed in a raw data representation. This will be particularly pertinent for teams navigating the intricacies of multi-cloud and hybrid ecosystems, where data comes from an array of on-premises, public cloud, and private cloud sources. A unified perspective across these environments

As an enterprise-ready vendor, we’ve fully embraced the power of open standards and unified collection. We’ve seen first-hand how our community is better together and we firmly believe the future of observability is standardizing on OpenTelemetry.

It can feel overwhelming to get started with OTel, especially if your system is complex. Don’t worry! There are plenty of OpenTelemetry tutorials available online to help you get started. Here at Sumo Logic we’ve put together this guide to using the common OpenTelemetry demo application with Sumo Logic.

Still feeling overwhelmed? That’s why we’re here. As a good partner, we help you leverage OpenTelemetry easily and efficiently. But don’t just take our word for it! Check out this short video on how SAP leverages Sumo Logic and OpenTelemetry.