5 Reasons why OpenTelemetry will boost Observability and Monitoring

In March 2019, the communities around OpenTracing and OpenCensus announced their decision to merge both projects into a new, open observability standard called OpenTelemetry. This move has big potential boost observability and monitoring topics especially in the field of open source monitoring solutions. See why ...

With the spread of microservices and DevOps principles, open source tools for observability and monitoring are on the rise. Hereby observability is a rather new term in the context of software systems, that describes all activities with the goal of gathering mainly three types of data from a target system: metrics, traces, logs / events. Observability lays the basis for managing the performance, availability and health of a software system.

As distributed tracing gained in popularity, an open standard emerged – OpenTracing. OpenTracing unified the way traces are collected and tracing information is propagated through the system components. As OpenTracing covers only one of the three pillars of observability, later on, additional standards arose: OpenMetrics and OpenCensus. While OpenMetrics defines a standard for a metric exposition format, OpenCensus is a standard that covers both, collection of metrics and distributed traces. Although the tracing part of OpenCensus hardly differs from OpenTracing in functional terms, the two standards are technically incompatible.

A standard loses importance if there are several of them. As both, OpenTracing and OpenCensus, gained in popularity, users who want to implement observability are now spoilt for choice.

Fortunately, the communities around OpenTracing and OpenCensus have put their heads together and decided to merge the two standards. The new single standard OpenTelemetry has been announced in April 2019:

I’m convinced that OpenTelemetry will significantly increase the importance of open source tools for observability, monitoring and application performance management. Let’s see why…

1. OpenTelemetry has excellent chances to become THE SINGLE standard

With OpenTelemetry both relevant communities (OpenTracing and OpenCensus) joined forces. As both communities have quite big user bases, the resulting merged user base will be even bigger. The network effect will take hold and prevent overlapping standards from emerging alongside OpenTelemetry.

2. OpenTelemetry decouples data collection from monitoring tools

In our customer projects we often see the challenge of aligning monitoring topics within an organisation and providing a holistic, end-to-end view through all software components. There are different teams responsible for different software components and each team has its own preferences regarding monitoring tools or data collection standards. Each tool comes with its own way of collecting data, which inevitably leads to the problem of creating a holistic understanding and view of the overall system. With OpenTelemetry there will be one single standard that consistently defines how data is collected and how information is propagated across system components (W3C TraceContext).

While more and more tools (open source and commercial) will adapt the OpenTelemetry standard, data collection will be steadily decoupled from questions regarding monitoring tool selection. Popular open source observability tools like Zipkin, Jaeger, Prometheus, Skywalking, inspectIT Ocelot, etc. already support one of the two existing standards, and thus, will support OpenTelemetry in the future. OpenTelemetry fosters the interchangeability and integrability of open source monitoring tools which perfectly aligns with the core idea behind the OpenAPM initiative. That’s why I think that OpenTelemetry will fuel the success of holistic, combined, open source monitoring and application performance management solutions even further.

Beside the open source tools, more and more commercial monitoring and APM-vendors are providing support for OpenCensus, OpenTracing and, in the future, OpenTelemetry: Google Stackdriver, DataDog, Dynatrace, Lightstep, Instana, SignalFX, etc.

So, OpenTelemetry has a good chance to generally decouple data collection from tools that consume the data for monitoring and APM purposes.

3. OpenTelemetry will cover all observability pillars

As OpenTelemetry will fully integrate the functionality of OpenTracing and OpenCensus, it will from beginning cover both: collection of metrics and distributed traces. You may now ask: What about the third pillar of observability – Logs / Events ? Well, the OpenTelemetry project provides the following answer to this:

“OpenTelemetry will not initially support logging, though we aim to incorporate this over time.”

With this in mind, from a conceptual point of view there won’t be a significant gap in data collection formats that would need additional standards or tools covering them.

4. OpenTelemetry follows a polyglot approach

From a programming language perspective, today’s software systems are internally highly diversified. It’s not unusual that software systems consist of components that cover different programming languages like Java, GoLang, .Net, NodeJS, Python, etc. At the same time monitoring and observability are cross-cutting concerns that require unified concepts alongside with tooling for data collection which have to be cross-language compatible. OpenTelemetry comes with generic data collection concepts and inherent support for all relevant programming languages. With OpenTelemetry programming language boundaries are not a hurdle for consistent observability.

5. OpenTelemetry fosters automated data collection

A unified, single standard for data collection allows to join forces and align tools for automated data collection. For non-managed programming languages like C, C , Go, etc. there will be common SDKs and unified libraries that simplify data collection. Managed languages like JVM-based languages and .NET even provide ways of fully automating instrumentation for data collection. A single standard like OpenTelemetry lays the basis to not only unify data collection formats and libraries but also the utility tooling around it. OpenTelemetry based instrumentation tools like inspectIT Ocelot (OpenCensus- / OpenTelemetry-based Java agent) allow to flexibly automate instrumentation while retaining the freedom of choosing from a variety of tools for data consumption such as analysis, visualisation, alerting, etc..

I’m looking forward to November 2019, when OpenTelemetry will officially replace OpenCensus and OpenTracing. Let’s see how it will evolve!

What do you think about OpenTelemetry? Do you share my opinion regarding OpenTelemetry? Are you of a different opinion? Leave a comment at the bottom of this blog post!