A survey of 525 organizations that have embraced some level of observability suggests IT organizations that have invested early in acquiring these platforms are starting to pay off. Organizations identified as “observability leaders” were six times more likely to have accelerated root cause identification capabilities after adopting observability.

The State of Observability 2021  survey was conducted by Enterprise Strategy Group (ESG) and published today on behalf of Splunk

Only 11% of the organizations surveyed are designated as observability leaders based on their ability to varying degrees to satisfy four criteria as defined by ESG: observability experience of more than 24 months, ability to correlate data, efforts to rationalize vendors, and adoption of AI.

Among those observability leaders, 43% said they’ve significantly accelerated root cause analysis. Nearly half of those leaders (48%) said they’re “completely confident” they can meet application availability/performance commitments. Just under half of the leaders (45%) reported they have launched eight or more new products/revenue streams in the last year.

In general, nearly half of the total survey respondents (45%) respondents indicated that service failures have lowered customer satisfaction, with 30% of them indicating that a service failure has led to lost customers entirely. The fact that many organizations are experiencing customer service challenges that can be traced back to IT issues is not all that surprising. As organizations invest in digital business transformation initiatives, however, their tolerance for those IT issues is declining.

Achieving observability is a challenge

Observability is hardly a new IT concept. It’s been a core tenet of best DevOps practices for many years. Achieving observability has, nevertheless, been challenging. At the core of the issue is the array of IT monitoring tools that IT organizations employ today. Any time there is an issue IT teams need to correlate data from multiple tools to determine the root cause of an issue. Each tool has its own method for collecting data and its own unique user interface. IT organizations typically convene a “war room” to compare and contrast metrics surfaced by these tools to rule out various platforms as the source of issue largely by process of elimination.

In contrast, observability platforms leverage agent software embedded within applications to correlate metrics, logs, and traces in a way that provides more context within a single console. Rather than spending hours in a war room navigating multiple monitoring tools, an observability platform rationalizes the number of monitoring tools required. That approach helps pays for the acquisition of the observability platform.

The primary reason observability platforms are just now starting to gain traction is the rise of open source agent software based on the OpenTelemetry specification that reduces the cost of instrumenting applications. In fact, the Splunk approach to observability is dependent on applications being instrumented using open source agent software, said Spiros Xanthos, vice president of product management for observability and IT Ops at Splunk.

The Splunk observability platform combines its IT operations platform with an observability platform it gained with the acquisition of Rigor in 2020. The end result is a more proactive approach to managing IT that doesn’t depend on requiring IT specialists to convene a meeting to reactively determine the root cause of an issue, said Xanthos. “The war room will fade away,” he said.

It may take a while before observability platforms supplant legacy IT monitoring tools. Most legacy applications today are not instrumented using agent software. Over time, however, modern applications that have been instrumented by developers will eventually replace those legacy applications.

Regardless of how observability is achieved, modern IT environments have become too complex to manage without it. The only thing that remains to be determined is how long existing approaches to IT monitoring will need to be maintained while waiting for the promise of observability to finally be fulfilled.