Grafana: Turning data chaos into developer efficiency and CFO savings

In a 'fundamentally changed world' where software is no longer contained in a single box, there's growing awareness of the necessity for observability tools not just to keep track of systems and their various components, but also to control potential cost blowouts enabled by the very ease with which hyperscale services are ordered and delivered. That makes observability a versatile set of tools with appeal to developers and finance chiefs alike reckons Perth-based Grafana co-founder Anthony Woods.

Grafana competes in a pond alongside the likes of Datadog, Vantage, Splunk and New Relic. At its core, the company's solutions help software engineers peer into the nuts and bolts of modern applications and infrastructure, or, as Woods puts it, "We turn the black boxes of microservices into glass boxes, so software engineers can better understand their  applications and infrastructure."

The necessity for observability rests on a well-worn foundation perhaps best described by management guru Peter Drucker. 'You can't manage what you can't measure'. Rendered transparent, those black boxes reveal useful information.

Over the past decade, Woods notes, the world has transformed. He channels Marc Andreessen. "Software is eating the world. Every company is becoming a software company, and just the pace of innovation is accelerating. The move from monolithic applications to microservices has boosted developer productivity, allowing faster shipping of features [but] fragmentation creates complexity.

"When something goes wrong, you need to see which piece in the puzzle is causing the problem," Woods explains. "You just need so much more data to be able to understand what's happening."

It's not just developers who benefit from improved visibility across their software estates. The emerging field of FinOps (the practice of managing cloud finances) makes observability essential for both technical teams and CFOs.

As cloud sprawl enables easy spin-ups of resources, visibility can quickly erode, leading to unchecked costs. Observability provides the insights needed to optimise spending. "One of the interesting things about observability is that it's kind of redefining every day as to what it encompasses," Woods notes. "More and more you see it bleeding into basically anywhere where there's data that helps understand the context of what's happening in your business."

For developers, observability means faster issue resolution in an era where AI is accelerating code production. Woods says along with AI accelerating the pace of software development (not all of which is of the highest quality), up goes demand for robust monitoring.

Which is itself AI powered; he says Grafana leverages AI to 'distil signal from noise', with machine learning identifying anomalies and building knowledge graphs that map relationships between applications, infrastructure, and dependencies.

There is something of a catch, though. Those in the trenches will have heard whisperings around the cost of observability solutions themselves, which can very quickly get out of hand since there is so much that can be monitored, logged, and shovelled into endless databases. Woods is far from oblivious (after all, observability is practically part of his persona). Added to which, Grafana's observability survey revealed that 74% of respondents prioritise cost when selecting vendors. "Telemetry gets expensive at scale…but the value kind of plateaus," he says.

Grafana's answer is Adaptive Telemetry which puts a level of control in the hands of (probably) the CFO. Premised on the simple principle that not all data is useful, Adaptive Telemetry essentially focuses attention on the salient bits and leaves everything else be. Of course, what's important and what isn't may change from time to time, and Woods says Grafana makes sure to provide that level of flexibility and adaptability.

The result is that costs come back under control with reductions of up to 50% in some cases. "It's a lever to observability teams. They decide whether they want to turn the tap up and bear the cost because it provides value, or down if not."

Adaptive Telemetry is an extension of the value principle which has underpinned Grafana's traction in global markets. Founded as a global entity from day one, with Woods in Australia, CEO Raj Dutt in New York, and creator Torkel Ödegaard in Stockholm, the company focused initially on the US and Europe.

Woods says open source was key to breaking in. "Our mission from day one was we're going to build technology that solves real problems and adds value to engineers, and work out how to make money later," Woods recounts. This led to organic adoption: engineers downloaded Grafana's tools, derived value, and spread them internally.

Turns out that 'shadow IT' worked well for Grafana, building credibility and momentum.

While Grafana has achieved significant traction in the US and Europe, Woods candidly admits adoption in Australia and New Zealand (ANZ) has been slower, though it's accelerating. He says the region represents just 4% of Grafana's business, below the expected 8%, due to cultural risk aversion. "We tend to avoid open source, preferring 'safe'….you know, 'you never got fired for buying IBM'. But that's changing and we are seeing 80% year on year growth, though that is from a low base.

Ultimately, Grafana's success hinges on its relentless focus on value, exemplified by its Adaptive Telemetry. By cutting bills, it fosters loyalty and upsell opportunities. "We had that conversation with our board, and they said, 'What do you mean? You're going to build a feature that reduces your customers' bills by up to 50%?' And the response was 'yes, because it's good value for customers'."