Metrics and Measurement

Paths to Value

15 Sep 2015  

It makes my day when I am asked a question I can’t answer completely and easily. The students in Columbia’s Information and Knowledge Strategy program rarely fail to disappoint in this regard.

Where do KPIs come from?

One IKNS student has been using the KVC framework in her “day job” to design program evaluations for a global NGO. Her question had several aspects: “Who designs Key Performance Indicators? Is that job, a profession, or what? How and where do you learn how to do it? How do you know it’s right?”

My first instinct was to answer from my personal experience. Shortly after I left college — in The Digital Dark Ages — I was tasked to design  a set of performance metrics for day care agencies under contract with the State of New Jersey. I was officially a “Contract Administrator”, not a metrics designer. I didn’t know what I was doing — metrics design was just part of my job. I found a book on it, and somehow managed (like the one-eyed man in the Land of the Blind) to pull it off to everyone’s satisfaction.

The Balanced Scorecard industry

KPIs reached industrial strength with the development of the Balanced Scorecard in the early 1990s. People had begun to realize that financial performance metrics only took you so far — but typically these were the results of other events and activities that were operational in nature. Measuring those operational things (customer satisfaction, for example) was seen to have its own value. Metrics became a mini-industry — businesses were created to design and implement KPIs and (later) enterprise dashboards to monitor them.

In all this, there’s good news and bad news. The good news is that enterprise management has entered a new age of empiricism. Everyone wants to be evidence-based and metrics-driven, instead of gut-feel-and-instinct driven as previously.

The bad news is that in our quest for metrics, we are relying heavily on our ability to find the right metrics. Metrics do not grow on trees; they require resources (people, time, technologies) to develop and to implement. They themselves are investments, each having some (greater or lesser) ROI.

Metrics matter — even when they don’t

Metrics have major consequences — for people’s job performance, evaluations, compensation, and so on. So much so that there is a risk of “playing to the numbers”, wherein clearing the metric bar becomes as important as — or more important than — the underlying performance that is being measured.

Entire industries typically follow a standard and relatively narrow set of metrics in competing with each other. When one player bucks the trend and re-thinks the “metrics that matter”, it can have a transformative effect on that player’s strategic fortunes — and, eventually, on an entire industry.

When one major league baseball team (the under-capitalized 1982 Oakland As) adopted a new set of metrics (sabermetrics), they began to win more games than other teams who had not yet changed. Michael Lewis famously dramatized this in his book Moneyball, which I recommend as background to anyone designing business metrics.

Now that this set of insights has permeated the pro baseball industry, it is less of a competitive advantage than it was then. The quest for analytic dominance should be an ongoing process, not a one-off project.

Measurement is good — IF you are measuring what matters, strategically and tactically. If you are not, measurement is at best a waste of resources — and at worst a distraction from seeing what actually does matter.

The shortest path to value

I recently discussed with my Columbia students the issue of selecting optimal metrics for Knowledge Management initiatives. I used the meme of shortening the path to value.  First I drew the distinction (illustrated below) between “output” metrics and “outcome” metrics — with both being benefits-based, but with the latter having the added value of measuring along the same dimensions that key stakeholders would measure — for example, the C-suite or investors.Powell ENTERPRISE KNOWLEDGE 2015-08-06 p6

The other distinction I drew is between results that are more easily measured (and, therefore, more easily demonstrated) than others. The matrix below shows the results for eight benefits frequently mentioned in studies of Knowledge Management projects.  (I used a recent Accenture study as the primary basis, supplemented by my own professional experience.)

Powell ENTERPRISE KNOWLEDGE 2015-08-06 p7

These benefits range from readily measurable and outcome-based (upper-right quadrant, like cost savings) to relatively problematic to measure and output-based (lower-left quadrant, like better decisions).

The upper-right is clearly where one finds the highest return. I argued that the most direct path to the upper right quadrant is the “path of least resistance” when it comes to justifying projects and budgets. This is where you want to be when the next business cycle-driven downturn hits, and budgets are cut back.

This principle could apply well beyond knowledge management projects — to most any enterprise activity or initiative.  You should always be “moving the needle on enterprise value” — in aspiration, if not in fact.

Leave a Reply

Recent comments

  • Trip on Piercing the Enterprise Bubble: “Hi Tim, True, doesn’t have to be binary. Would love to chat. Best, TripDec 10, 11:32
  • Tim Powell on Piercing the Enterprise Bubble: “Hello Trip, Thanks for reading and for your comment. In fact, I agree with you in an ideal sense –…Dec 10, 09:36
  • Trip on Piercing the Enterprise Bubble: “Tim, ecosystems are networks – edge values are just as important as node values. You lose the usefulness of the…Dec 9, 20:53
  • Egy on Counterfeits: the golden age: “If French TV is anything to go by, then yes, there are fake cgetriates being made that do not contain…May 16, 07:05
  • Tim Powell on Whatever Happened to Knowledge Management?: “Dear Dr. Spender, It’s an honor to have your comments here. I have been dipping into your new book “Business…Feb 17, 18:04
  • JC on Whatever Happened to Knowledge Management?: “These are sage comments. They hinge on the notion of ‘separation’ between the organization as a domain of value-creating human…Feb 16, 10:46
  • Tim Powell on The knowledge archipelago: “Illustrates the point well, don’t you think? As far as I know, it’s in the public domain. Cheers, tpOct 2, 12:13
  • Felix on The knowledge archipelago: “Hi, could give me a reference to the cartoon? Is there still a copyright on it? I would like to…Oct 1, 14:08
  • Susan Murphy on “Knowledge” is a National Security Issue: “This article so clearly explains why our economy — and country — have stalled. If only more people understood and…Sep 8, 12:05
  • Yesitha Batepola on Competitive myopia: “As an undergraduate student, I have to thank you since you clearly explained what was a grey area at first…Jul 3, 16:52