Metrics and Measurement

Metrics That Matter

22 Oct 2013  

I had the great pleasure recently to address the students in Guy St. Clair’s class (K4301) at Columbia University’s Information and Knowledge Strategies (IKNS) program.  These are tomorrow’s leaders in developing and managing knowledge-based strategies.

My most challenging question

Of course I told them about the Knowledge Value Chain, and was gratified by their positive reactions.  I also got my most-received question — one that over time has also become one of my most challenging.

That question is, “What is your favorite metric for knowledge?

It’s a logical question, and one that it would be useful to know the answer to — if it could be answered definitively.  But there is no single correct answer.  And it’s challenging, in that it may signify that I have fallen short in my efforts to communicate my core message:  integrating knowledge tightly with business processes.

Inputs, outputs, outcomes

In the KVC Handbook 4.0 (upcoming) I identify three classes of metric that can be applied to knowledge processes and assets:  inputs, outputs, and outcomes.

Metrics arrows

Inputs are (not surprisingly) what you put into the process — resources like money, people, time, and effort.

Outputs are ‘what you get’ for the input — like web site pages produced, documents captured, users served, and reports produced.  The ratio of Outputs to Inputs measures the efficiency of a process.

While these are all valid measures, they each fall short of measuring the impact of our investments and efforts on our organization’s goals and strategies.  These are Outcomes — benefits, value received, impact, and business results.  The ratio of Outcomes to Inputs measures the effectiveness of a process.

Strengthening the business case

As we move our metrics toward the right in our diagram, we move from measuring investment toward measuring return on that investment.  The farther we go to the right, the more our knowledge metric approximates the business metric that it supports — and the stronger is our business case and ROI claim.

Consequently, my advice is to — rather than create new metrics for knowledge — use those business metrics already in place.  If for example a knowledge process supports sales-related decisions, its value derives from the incremental sales that result from its deployment.

Claims and credibility

Though this may seem obvious, it is rarely practiced in the real world.  Why?

  1. It may be hard to measure benefits, especially when they may accrue over a period of time.
  2. It is even more difficult to isolate the incremental contribution of knowledge to providing such benefits.

Claims of such benefits may be easily distorted or exaggerated, and may even move the claims-maker into a position where his or her credibility is questioned.  As a result, people tend to measure outputs, rather than outcomes.  Outputs are typically more tangible and therefore easier to measure than outcomes — but they fall short of measuring impact — way short, in most cases.

In short, too often what we are measuring doesn’t matter — because what matters, we can’t measure.  This is the core conundrum of knowledge metrics.

Outcomes proxies

Another option is to measure proxies for outcomes — typically consisting of client/user opinions of the value added by a knowledge process or product.  This is a pretty good approach — provided that it explicitly includes reference to how the knowledge contributed to business metrics and problem solutions.  In other words, it’s not just a question of whether the knowledge service was good, bad, or indifferent — but rather how much (on a quantitative scale) it contributed, and what the value of that contribution was.

My best answer to “What is your favorite metric?” is “The metric that my user/client uses to measure whether his or her problem has been solved, with my help.” This approach links and aligns the value and knowledge parts of the process.

When our client wins, we win.

Image reprinted from The Knowledge Value Chain Handbook, upcoming 2014.  Copyright © 2014 TW Powell Co.  All rights reserved.

1 Response

  1. […] post – Metrics That Matter – gives us a useful and practical framework for our discussions with others in the company, […]

Leave a Reply

Recent comments

  • Trip on Piercing the Enterprise Bubble: “Hi Tim, True, doesn’t have to be binary. Would love to chat. Best, TripDec 10, 11:32
  • Tim Powell on Piercing the Enterprise Bubble: “Hello Trip, Thanks for reading and for your comment. In fact, I agree with you in an ideal sense –…Dec 10, 09:36
  • Trip on Piercing the Enterprise Bubble: “Tim, ecosystems are networks – edge values are just as important as node values. You lose the usefulness of the…Dec 9, 20:53
  • Egy on Counterfeits: the golden age: “If French TV is anything to go by, then yes, there are fake cgetriates being made that do not contain…May 16, 07:05
  • Tim Powell on Whatever Happened to Knowledge Management?: “Dear Dr. Spender, It’s an honor to have your comments here. I have been dipping into your new book “Business…Feb 17, 18:04
  • JC on Whatever Happened to Knowledge Management?: “These are sage comments. They hinge on the notion of ‘separation’ between the organization as a domain of value-creating human…Feb 16, 10:46
  • Tim Powell on The knowledge archipelago: “Illustrates the point well, don’t you think? As far as I know, it’s in the public domain. Cheers, tpOct 2, 12:13
  • Felix on The knowledge archipelago: “Hi, could give me a reference to the cartoon? Is there still a copyright on it? I would like to…Oct 1, 14:08
  • Susan Murphy on “Knowledge” is a National Security Issue: “This article so clearly explains why our economy — and country — have stalled. If only more people understood and…Sep 8, 12:05
  • Yesitha Batepola on Competitive myopia: “As an undergraduate student, I have to thank you since you clearly explained what was a grey area at first…Jul 3, 16:52
%d bloggers like this: