Metrics and Measurement

Metrics That Matter

22 Oct 2013  

I had the great pleasure recently to address the students in Guy St. Clair’s class (K4301) at Columbia University’s Information and Knowledge Strategies (IKNS) program.  These are tomorrow’s leaders in developing and managing knowledge-based strategies.

My most challenging question

Of course I told them about the Knowledge Value Chain, and was gratified by their positive reactions.  I also got my most-received question — one that over time has also become one of my most challenging.

That question is, “What is your favorite metric for knowledge?

It’s a logical question, and one that it would be useful to know the answer to — if it could be answered definitively.  But there is no single correct answer.  And it’s challenging, in that it may signify that I have fallen short in my efforts to communicate my core message:  integrating knowledge tightly with business processes.

Inputs, outputs, outcomes

In the KVC Handbook 4.0 (upcoming) I identify three classes of metric that can be applied to knowledge processes and assets:  inputs, outputs, and outcomes.

Metrics arrows

Inputs are (not surprisingly) what you put into the process — resources like money, people, time, and effort.

Outputs are ‘what you get’ for the input — like web site pages produced, documents captured, users served, and reports produced.  The ratio of Outputs to Inputs measures the efficiency of a process.

While these are all valid measures, they each fall short of measuring the impact of our investments and efforts on our organization’s goals and strategies.  These are Outcomes — benefits, value received, impact, and business results.  The ratio of Outcomes to Inputs measures the effectiveness of a process.

Strengthening the business case

As we move our metrics toward the right in our diagram, we move from measuring investment toward measuring return on that investment.  The farther we go to the right, the more our knowledge metric approximates the business metric that it supports — and the stronger is our business case and ROI claim.

Consequently, my advice is to — rather than create new metrics for knowledge — use those business metrics already in place.  If for example a knowledge process supports sales-related decisions, its value derives from the incremental sales that result from its deployment.

Claims and credibility

Though this may seem obvious, it is rarely practiced in the real world.  Why?

  1. It may be hard to measure benefits, especially when they may accrue over a period of time.
  2. It is even more difficult to isolate the incremental contribution of knowledge to providing such benefits.

Claims of such benefits may be easily distorted or exaggerated, and may even move the claims-maker into a position where his or her credibility is questioned.  As a result, people tend to measure outputs, rather than outcomes.  Outputs are typically more tangible and therefore easier to measure than outcomes — but they fall short of measuring impact — way short, in most cases.

In short, too often what we are measuring doesn’t matter — because what matters, we can’t measure.  This is the core conundrum of knowledge metrics.

Outcomes proxies

Another option is to measure proxies for outcomes — typically consisting of client/user opinions of the value added by a knowledge process or product.  This is a pretty good approach — provided that it explicitly includes reference to how the knowledge contributed to business metrics and problem solutions.  In other words, it’s not just a question of whether the knowledge service was good, bad, or indifferent — but rather how much (on a quantitative scale) it contributed, and what the value of that contribution was.

My best answer to “What is your favorite metric?” is “The metric that my user/client uses to measure whether his or her problem has been solved, with my help.” This approach links and aligns the value and knowledge parts of the process.

When our client wins, we win.

Image reprinted from The Knowledge Value Chain Handbook, upcoming 2014.  Copyright © 2014 TW Powell Co.  All rights reserved.

1 Response

  1. […] post – Metrics That Matter – gives us a useful and practical framework for our discussions with others in the company, […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Comments

  • Tim Powell on A brief pause: “Thanks, Les. My frequent conversations with you over the past year or two has helped my thinking a lot in…Jan 15, 13:07
  • Les on A brief pause: “Excellent advice thank you for your terrific reflection piece Tim!Jan 15, 09:40
  • Glenroy London on Knowledge Erosion: How to Avoid It: “Hi Tim I am knoco caribbean. About to join the global km family. Exploring km frameworks for design, development, implementation,…Jul 12, 08:52
  • Tim Powell on War of the Words: “Glad to oblige TJ — and thanks for your note — but I do encourage to try it for yourself…Jun 23, 08:10
  • T J Elliott on War of the Words: ““Chat credited me with founding and/or leading 20 different companies and writing 13 books. In fact, I founded one company…Jun 22, 22:42