Tim Wood Powell     The value of knowledge

My early studies in evolutionary biology provided a solid foundation for my later work in business strategy. The systems view of the earth and its living inhabitants — our physical world — formed a powerful, indelible metaphor readily adaptable to our epistemic world — of Data, Information, Knowledge, and Intelligence.

In nature, species and other biological groups are born (by genetic mutation), adapt and thrive (by survival of the fittest), and are eventually supplanted by more adaptive species. Some species are amazingly resilient; sharks have been on earth for 450 million years. Others die off entirely, like the dinosaurs — who, along with most other land-based species, did not survive an asteroid hit to our earth 66 million years ago.

In adapting to an ever-changing ecosystem, species find their respective ecosystemic niches. There are fishes that have adapted to living deep in the ocean — so deep that, if you force them to move higher, they explode from the reduced water pressure.

Our epistemic ecosystem

I have long regarded epistemics as a complex, dynamic socio-econo-semiotic ecosystem. It’s a vast ocean — with tides, waves, storms, and virtually unfathomable depths and layers (pun intended).

As any navigator will tell you, it’s easy to get lost if you don’t know the territory. But, if you do, you can harvest significant rewards.

In the South Seas, divers find pearls without engaging any equipment, even SCUBA. They know where to look and what to look for. An uninitiated outsider would likely come up empty-handed — or even die trying.

Information follows this same logic. If you know where to look for it, and how to discover it, your success, though not guaranteed, is much more likely than if you do not.

The role of experts

In the practice of research, it is axiomatic that ‘Knowing has value. But even more valuable than knowing is knowing who knows’ — meta-knowledge, in other words. Experts are those who ‘have expertise’ — mastery of some specified body of knowledge. In doing research, I always seek out experts first — in order to create real-time dialogues with them.

I would always value a conversation with a person who ‘knows the territory’ in which I am interested, over ten — or even a hundred — conversations with people who do not.

The false lure of scale

Once you find an expert who knows the answer to your question, it’s a waste of your time (and money) seeking others who do not. An economist would say, the marginal return to scale for expertise is small, relative to the marginal cost of that scale. In my experience, this is axiomatic — close to a natural law, as illustrated in Figure 2.

Yet, we are drawn irresistibly to scale as a proxy for veridicality — ’truthfulness.’ If all of us together don’t know something (’socialized ignorance’), it’s somehow more reassuring than each of us not knowing individually. It’s almost as though we believe, at some emotional level, that we ‘vote’ for the truth.

Large language models

LLMs claim to have transcended this simple logic. They essentially represent that by putting all the world’s knowledge into one vast inter-connected container, we can then retrieve it more easily. This makes intuitive sense — until you follow it to its logical consequences.

By way of analogy, let’s play a mind game back in the physical world. If you were tasked with finding pearls, you could park beside the nearest ocean, put on your snorkel and flippers, and dive in. If you were already a good swimmer, it’s not theoretically impossible that you might, given enough time — we’re talking several decades — be successful in finding some pearls.

But more likely, to achieve that mission, you’d do some research, hop a plane to the South Seas, and hire a local diver expert in such things. The key, of course, is in knowing that pearls are produced naturally by a certain kind of oyster, and where that oyster lives, its life cycle rhythms, and so on — the pearl ecosystem, in other words.

Hunting information

In teaching intelligence practitioners how to find information, I’ve always used the same rule: to find information, first find where it ‘lives’ — who produces it, when, and why. That meta-information gives you the key to locating the information itself.

The notion of putting the all world’s information into one large container, an LLM, is fantastic — in the literal sense of being based on fantasy. Though discussed for at least a century, starting with people like H.G. Wells, it’s science fiction, at best.

Science because it is logically possible — fiction because it is, in practice, not even remotely feasible. In order to do so would take resources well beyond our capacities. Fascinating and fun — but quintessentially impractical, fanciful, even nutty.

Boiling the ocean

The analogy I use is ‘Boiling the ocean to make a cup of tea.’ Even if theoretically possible in some imagined world where resource constraints can be ignored, it is far outside the realm of worldly possibility — where those resources are tangibly finite. Wouldn’t you, instead, just boil a kettle of water on the stove?

The resources expended should fit the problem at hand in the most effective and efficient way available. Costs should be commensurate with benefits — the essence rule of ROI.

Finding my story

I recently ran a test using myself as the topic. Since I know the facts, this would save me a verification step. And my biography has been posted online in various formats at various times over the years.

I asked ChatGPT who I am — and received a fanciful mashup of:

• things that are fully correct and true
• things that are partly correct
• things that are plausible but incorrect, and
• things that are implausible.

I knew immediately which is which, of course. But people, even those who know me pretty well, might not.  The results reminded me of what I might look like through a funhouse mirror that distorts everything familiar.

So the ‘return’ was spotty at best. And the ‘investment,’ though I don’t know exactly ,was almost certainly have been incommensurately large, had it been sold on an unsubsidized basis (I was using a money-losing ‘free’ model). The training and inference costs of ‘boiling the ocean’ to find my bio were overkill — simply not worth it.

If you want my current, accurate bio, you’ll find me on LinkedIn. It’s dead easy — and an LLM could theoretically have ‘known’ that. But it didn’t, it doesn’t, and many believe it can’t.

An experienced human researcher would likely know that — and even a smart layperson who approached it thoughtfully.

Epistemic efficiency

Epistemic resources should be evaluated in the same ways physical resources are — plant, equipment, parts, inventory, and so on. Each carries costs and benefits.

Now that ‘knowledge as a service’ is being offered by LLM providers, we need to compare the benefits and costs of solving any particular enterprise problem with machine intelligence, human intelligence, or a hybrid. This industrialization of intelligence makes such evaluations imperative — and easier, since the costs are more tangible. Especially as the cost of metered tokens rise, we recommend that you continually ask, What is the best way to address any given issue, given the twin concerns of effectiveness and efficiency?

Occam’s economic razor

Boiling the ocean is clearly not resource efficient — that’s essentially what the fanciful metaphor means. The ancient philosophical principle of Occam’s Razor is instructive here: “Entities must not be multiplied beyond necessity.” That is, in explaining things, we should use the simplest available explanation that works.

This was reiterated centuries later by Albert Einstein in his famous guidance to the effect, “Things should be made as simple as possible — but no simpler.”

In our economic life, we apply a similar principle almost without thinking — to fix a problem, we use the least resource-expensive solution that works. We can (and should) apply this same thinking to epistemic resources — lest we bankrupt ourselves chasing chimeric science-fictive solutions.

Follow the money

But how do you know which information has the greatest value for our solution?  Another simple rule is to follow the money; the information of greatest value to the enterprise is that which corresponds to the flows of economic value.  To find where the information ‘lives,’ find where the money comes from.

I used this insight years ago in building a Voice of the Customer research practice at a global research company.  From customers flow revenues — therefore their collective ‘voice’ is the beset indicator of that value.

Microsoft used that insight in building the value of their software in the 1980s.  Back when they were just another choice of enterprise guyers — and not always a dominant one (my global firm used WordPerfect) — they captured and used customer feedback to relentlessly improve their products.

A similar story is told by Tadashi Yanai, who built the clothing retailer Uniqlo into a global powerhouse.  According to a recent New Yorker profile, “The company applies a sociological attention to the gestures and dilemmas of people’s daily routines; once harvested, these insights are quickly incorporated into product designs.”

Uniqlo does not outsource its customer service, seeing it as an opportunity to gather 31 million pieces of ‘voice of the customer’ feedback per year. This has helped make them massively successful.

This is why I find it especially ironic that some companies are looking to outsource customer relationships to bots.  It’s a low-cost, potentially high-return activity, with huge impact on the company’s reputation.  To diminish it is a missed opportunity.

Building knowledge value consists of two primary challenges:  (1) knowing where and how to find information (process optimization) and (2) knowing which information is valuable (strategic relevance).  In building a successful knowledge strategy, (2) must precede (1) to maximize effectiveness and efficiency.  And (1) must include considering a range of alternative tools — including the simplest, most straightforward, least expensive ones.

featured

---