Tim Wood Powell     The value of knowledge

The KVC framework is now in its 30th anniversary year. Clients have lately been asking me, Is KVC still relevant in the Age of AI?  When are you going to revise it?

Spoiler, yes, KVC is still relevant — as demonstrated in its usefulness in planning applications for AI.  Herein I apply KVC as a broad roadmap for AI — while noting the potential need for a significant course correction to language-based AI models.

Background – envelope logic

I first outlined the KVC model literally on the back of an envelope in 1994. I had just been elected to the Board of the Society of Competitive Intelligence Professionals (SCIP) — then a dynamic and growing organization. So I was supposed (by those more junior) to be an expert — and was asked by one of them the excellent question, “What is it that we CI people do?”

I didn’t mention to her that I was still working this out in my own mind. But I did my best — explaining that competitive intelligence practitioners serve four fundamental roles:

• Understand our client’s business problem(s),
• Find information potentially relevant in addressing those problems,
• Convert that information into useful ‘intelligence’ to support the client’s decision making, and
• Communicate that intelligence back to our client.

I rough-sketched a diagram, always useful in illuminating the abstract. Several months later, I was pleased to be asked by the then-reigning gurus of CI, Ben Gilad and the late Jan Herring, to write a chapter outlining the analysis process for their upcoming textbook The Art and Science of Business Intelligence Analysis (now out of print.)

By then I’d had a chance to think more about the framework and share it with some clients. The result was as in Figure 1 — the first of several published iterations of the KVC.

As you see, I deferred to their name ‘business intelligence,’ which was in vogue at the time.  As I worked with it more, I broadened the end goal from’ results’ to ‘value’ — which became part of the overall name.  And I added a seventh process stage, Action — which was implied by the earlier ‘Execution.’  These were cosmetic changes — the model at core remained pretty solid.

I began to weave the seven-stage KVC framework into formal presentations that over time developed a global audience. (The internet was still a novelty, so this was done almost entirely through personal appearances.)

A client asked me to annotate the PowerPoint for his team’s use and reference.  This led me to think of this as a ‘user’s manual’ for any intelligence process. Thus the KVC Handbook took the graphics-rich form that it has today. My firm used it as the guidebook for our multi-day onsite KVC Clinic.

When I later lectured at Columbia, I had copies offset printed as a student text — by then in Version 4, which is still available.

Trapezoid and triangle

The KVC is most easily understood as two separate but connected process components (Figure 2):

• the trapezoid of Epistemic Resources (Data, Information, and Knowledge); and
• the triangle of Agency (Decisions, Actions, and Value production.)

Each component contains sequential sub-processes through which content moves over time — as in a manufacturing process.  The trapezoid culminates in Knowledge, the triangle in Value — hence the name Knowledge-Value Chain for the unified framework.

Bridging these two components is Intelligence — the transforming link between Epistemics and Agency.

Epistemic Resources support and merge into Agency in producing enterprise Value — however defined by that enterprise (i.e., company, government agency, NGO, etc.)

The epistemic proxy

I designed the KVC as a general template potentially useful in a range of client situations. Does it still work in our AI-dominated world?  Let’s see…

Generative artificial intelligence (GAI) aspires to (and claims to) serve as a proxy for the Epistemic trapezoid — a lower-cost, faster-acting substitution. Having been a business researcher for much of my career, I can attest that doing this effectively and efficiently would be a major breakthrough in the knowledge-value calculus.

Real-world GAI use cases

But does it work?  In reading industry and academic research papers on GAI — as I do almost daily — it’s tempting to become bedazzled by technicalities, abstractions, and benchmark leaderboards. So in gauging GAI’s usefulness, I make a point of asking people who use these products how they are using them, in what circumstances they work best, how they would cope if they were suddenly deprived of access to them, and so on.

I’ve found that people’s GAI uses can broadly be described by the GPT acronym contained within one of the earliest commercialized LLM-based models, ChatGPT. “Generative,” “Pre-trained,” and “Transformer” each summarizes one major aspect of GAI use cases.

GPT description

My anecdotal observations are supported by several independent studies finding that GAI initially focused on three major use cases, often in combination:

1. Create content (output — the Generative aspect). This is the ‘talking book’ feature enabled by Natural Language Processing (NLP). People draft emails, blog posts, term papers, reports, computer code, images, videos, sounds, even novels and scientific papers. Studies say that upwards of 50% of internet content today is AI-generated.
2. Find and retrieve data (input — the Pre-Trained aspect). Many people use GAI as a higher-powered version of search — without the time-consuming need to click links and review the underlying source information. Basically this is achieved by querying a vast set of ’training data’ with user prompts. While training data sets may be proprietary, more often they are widely available on the internet. (Wikipedia entries and Reddit posts are reported to account for over half of the content of major commercial LLMs.)
3. Execute some transformation between (1) and (2) (throughput — the Transformer aspect.) The general transformer model was pioneered by Google in 2018 and is the ‘engine’ powering LLMs. Except for models specified as open weight, each company’s proprietary transformer implementation is a closely-held trade secret. Consequently, though it is a key product element, this remains largely a ‘black box.’ It’s powerful and wide-ranging, enabling translation among languages — as well as content transformations like transcribing meeting notes or converting a text narrative into a Powerpoint deck.

KVC mapping

The KVC was designed decades before GPTs mainstreamed in 2022. But since KVC is a generic intelligence template, it’s not surprising that each of G, P, and T also represents a key element of the KVC process.

Specifically, these align to the lower ‘trapezoid’ part of the KVC — shown in a more detailed version (Figure 3) that includes (in yellow) each transform step:  Acquire, Process, Analyze, and Communicate.

• Generative describes what Communication does (the Output.)
• Pre-trained corresponds closely to the KVC’s Data Acquisition (the Input.)
• Transformer matches the KVC’s Processing and, to a lesser extent, Analysis-Synthesis (the Throughput.)

A GPT, in effect, pre-emptively replaces the human execution of these ‘trapezoid’ steps.  Several open questions remain:

• How effective is this substitution, in terms of the quality and reliability of the output?
• How efficient is it, in terms of benefits received relative to costs incurred (i.e., the ROI)?
• What is the long-term effect on human intelligence of assisting (or replacing) the sub-steps: identifying data, locating and retrieving it, cleaning and organizing it, analyzing and synthesizing it, and fashioning it into some useful?
• What’s the purpose of this replacement?  For me, the process of intelligence is itself interesting and fun — so much so that I made it my life’s work.  It escapes me why anyone would want to forego that ‘joy of knowledge.’  I sympathize with those who do this willingly — and feel protective of those who fall into it unwillingly and/or unknowingly.

The argument we generally hear is that GAI saves time, thereby increasing productivity.  That said, the hard evidence remains murky — and many organizations are finding GAI’s ROI challenging to quantify.

The future

So much for LLMs Phase One — Chatbots that provide rapid answers and content. But what about the emerging LLMs Phase Two — Agents, where real work is expected to be done?

As pioneering modeler George Box said, “All models are wrong…,” while adding,“…but some are useful.” (We note that presumably Box was not defending all wrong models as necessarily useful — just some of them.)

The KVC is no exception — it deserves tweaking and/or qualification in some respects. (More on this to come.)

What makes a model useful?  To me, ‘useful’ consists primarily of being:

• Diagnostic — able to identify and correctly diagnose present problems and opportunities, and
• Predictive — able to chart and help navigate the course ahead.

Let’s evaluate the KVC against these two criteria, in reverse order.

Is KVC predictive?

Regarding ‘predictivity,’ I’m pleased to find that, at a high level, the stated serial intentions and direction of the GAI industry map closely to the KVC, as shown in Figure 4.

According to formal studies and my informal observations, most current use of GAI is as ‘answer bots’ — producing Intelligence that could be used as the basis for a human decisions and actions. Google’s AI Overviews are a widely-known example of this.

A logical step would be to make Decisions algorithmically (‘decision bots.’) Closely tied to that are AI agents that autonomously take Actions. At the highest stage of development, there are envisioned multi-agent clusters, whose job is to autonomously create Value, as defined by the user. This can range over a wide range of intended outcomes for example:

• online shopping,
• booking complex travel arrangements,
• building an investment portfolio, and even
• selecting targets for drone attacks.

Each of these ‘future’ developments is already being discussed, prototyped, tested, and (in a few cases) deployed. So we’re not stretching to say that the KVC promises to hold up conceptually in the near future.  As such, it could be used as a planning tool for both AI providers and clients.

Is KVC diagnostic?

Here, things get more complicated.  The KVC was originally conceived as a diagnostic tool — ‘Where is the chain broken?’ is the construct I used in my early descriptions. I have used the KVC with clients and to analyze some public case examples, for example Boeing.

Until now, though, I had not applied it directly to GAI. Let’s try it and see what happens.

Data are foundational

The KVC model makes explicit that:

1. Our world stands on Data — like Atlas, the mythical titan on whose shoulders the earth was thought by the ancients to stand.
2. The quality and relevance of Data are therefore crucial to analysis, decision making, and action-taking. Data is the raw material for all of those, in succession.

While ‘good’ (high-quality, relevant) data doesn’t guarantee good decisions and outcomes, those results are significently less likely in the absence of good data.

Where do data originate?

Mythology aside, where do data come from?  When I, a business researcher, developed the KVC, what I had in mind at the foundation were Data acquired in an intentional, goal-seeking way — including consumer interviews, scientific research, even technology-based intelligence, for example, that produced by satellites.  Highly curated and quality-assured prior to input, in other words.

During my summer as a bio-engineering research assistant, we worked essentially the same way:  we had a hypothesis, against which we collected relevant lab data in order to rigorously test the validity of that hypothesis.

In contrast, much of the training data of commercial LLMs consists of words scraped robotically from the internet.  There is little question that this is time- and cost-saving.  If this were to occur in a legal and ethical manner while maintaining parity of output quality, it would indeed represent a revolution in epistemic economics.  And this is essentially what’s being claimed by GAI sellers and promoters.

The question is, how valid as data are words on the internet?

Hyper-tokenization

In KVC terms, words and language are Information — literally ‘next level’ beyond the data acquisition and processing steps. So in pre-empting these steps with LLMs, we’ve rendered them superfluous — thereby yielding productivity, right?

Yes — if you assume that words form a reasonable proxy for data about the world.  That assumption is key to the entire LLM approach — but is it justifiable?

Language often contains elements of data — which explains why, when I worked as a corporate intelligence analyst, much of my time was spent in extracting the data (usually numbers) back out from the words of a text. I was, in effect, de-processing — moving back down the KVC in order to re-discover the data therein.

In effect, words are pre-tokenized — mediated expressions highly abstracted from reality. Words are ’signifiers’ corresponding (roughly) to those things ’signified’ (as shown in the table).  But (as noted by Saussure, Wittgenstein, and others) that relationship is essentially arbitrary:  my “dog” remains physically the same entity when my French-speaking friend calls it a “chien.”

When we tokenize on the already-tokenized, we’re essentially hyper-tokenizing — which rapidly compounds the noise inherent in each discrete tokenization step. This exposes us to bot hallucinations, randomized answers, model entropy and collapse, rogue breakouts, and so on — each endemic to the LLM approach, not just shortcomings correctible after the fact.

And that’s not the only problem. Words are inherently unreliable and unstable.  They are unreliable by virtue of being vulnerable to interpretation, misunderstanding, and misuse (whether intentional or not).  And they are unstable in that their meanings evolve over time, sometimes significantly.  Words adhere to only the most fuzzy of logics.

Words and numbers

Words complement — and differ in key respects from — our other sprawling symbolic system, numbers.  ‘Language people’ are typically thought to be unsuited or unwilling to understand mathematical concepts.  Likewise, ‘math people’ tend to see words (not to mention, other aspects of life itself) as imperfect, non-rational, and other messy and non-optimal things.

Words are, by nature, expansive; numbers are reductive.  Math distills and compresses expression down to its essence.  It’s concise, clean, rational, and elegant.

But when we use math to describe and delimit language — what GAI essentially does — we’re on philosophical thin ice.  In its drive toward concision and rationality, math intentionally jettisons the richness of thought behind language.

But — and I trust this doesn’t come as a big surprise — people are irrational.  Essentially so — that’s part of what makes us human.  There’s even a behavioral division within economics that studies this rigorously.

Side track

We design our information artifacts largely for the ultimate purpose of describing, mimicking, or representing the ‘true world outside.’ Most useful information systems — and even video games — are thus designed to stand in for some underlying reality. An inventory system, for example, shows us, at a glance, what is in stock — without our having to physically inspect and count the items themselves.  Likewise, a GPS maps the road ahead — so we can navigate our car accurately down the physical road.

Seen in that light, models built entire on words seem little more than an interesting side track — a diversion, a red herring, an epiphenomenon, a dead end — however spectacular and endearing. And, with all the money and media attention, a distraction — the result of which (if it collapses, as some predict) could result in a next ‘AI winter.’

Added to which, leveraging pre-existing tokenization by scraping from the internet has left the GAI industry having to defend against a cascading chorus of content creators demanding consent, credit, and compensation for the use of their work (as I discussed previously.)

“The light is better”

We know the instructive story, however apocryphal, of the drunk looking for his lost keys under the streetlight — not because that’s where he likely dropped them, but because the light is better there.

Looking for ‘truth about the world’ within language is convenient, especially now that so much of it is readily available digitally on the internet. But this begs the question: is the internet just a streetlight under which we’re all looking for our ‘lost keys’ of truth about the world?

L of a difference

There’s general agreement that LLMs are unreliable as truth-finders (though some argue that they’re as good as people in that regard.) It’s likely that this is because they sit on an unreliable foundation — language, rather than on the world that language represents.  LLMs are like apartment complexes built on a foundation of sand (which has also been tried, with disastrous consequences.)

Some of the most advanced AI researchers apparently understand this. World models — as advocated by AI thought leaders Gary Marcus, Yann LeCun, Demis Hassebis, Mira Murati, and others — have some obvious advantages over Word models (LLMs.)

And new products like Jev promise to get ‘beneath the words.’

The cost of a wrong turn

Why does any of this matter? LLMs are enormously expensive to build — requiring exotic hardware and astronomically-paid expertise. If their usefulness is not at least commensurate with the current headlong AI build-out, we’ve expended lots of money and effort that could have been more wisely invested elsewhere. Some analysts say that we’re throwing vast quantities of good money after bad, competing hard in a non-winnable, negative-sum game.

Our challenge

Taking the long view (as I often do), I’m reminded that written language itself is a relatively new human artifact — having been invented a little more than 5,000 years ago.  That’s less than 2 percent of our 300,000 year run (so far) as the human species.  It’s entirely conceivable that we’re just getting ‘warmed’ up on using our language tools most effectively — and that this process describes how we’ll continue to evolve.

I’ve employed a KVC analysis herein to suggest that the ‘far future’ of AI may require a fundamental reconsideration of what data is — the data that most closely describes, even mimics, the world — something ontological and sub-linguistic.  Alongside that will be critical questions of data value, ownership, control, and sovereignty.

These are conversations that will matter.  They will require the convening of experts in analytics, psychology, philosophy, economics, and governance (of both enterprises and societies.).

On a personal note, this weaves together a bunch of seemingly disparate threads I’ve pursued in my education and career.  It’s an interesting time to be alive!

featured

---