Tim Wood Powell     The value of knowledge

I began my career long ago in the insurance business, where risk is (literally) the coin of the realm. Companies buy and trade risk — it’s the currency of our volatile, uncertain, complex, ambiguous (VUCA) world.

In conversation, we often associate risk with something dangerous, to be avoided. And, in a technical sense, risk has a downside, known as threat. But it also has an upside, known as opportunity.

Two sides of the coin

These two sides of the risk coin can offset each other. Opportunity may carry a possibility of threat — if I take that new job, what happens if it turns out I don’t like is well as my current one?

Conversely, threat often carries within it seeds of opportunity. When our downtown New York City apartment was flooded in 2012, we used it as an opportunity to build it back even better.

Risk evaluates the balance and trade-offs between these positive and negative aspects of risk. We can then move forward with making decisions and taking actions based on that evaluation.

We’re all exposed to risks constantly — that’s life. These risk exposures can be intentional (as when I buy a lottery ticket) or not (my car breaks down on the highway.) Prescription drugs, as near-miraculous as their effects may be, are required (here is the US) to list their potential side effects alongside their benefits.

As individuals, we usually conduct any risk evaluation quickly — even instinctively, without thinking about it.

Organizational risk

Organizations do risk evaluations too — though these are typically more deliberate and transparent. Larger organizations may even have staff assigned full-time to this enterprise risk management function.  At any given time, a given organization can face thousands of risks in different aspects of its operations,

Organizations must continually commit resources — time, effort, attention, and money — anticipating a future that is (by definition) unknown and unknowable. Organizations set budgets, for example, based on a best estimate of what financial results the coming year will bring — well aware that things could change by that time. Businesses typically take months developing and introducing a product, not knowing exactly how it will be received.

Managing risk

All of these investments in an uncertain future involve risk — that’s the definition of risk. So ‘managing risks’ does not mean avoiding them — which is not even possible. Managing risk successfully requires being aware of what the relevant opportunities and threats are, and what their potential impacts and likelihoods are.

Risk is a ‘key success factor’ — I’d argue, the most value-impactful one. Organizations that manage risk well are better performers overall — it’s one of those existential skillsets (like continual learning, to which it’s closely related).

Uncertainty becomes risk

You’ve heard the term ‘calculated risk’ — but that’s redundant, since measurement and evaluation are embedded within our definition of risk. All risk is calculated, whether or not such calculation is formalized.

A related idea we hear about a lot these days is uncertainty. (It’s the single word appearing most often in a recent survey of CEOs conducted by The Conference Board.)

Uncertainty is the stage before risk — where we ‘just don’t know.’ To move from uncertainty to risk, we measure it, as shown in Figure 1.

Risk mapping

But where do we start? How do we even know what to measure? Using a technique called risk mapping, we first describe the sources of uncertainty in our business environment. We can usually identify them as being opportunities or threats. But some things could be either one, or have elements of both, so we leave those as unknowns.

Opportunities can often be interpreted as benefits — so we associate those terms one one side of the spectrum. Likewise with threats and costs, on the other. The result is a risk profile, which generically looks like Figure 2.

Note also that these risk do not occur in a vacuum — they are contextual and depend on the specific business case to which they’re being applied.

An example — AI risk

How does this work in practice? One large and significant risk area that leaders in most organizations are currently facing is AI, particularly generative AI. (And this too is shown in surveys of people at other levels.)

We have developed a generic risk map for genAI. It has two sides, one targeted at buyers of these services, the other at suppliers. These are ‘generic’ in the sense that they apply across a range of organizations, and must be tailored for any given organization in order to become fully operational.

AI buyer risk map

The buyer (demand side) map describes 12 generic risk clusters, as shown in Figure 3. These are assigned as threat, opportunities, or unknowns based on the likelihood of where they fall along that dimension.

Further evaluation could result in their being assigned to a different cluster.

Each cluster is assigned a number only as a label — not to indicate its relative importance.

Here’s an example for each cluster.  First, the things that are obviously benefits:

• 01. Productivity.  Still the main selling point for LLMs, clustered into code generation, content (like emails and marketing pitches), and customer service.
• 02. ‘Knowledge in a bottle.’ The promise that we can access the world’s knowledge easily to solve our problem.
• 03. Amusement.  Let’s admit that these are interesting toys that we can have fun with.  In fact, deep within some license agreements, they basically claim that’s the limit of their legal liability.

Now some things that could fall either way — unknowns:

• 04.  Output quality.  In quality-sensitive use cases — medical and legal research, for example — LLM output has been found to produce quality significantly below what is commonly required.  If you’re a firm whose value proposition rests on ‘output quality’ — a law firm or consultancy, for example — this can become a direct link to reputatonal risk.
• 05. Litigation and regulation.  This one is especially fast-moving, as new regulations are enacted often around the world, and new litigation (mostly for copyright infringement and product liability) is filed weekly.
• 06. Copyrightability.  In the US, copyright policy currently requires human authorship for protection.  Producers of IP — software and music, for example — may find their AI-assisted output is not legally protected.
• 07.  Liability exposure. This is an emerging area in which the user of an AI can be sued, for example, for negligence.
• 08.  Vendor viability.  Most AI producers currently lose money — lots of it. While some are divisions of successful business that do other things, others are AI-only — and subject to boom-and-bust cycles.

Next, some things that usually fall on the cost side of the equation:

• 09. Security and privacy.  Cybersecurity attacks by bad actors are increasing exponentially — aided and abetted by powerful AI-powered hacking tools (for example, Mythos).
• 10. Input quality and provenance. All AI models depend on data as their ‘raw materials.’  If the training data corpus is unreliable or compromised, the resulting errors flow through to the outputs.
• 11. Cognitive offloading.  Several studies have shown that when a bot provides answers, people tend to rely on them and ease up on checking them.  Then, over time, their cognitive ability to do so can lose its sharpness — or even erode permanently.
• 12.  Return on investment.  ROI is the mother of all risks; all risks have value implications. When the benefits of AI are weighted against the costs — after being fully loaded for time lost to things like checking, rework, and fixing errors — as well as less easily measured things like reputational risk — the net gains may be small, non-existent, or even negative.  As I write, the cost of tokens is escalating rapidly — obviously another key driver.

Vendor assurances, however persuasive they may appear, are not sufficient.  You need assessments that are evidence-based and independent of bias.

AI seller risk map

An additional nine risk clusters are borne primary by the producers, the sellers of AI.  However, in practice they may in effect shift to the buyers of AI.

The provider (supply side) map, shown in Figure 4, is based on the input-throughput-output typology developed by Stanford University’s Human-Centered AI (HAI) lab.

These include upstream factors (model inputs), for example:

• 13.  Training data.  The quality, provenance, and legal status of training corpora can each add risk.
• 14.  Computing resources. The availability of capital to meet the unprecedented computing load of advanced models is finite.  Chip supplies may be constrained.  Data center construction is being opposed in many localities.
• 15. Labor.  The human labor required for model training and ‘reinforcement learning’ is significant.
• 16. Consumables.  Electricity and cooling water are also finite resources.

From there, the model itself has throughput risks:

• 17.  Model integrity.  The model can be functionally compromised by prompt injection, jailbreaking, or other hacks.
• 18.  Data integrity.  The privacy and security of the training data can be compromised.

And there are downstream risks in how the model is applied:

• 19.  Depiction of humans.  Impersonation, revenge porn, and child sexual abuse materials are a few of the more unsavory applications.
• 20. Depiction of non-humans.  Counterfeits and other IP theft are rampant, as I discussed here.
• 21.  Use of generated content.  Targeted personal attacks and propaganda warfare are enabled.

Note that these last five bullets — and much of the public awareness of these risks — comes from the producers themselves, in product cards and published papers.  I also lean heavily on independent industry and academic studies.  More recently, the court filings supporting the many lawsuits against, and even criminal probes of, these models have revealed new insights.

Probability-Impact

Behind each of these 21 risk clusters is set of specific risks for each organization — and for each business case therein.  And behind each specific risk is a PI(t) evaluation — Probability and likely Impact of an adverse event within time t.  So, yes, it’s complicated — at any given point in time.

Risk is dynamic

And it doesn’t end there. To complicate things even further, the risk ecosystem is in constant motion.  New AI risks and exposures arrive fresh nearly every week — and this seems to be a permanent state of affairs, with a likelihood of its accelerating even faster.

As our risk is dynamic, so must our management of that risk be dynamic — beyond static criteria and performative check-boxes.  Flexible, adaptive, resilient. As fast as risk evolves, risk measurement and management must adapt and anticipate even faster.

My message is not necessarily to avoid using AI.  Nor would I advise you to use a prescription drug before reading the warning label.  Rather, it’s to remind us that AI’s promises are offset by its costs and uncertainties — and to ‘look before we leap.’  As with any investment or business decision, we should evaluate these factors before committing significant resources — there’s no ‘AI FDA’ to do it for us.  We should ask the usual tough questions (1) What specific benefits does it potentially provide? (2) Are there other, more direct, ways to achieve these same benefits? (3) What are its true total costs — both current and future?

Photo: Hudson Sunset (2012), © TW Powell

---