Why Can't Johnny Quantify Risk?

(With apologies to Rudolf Flesch)

Cybersecurity departments have been encouraged from the start to learn to speak the language of business - risk. But this has been a bit like asking them to drive a NASCAR race in the family sedan. We haven't equipped our cybersecurity people with the tools they need to run that race, nor have we trained them to drive at their full potential.

Qualitative vs Quantitative Risk

Let's start by recognizing that quantitative risk is relatively difficult to do well to begin with. Things like actuarial tables, actual calculated numbers, and other variables that depend on actual or appropriately derived values. Those values require knowing quite a bit about the company, various revenue streams, sunk, and operating costs. In many organizations nobody really knows those numbers outside of perhaps a few individuals in the finance department.

Qualitative risk is easier to come by in comparison, but also less reliable. Estimates, educated guesses, and basic one to ten rankings are acceptable in this paradigm. However, even those require a firm grasp of the realities of the organization, the environment, and outside forces. However, in the vast majority of organizations this is the risk paradigm that can be supported.

So we've already set up one hurdle for Johnny Cyber, his organization probably can't support quantitative risk analysis anyway.

Cyber Risk ≠ Business Risk

Cyber risk is certainly a business risk, but it isn't business risk in and of itself, in part due to language. In cybersecurity we talk about risk in terms of things like exploitable vulnerabilities open to the Internet. We speak of the risk of clicking a malicious link in an email, and the like. These are serious risks, but need to link to actual business risks to matter to the business. How will those vulnerabilities impact income? How will they cost the company money? How will they impact the organization's reputation?

Yet these are the sorts of things we train our cybersecurity people to think about: vulnerabilities and the like. And cybersecurity experts can tell you that any one vulnerability could be the one that lets the bad actors do whatever it is they wanted to do. Probability is a mess at best. Impact is no better. Confidence in detecting the malicious activity ranges anywhere from pessimistically low to ridiculously overconfident, which also skews the analysis.

Let's stop asking Johnny to do something the rest of our organization can't do either.

Technical Tools for Technical Purposes

Have you ever taken a look at the landscape of cybersecurity tools out there? In the past, Momentum Cyber has posted their "CYBERscape," showing their view of the cybersecurity vendor landscape. Have a quick look:

In 2023 Momentum Cyber identified 18 different categories of cyber solution types. Of these, only the following are not what I'd call dominated by technical tools:

• MSSP
• Risk and Compliance
• Security Ops and Incident Response
• Security Consulting and Services

Even these generally had a number of "just throw a tool at it" vendors in the category, or offered people-based services that output technical solutions. Even the Risk and Compliance category itself is subdivided into Security Awareness & Training, Pen Testing & Breach Simulation, and Risk Assessment & Visibility (which has most of the vulnerability identification vendors in it), before having small categories for GRC and Risk Quantification. Almost as if the vendor community hasn't seen fit to focus on this very important task, and the observers are having to justify the idea that yes, some vendors exist for this space, see?

Taking this a step further, what kind of output do we get from these technical tools? Well, that output is generally valuable to technical workers: system administrators, threat hunters, SOC analysts. When's the last time you got a risk-based report out of your firewall? Or a business-aligned risk report out of your VM tool?

It seems clear to me that we aren't providing Johnny with risk conversation ready tools and outputs.

Competing or Nonexistent Risk Standards

Depending on your point of view we either have a plethora of standards for assessing, managing, and reporting on cyber risk or we have none. So there is no reliable way to compare organization A to organization B, and no way to compare the security program of organization C's departing CISO from the program implemented by their new one. There aren't even agreed upon metrics to form the basis of such an analysis.

Perhaps it is fair to expect a CISO to choose from among the standards that exist and then build their risk program to that. After all, the CISO is supposed to be the domain expert. That said, of those standards, which even relate to business/organizational risk? Which relate to it in a way that matches the corporate risk profile?

Could we offer Johnny an approved, viable risk framework/standard/program to use?

Why Are We Talking About Risk Anyway?

If we whittle the concept of risk down to the fundamentals, it is really a measure of two things:

• The probability of a "bad" event happening
• The cost of the impact of that "bad" event

In some fields, that probability is easy to identify because we have strong record keeping, historical data, and a relatively predictable set of conditions:

• Healthcare (you have a less than 3% chance of surviving linger than 5 years with this particular cancer)
• Home Insurance (your house is unlikely to flood due to you living on a mesa, however you chances of it being blown over in a tornado are very high because you live in tornado alley)
• Car Insurance (your chances of having your car totaled in a crash are once in every X miles of driving in your area)

But in cybersecurity we have no such probability markers:

• What is the probability that your company will be infiltrated via an unpatched KEV?
• What is the probability that your company will be infiltrated by a malicious insider?
• What is the probability that your company will even detect the "bad" event?

We simply don't have this data. We don't require every company report on this data, and in many cases I'm not even sure an "autopsy" gets performed in order to get the information necessary to do identify the root cause or initial point of infiltration.

Setting probability aside, do we have a handle on event cost? That one is pretty difficult as well. Will your breach likely involve law suits? Will it involve government fines and sanctions? Will your insurance provider cover it or cut you loose? We don't have the history that, say, the auto insurance industry has to help them determine what various types of collisions are likely to cost.

All of that suggests to me that perhaps we're barking up the wrong tree. Maybe risk isn't the best frame of reference for modern cybersecurity conversations.

Really? Stop Mapping Risk?

I know, I just committed heresy and my membership in all the trendy cybersecurity clubs is probably about to be revoked. But let's face facts:

• We don't have a commonly accepted risk framework for cybersecurity
• Our tools don't talk to us about "risks," they talk to us about technical details
• We don't have the actuarial data to really do quantitative risk assessments

I recall reading somewhere that the definition of insanity is doing the same thing that doesn't work over and over again in the hopes it will finally work. (Or something like that)

I recently had the chance to hear John Kindervag discuss an alternative idea: focusing on danger. You may know John through his previous work, which included introducing the concept of Zero Trust in cybersecurity. John's argument boils down to this: probability doesn't matter, action does.

For example, when there's a tornado nearby you take cover - you don't guess how likely it is to hit YOUR house, you get in the basement. Why? Because you're protecting your most important asset, yourself and your family. You have a clear understanding of what you want to protect and you know what to do to protect it. It is a compelling argument, but will require some radically different thinking both within cybersecurity circles and within corporate leadership.

So What Can We Do?

It seems we have two basic options. We can keep banging our heads against the "risk" wall and hoping to chip away at it before we give ourself concussions, or we can look to another way.

Perhaps Johnny still won't be able to quantify risk, but perhaps he can provide more risk-aligned content and input to the organization, and that's a good start.