Trusting Your Instincts: Data isn’t everything

Episode #217 Trusting Your Instincts: Data isn’t everything

Recently, we looked at high-impact decisions: the ones that have potentially significant consequences and that strike fear into the hearts of most leaders when they have to make them. These types of decisions require a lot of diligence, so it’s entirely appropriate that we seek to acquire as much data as possible, to eliminate any major risks, and to satisfy ourselves that we aren’t going to do anything stupid when the stakes are high… but the vast majority of our decisions just aren’t like that.

These days where we have cheap, instantaneous access to virtually limitless data sources, you’d think our decisions would be getting much better. But I’m not so sure… There’s no substitute for judgment, and there are times when your instincts have to guide your decisions, even to the point where they override the available data.

DATA ASYMMETRY IS DISAPPEARING

There is now so much information that many industries are actually finding they have to redefine their business models. The old days of information asymmetry are rapidly disappearing. For example, many years ago, buying a house was a much trickier thing – particularly given it represented the largest investment an individual is likely to make in their lifetime. It was really hard to get any useful information like:

• The history of sale prices on the house

• The current pricing of like-for-like comparators in the area

• The demographics and crime rates of the neighborhood

• Any modifications that may have been made to the property; and

• Any previous issues with the house.

These days, all of that information is readily available through online searches. You can find out pretty much anything you need to know about a house without leaving your own living room. So there’s way less information asymmetry between the buyer, the seller, and the agent and therefore there’s way less chance of making a poor decision.

The same goes for cars. When I bought my first car in 1980, I had almost no information to help me to work out whether I was getting a reasonable deal or if I was getting ripped off. I had to just trust the used car salesman who sold it to me. I know, a scary concept! And it does go some way to explaining their (largely undeserved) poor reputation. But hey, we all managed to survive, right?

But just contrast that with the car I bought 12 months ago – I knew a hell of a lot more! I knew every available price from car dealers within a hundred mile radius. I knew the pricing of all the optional extras. I knew the warranty and service benefits of every dealer. I could compare like-vehicles from other manufacturers. I read owner reviews from people who’d bought the car and driven it extensively in all sorts of conditions. I had access to a large number of independent ratings for both the vehicles and the dealers who sold them – and sure, I took a lot of guidance from all of that, but I still went for the one with the red leather seats and the sexy alloy wheels! The buying process is never entirely analytical. The key question is: does all this information guarantee better decision-making?

DOES MORE INFORMATION IMPROVE DECISIONS?

I listened to a podcast interview a little while ago with Malcolm Gladwell, the author who wrote one of my all-time favorite books, Outliers – if you haven’t read that book, I strongly recommend that you do. Gladwell was being interviewed on Steven Bartlett’s podcast, The Diary of a CEO, and they had a brief, but interesting exchange about whether or not decision-making had improved now that we have so much more information available.

Gladwell’s view is that clearly the proliferation of information isn’t making it any easier for us to make good decisions. On the contrary, he thinks our decisions are actually becoming worse. Now, Gladwell says that reality can be distorted. For example, we might have a dozen criteria for making a decision, but we don’t know how to weight those criteria properly. It’s hard for decision-makers to focus on too many things at once because there’s only a limited amount of space in the head. And as he says, “We clutter our decision-making process with extraneous information in the hope that it makes us better off in the end. It’s a fool’s game.”

His conclusion: focus on what’s most important. Too much information makes it harder for us to process the information we actually have. I may be suffering from a little confirmation bias here, but this is entirely consistent with my experience of decision-making. This is why my mantra of simplicity and focus really rings true in most areas of leadership.

With so much information available, though it also seems to be much harder to get to what I call the objective truth. The social trend today is to say that if any individual thinks something is true, then that’s a perfectly acceptable version of the truth. It’s not, it’s just an opinion – and opinions are often just plain wrong. But there seems to be an increasing trend for people in all walks of life to selectively cherry-pick partial facts to support their existing view or their desired case.

Reliable sources of information are now much more difficult to discern. I think one of the most fun expressions I’ve heard in years was the phrase that came out of American politics: “alternative facts”. I remember a press conference held after a professional NBA basketball game in the US in early 2017. A journalist posed a question to Mike D’Antoni, the head coach of the Houston Rockets. When asked to comment on the fact that the Rockets had lost five of their last eight games, D’Antoni said, “Actually, we won all those games. I’m going with the alternative fact thing.” Gold!

DATA CAN BE DECEPTIVE

I just want to briefly make two important points about the information that gives us false comfort when we make decisions:

We have a strong tendency to overvalue quantitative data

If it appears on a report that’s been generated from a database, if it’s a statistic that’s been generated as a result of academic research – even if it’s numeric rather than verbally descriptive, then we are much more likely to believe it and place overweight importance on it when we make a decision. We tend to not question the source as much as we should. For example:

• How did that information get on the database in the first place?

• Was it through a manual process (which is error prone by definition)?

• How accurate is it likely to be?

• What doesn’t the data describe or capture?

• What assumptions lie behind it?

• What was the research methodology used to support the hypothesis?

• Was that research paper peer reviewed and validated?

The fallacy of the implied level of accuracy

I first became aware of this when I began to critically analyze business cases to support investment decisions about 20 years ago. Let’s consider an example:

Perhaps a business case is seeking approval for a potential investment of $50 million. That business case might project a net present value (NPV) of $4,327,000 (in other words, that’s the likely positive return over the life of the investment). Because the NPV is expressed in the business case to the nearest $1,000, we intuitively ascribe a level of accuracy to that outcome that simply isn’t warranted. It may make us feel as though the positive outcome described is much more likely to eventuate because it implies a rigor in the underlying analysis that simply isn’t there.

When you look really closely at the underlying assumptions, there are often potential swings of many millions of dollars depending on which risks materialize. So in examples like this, quite often if the NPV turned out to be accurate to the nearest $5 million rather than $1,000, that would be a pretty good outcome.

THE ROLE OF JUDGMENT IN DECISION-MAKING

I came across an article in The Economist a little while ago, ‘When to trust your instincts as a manager’. The author makes a great point about the way organizational processes are designed to stamp out instinctive responses and reduce the amount of variability in decisions. Undoubtedly, this has its benefits, but it can also stifle the role that experience and judgment play.

One example cited in the article is really instructive: in a well-known experiment conducted about 10 years ago, volunteers were asked to assess whether a selection of designer handbags were counterfeit or real. Some people were instructed to operate on instinct, and others to deliberate over their decision. Intuition worked better for those who had owned at least three designer handbags in the past. Indeed, their intuition outperformed analysis. So what does this tell us? Well, the more expert you become, the better your instincts tend to be.

The article also mentions the well-established psychological phenomenon, known as verbal overshadowing. This captures the danger of overthinking things. For example, people are more likely to misidentify someone in a lineup if they’ve spent time writing a description of their faces. And because of our innate fear of getting things wrong, leaders often suffer from the problem of analytical overshadowing. They endlessly analyze a simple problem until it turns into a complex one.

Knowing when to use intuition in the workplace rests on its own form of pattern recognition:

• Does the decision-maker have real expertise in this area?

• Is this a domain in which emotion matters more than reasoning?

• Above all, is it worth delaying the decision?

The article concludes that slow thinking is needed to get the big calls right, but fast thinking is the way to stop deliberation turning into dither.

The moral of the story is that in general, we should trust our instincts far more than we do. We’re constantly taking in information ourselves:

• We use all our five senses

• We read people’s tone and body language

• We pick up on nonverbal and visual cues

• We use our experience and judgment to assess the reliability of the information that we analyze; and

• We calibrate multiple, often competing sources of information for consistency, patterns and exceptions.

We just need to listen to that inner voice.

Now, don’t get me wrong, it doesn’t mean we should ignore key facts or hard data. But it does mean that we should constantly challenge the data we see, particularly if it feels inconsistent with our instinctive reactions. We need to place a higher weighting on our instinct and experience, and we shouldn’t just accept the information that’s presented to us on face value. Simply learning to ask good, searching questions can bridge the gap between the data and our experience or gut feel.

Make sure you remain curiously skeptical of any information that’s positioned as an undeniable fact. Over time, experience will fuel your instincts. You’ll naturally get better at it, and you’ll be better able to judge what mix of data and instinct will give you the best results under different circumstances. A quote I often use from Will Rogers is, “Good judgment comes from experience, and experience comes from bad judgment.”

THREE FOCUS AREAS TO IMPROVE YOUR DECISION-MAKING INSTINCTS

So if this is all about improving your judgment so that you can get to the point where you can easily spot a genuine Gucci handbag from a knockoff, how do you get there? Well, I have three suggestions for where to focus if you want to improve your decision making instincts:

1. Focus on developing your own self-awareness

Self-reflection isn’t as common in senior leaders as you might expect. As time goes on, they tend to believe their own bullsh!t. Their ability and success in winning promotions leads them to think that they’re actually an expert in many areas when in reality, sometimes the only thing they’re truly an expert in is the art of getting promoted!

But many corporate executives – and yes I’m making a distinction here by not calling them leaders  – develop a carefully curated professional image over time, and this becomes the most important thing to them. They protect it fiercely, and they reject any information or feedback that threatens this self-perception.

There’s also the Dunning-Kruger effect, which is the tendency that we all have to perceive ourselves with a false sense of competence. We tend to overrate our capability in almost every area: everything from our driving ability to our competence in financial investing.

Simply being aware of these two facts of life should give you the motivation to constantly reflect and question yourself. That is, of course, if your aspiration is to be a strong and capable leader. Don’t just reflect on your ability, but reflect on your motives and behaviors as well. The greater the extent to which you can develop your level of self-awareness, the more likely your decision-making instincts are to be helpful and accurate.

Learn more about the Peter principle with Episode #50: Are All Leaders Incompetent?

2. Make sure you retain an internal locus of control

With an internal locus of control, you’ll feel a high level of responsibility for anything that happens around you. People with an external locus of control are likely to blame unfavorable outcomes on external factors, and not look at the part that they themselves have played. The trouble with this is they never properly evaluate their own poor decisions. If this is the way you view the world, your judgment will never improve.

Think of it like this: if you don’t recognise and accept the role that you’ve played in a less than favorable outcome, you will learn nothing. And if you don’t learn, then instead of ending up with 20 years of decision-making experience, you’re going to effectively end up with one year of experience repeated 20 times over.

3. You need to be disciplined about reviewing your decisions

One of the tools we developed at Your CEO Mentor is a decision tracking template. It simply enables you to track, understand, and review how you make decisions. It’s funny because many leaders think that this is beneath them, “Oh, Marty, I make decisions every day and I think I’m pretty good at making them. I’ve got a lot of experience.” There could be a touch of the Dunning-Kruger effect here…

When you actually track what you do in your decision-making process, it’s amazing the insights that emerge. You have to believe in the value of speed, and you have to know what makes a great decision so that you can preemptively work out whether a decision you are about to make is likely to be good, bad, or indifferent. Keeping yourself honest with diligent tracking can really help.

In the end, decision-making is a complex blend of art, science, and intuition. If you want to get really good at it, you need to focus on the things that will most help to improve your experience, your judgment, and your instincts. But it’s surprising how many leaders are prone to ignoring the very things that will make them better at what they do. Sometimes, I guess it’s just easier to believe your own bullsh!t…