Eliminating Selection Bias: Why is it so hard?

Episode #202 Eliminating Selection Bias: Why is it so hard?

Despite the very vocal and public focus on diversity and inclusion in the last 10 years or so, it still appears to be a huge struggle for most organizations. It appears that the inertia created by the entrenched structures, processes and attitudes are way harder to overcome than anyone might have imagined.

As with any change, overcoming inertia is critical. Speed and momentum are necessary prerequisites for change, which is why trying to do things incrementally is so often inadequate. I hear some CEOs say that they believe in evolution, not revolution. This seems to be a mindset that guarantees you’re going to go nowhere.

When it comes to diversity, many countries, industries and businesses have put rules in place to encourage speed of adoption and to accelerate the rate of change. Once again, this is clearly not working in most cases. Almost 20 years ago in the US, the National Football League, or the NFL as it’s known, put in place something called the Rooney Rule.

THE ROONEY RULE

The Rooney Rule was designed to eliminate selection bias against African American coaches in their sport, but it’s recently come under severe criticism. It’s not really the rule itself that’s the issue, but more the way it’s being applied by team owners and front office management. Are they genuinely adhering to the rule or just paying lip service to it?

At the end of the last NFL season, Brian Flores, an African American coach was sacked from his job as Head Coach of the Miami Dolphins – so he’s in the market for a new job. But earlier this year, he brought a discrimination lawsuit against the NFL for the way the governing body and the 32 NFL teams handle coaching staff appointments.

Flores said in the complaint, “The NFL remains rife with racism. The racial discrimination has only been made worse by the NFL’s disingenuous commitment to social equity.” At one point, he even goes so far as to say that in many ways, the NFL is “managed like a plantation.” All white owners watch games from their luxury boxes, reaping in billions of dollars while the majority black workforce puts their bodies on the line every week.

Ouch! But he makes a good point…

In his lawsuit, Flores alleges that he was contacted by the New York Giants for a sham interview for their head coaching position – even though the team had already decided to hire another white candidate. And he has the text messages to prove it. Aki Ito wrote a great article on this earlier this year for Business Insider. It gives a great summary of the issues at hand and offers some very sensible solutions, which I’ll touch on shortly.

The Rooney Rule was implemented by the NFL in 2003, in an attempt to increase diversity in coaching positions. The rule stipulates that for any head coaching job, the panel running the selection process has to interview at least one person of color. The idea was that this would result in more diverse representation in head coaching roles, and eliminate some of the obvious selection bias.

The Rooney Rule seemed to have some initial success, and was even adopted by businesses like Amazon and Facebook, which put their own versions of the rule in place. But as I write today, there are only three head coaches of color out of the 32 NFL teams. So after almost 20 years, it seems there isn’t anything that remotely resembles a level playing field. Players of color make up 70% of the NFL playing ranks, but less than 10% of the teams are currently headed by coaches of color.

Interestingly, because of the highly quantitative nature of performance in sport, a lot of research has been undertaken to determine if there is systemic bias in the NFL. And apparently there is. With the focus on head coaching jobs, there’s little to regulate what goes on below that level in the coaching staff. One piece of research found that black coaches were 88% less likely than white coaches to get promoted to the coordinator level, which is the level immediately below head coach – their direct reports, so to speak. And this was even when they had similar levels of performance and experience.

So in other words, the disadvantage for black coaches is already baked into the system. The discrimination starts far earlier than when they’re up for the top job, preventing most of them from ever getting there.

There’s one other point that I think it’s important to make in the interests of balance. Flores claims that he was discriminated against by the Miami Dolphins, who he says wouldn’t have fired him If he were white. He says that with his win-loss record, many other white coaches would’ve kept their jobs.

It reminded me of a number of high profile cases in business over the past couple of years, where people claimed to have been sacked purely because they were female, and if they were male, they’d still have their job. This may or may not be true. It’s really easy to cry foul and pile onto the bandwagon of righteousness when claims like this are made. But in my experience in the corporate world, this generally wasn’t the case. Particularly in the last 10 years or so, where boards and management teams have become more aware of the consequences of taking discriminatory action.

Often these events are more driven by fear and incompetence than they are discrimination. Now, in Flores’s own statement, he gives a couple of examples where he had a significant difference of opinion with the team owner of the Dolphins. So was he sacked prejudicially because he’s black? Was he sacked because he clashed with the owner? Was he sacked because his win-loss record wasn’t considered good enough? Was he sacked because his star players underperformed? Was he sacked because the front office team management didn’t think he had the capability to build a Super Bowl winning team? Was it a combination of all of these? Or was it something else entirely?

I guess we’ll never know, but the point is, there’s always more to this than meets the eye. It’s really easy to falsely attribute an event or an outcome to a single cause. And let’s face it, whether we’re talking about senior corporate executives or NFL coaches, these guys aren’t being paid minimum wage. They’re at the top of their game (pardon the pun), in the rarified air where markets reward performance above all else – head coaching jobs in the NFL attract salaries of five to ten million dollars per year.

SOMETIMES THE LOVE IS JUST GONE

In pretty much any senior role, you’ll be hired under an employment contract, and that contract will have a number of clauses dealing with how the employment arrangement can be terminated. Most contracts have what I like to call a “love is gone” clause – think of it like a no-fault divorce. When the love is gone, either side can opt to terminate the contract after a certain notice period is given and contractual financial payments are made. This is pretty routine, and I’ve relied on these clauses in the past.

Now, “love is gone” terminations are different to terminations for cause – in other words, as a result of poor performance. But when you sign one of these contracts, unless you’re incredibly naïve, you know what it means: I’ll be paid to do this job only as long as I’m doing it really well and delivering value for the people who pay me. It’s that simple. It’s not personal. It’s most often not prejudicial (although it can be). But this, my friends, comes with the territory.

THE DYNAMICS OF SELECTION BIAS

There’s no doubt that systemic bias exists in hiring – but sometimes, even with the best intentions, it’s really difficult to improve diversity. Quite often, you are simply dealing with a shallow pool of talent. In the NFL example, the concept that you don’t have enough people of color with the experience to be head coaches is clearly ridiculous, as they make up the lion’s share of the NFL player roster. But there are some industries where this is not the case.

For example, in engineering and technology-based industries, the pool of candidates is still predominantly male. And this probably starts quite early on in life, as girls have traditionally been steered away from the STEM disciplines in their education choices. Fortunately, this is now changing pretty rapidly, but when I was running CS Energy, only 13% of the engineering talent graduating from universities were female. Is it any wonder that you don’t see women dominating the leadership ranks of these types of companies?

The difference though, isn’t as simple or as obvious as it might seem. Sure, we had issues with being able to find females who had the experience or background to fill many of the more senior roles. But there were forces acting in the organization that made this harder than it should have been – even allowing for the scarcity of the female candidate pool. Let me give you an example of selection bias in action:

We were looking for a strong project manager to lead the overhaul project at one of our power stations. There were several candidates, one of whom was female. She was an excellent project manager – probably better than most we had at the time. Now the hiring manager who was accountable for the overall asset management process was two layers removed from me, and he had some pretty firm ideas about the requirements of this project management role. One of those requirements was that the project manager had to live onsite in Central Queensland for the duration of the overhaul itself. With a young family and as the main caregiver, it wasn’t possible for that female project manager to spend a couple of months away from home. And the hiring manager, like all of us, tended to hire in his own image.

So, he recommended a male who was able to relocate for the duration of the project. This guy typically worked long, hard hours and he was a good operator, but I wasn’t happy with the selection. The one-over-one sign off fell to an executive who reported to me. So I had a quiet word to him and suggested that the project manager didn’t have to be onsite for the full duration of the overhaul project. I asked him to go away and think about a way to build some more flexibility into the role itself so that we could appoint the female candidate.

He came back to me a day or two later, having spoken to the hiring manager and he reinforced to me all the reasons why the role needed to be filled by someone who could live at site full time. The executive chose to support the hiring manager with the decision. I suppose this is understandable because jumping in and overruling him would’ve had other serious consequences – and it wasn’t like we were getting an inferior candidate.

But the moral of the story is: when you have entrenched beliefs and attitudes at lower levels of the organization, it acts as a barrier to progress – not just for women, but for any minority group that’s not the typical appointee. The homogenous, safe candidate is often effectively ‘green lit’ all the way through the process. It’s sometimes hard to find the right candidate that will improve outcomes through diversity. But it’s even harder to change the mindset of those who occupy those positions of power, who are making the decision. Perhaps Brian Flores is bringing attention to that point now.

SEVEN TIPS TO REDUCE OR REMOVE SELECTION BIASES

I’ve collected seven ideas that represent some pretty practical and manageable steps to reduce or remove biases in the selection process. We can all do these things if we are serious about eliminating selection bias.

1. Look at diversity through a wide-angle lens 

If you see diversity just as gender balance, or cultural diversity, or minority group representation, you’ll potentially focus on the wrong things and make mistakes in the other direction. It’s also really easy to discount a diverse candidate when you make it a binary choice. Do I hire the man, the choice that makes me feel comfortable? Or the woman, the choice that I feel is risky?

Looking at all types of diversity, leads you to think of candidates who are “not me”, and that’s a better place to come from. So think about this:

• Diversity of experience in different technical disciplines.

• Diversity of industry experience.

• Diversity of company size.

• Diversity of the countries the candidate has worked in.

• Diversity of personality types.

I could go on and on! Once you start looking, you can find a way to add more value to almost any team by bringing in fresh ideas and new ways of working. And these only come with diversity.

2. Diversify the hiring committee 

You can’t hire in your own image if there are multiple images to satisfy. Having more diverse selection committees can often remove the natural prejudices that we all subconsciously carry around. Bringing balance to the committee can bring balance to the short list, and your ultimate selection.

3. Give your diverse hiring committee blind resumes 

For some roles, blind resumes work really well. These are the resumes that don’t have names attached, so you have no idea whether someone is male or female, or if they’re from a certain cultural or ethnic background. Admittedly, this would be a bit of a struggle for something as high-profile and publicly-transparent as NFL coaching jobs. But for many situations, this removes a layer of selection bias that often lurks just below the surface.

4. Take longer to hire 

One of the criticisms of the process for hiring NFL coaches is that it all happens at a bit of a flurry. At the end of the regular season, teams start announcing the sacking of the coaches who haven’t lived up to expectations – and it becomes very obvious, very quickly who’s in the market. The teams that need coaches move really quickly to secure the best candidates. And it’s highly likely that the pressure to decide quickly forces the selection biases to be even stronger.

If you can wait a little longer for the right candidate, then wait. On more than one occasion as a senior executive, after doing all the shortlisting and interviews, I decided I didn’t like any of the candidates. So I sent the executive search firm back for another round of the market. You have to live with these decisions for a really long time, so don’t rush them.

5. Preset the hiring criteria 

If you do this, you are less likely to eliminate or discount unfavorable variables – those that don’t match your image of what the candidates should look like. You can easily fall prey to attribution bias.

A big criticism of some business research is that it suffers from attribution bias. One of my favorite books is The Halo Effect by Phil Rosenzweig. In it, he uncovers this as a key failure of research in books, like Good To Great. The researchers take an outcome, and then they interview the protagonists to retrospectively attribute the reasons for success. This is not the same as the traditional research discipline of having a hypothesis and then a range of predetermined dependent and independent variables.

Using preset selection criteria in hiring and evaluating the candidates rigorously against those criteria helps to avoid selection bias.

6. Grow your talent from below too

The only way to implement long term solutions is to make structural changes to the way the organization works. In the case of the NFL, this would require the Rooney rule to be extended to all levels of coaching and all areas of non-player performance, including the front office staff.

In CS Energy, the only way we could reliably get executive-ready females was to bring them in at lower levels and support their development through the early parts of their career. Is it slow? Sure, but that’s what’s going to make the long term difference so the structural inequities are eventually removed. When we combine this with the other measures, this isn’t an either/or, it’s an “and”.

7. Think of the best person for the team, not the best person for the job

There’s definitely a premium for diversity, as long as you can harness and draw out the value it brings. This takes a leadership approach that makes inclusion and robust challenge an essential part of your culture.

Sometimes if you evaluate a candidate based purely on the criteria of the role itself, you’ll arrive at one answer. But when you start thinking beyond just what the role requires, to what you need to add to the team to make it better, this can help you to quantify the diversity premium. Thinking of the team lets you factor this premium into your decisions.

We’re all subject to many unconscious biases, and they work behind the scenes to skew our perceptions. Selection processes are the rare opportunity you are given to improve the quality and performance of your team. You don’t want to risk that because of a hidden bias you can’t control.

Want to know more about unconscious biases and how you can avoid them? Listen to Episode 192: Avoiding Common Biases.