Smart Managers Don’t Compare People to the “Average”

NICOLE TORRES: Welcome to the HBR IdeaCast from Harvard Business Review. I’m Nicole Torres. Our guest today is Todd Rose. Todd is the Director of the Mind, Brain, and Education Program at Harvard’s Graduate School of Education. He focuses on how to promote the principles of individuality and the practice of personalisation in the workplace and education and in society. He’s here to talk to us about his new book called THE END OF AVERAGE: How We Succeed in a World that Values Sameness.

Todd, thanks for joining us.

TODD ROSE: Yeah. Thanks for having me.

NICOLE TORRES: So our society seems to measure everything from behavior to performance to body size in terms of the average. So we use GPAs and standardized test to tell us how smart a student is. And employers use performance reviews to evaluate how well an employee is doing. And I think everyone is used to measuring skills and achievement against some norm.

But you say that this way of understanding people is scientifically wrong. So what exactly is wrong with averages?

TODD ROSE: So averages are really good when your goal is to understand the whole group. And for most of our time as a society, particularly since the Industrial Revolution, we really focused on the group and kind of just slotting people into their proper stations in life. But, nowadays, we’re shifting over to a deeper understanding of individuals, whether it’s in medicine or personalized learning or better ways of hiring and assessing performance. It’s that moment when you go from caring about the group to understanding individuals that averages actually break down. And they’re actually really bad ways of understanding individuals.

NICOLE TORRES: Yeah. So before we get into why we need to stop thinking in terms of averages, can you tell us a little bit about where the original idea came from, why we started thinking in terms of groups in the beginning? And why we embrace the practice of judging people in terms of what’s normal?

TODD ROSE: So when I started writing the book, I was actually interesting that exact question. The idea of the average person and comparing ourselves to that is so ubiquitous in society. It kind of just feels it’s just always been true.

But it turns out this idea of this average man– and they called it man at the time– was invented by one person, a guy named Adolphe Quetelet in the early 1800s in Belgium. He was an astronomer. And in astronomy, you actually use averages quite a bit to estimate, say the speed of Saturn. He would take a bunch of observations and it turns out if you average them together, that tends to give you a better measurement.

So he was living through a time when we got the first census data. We were collecting lots of information about people, but no one knew what to do with that, this jumbled mess of data. And he had this insight that would if you can apply the method of averages that we used in astronomy to understand people. So he does that, but what he carries over with it is a sort of moral judgment, I don’t know what it means.

So on the one hand, you can always calculate an average. But he thought for sure what it meant was that when you calculated say, the average size of a person’s body in Belgium, that that average size was true. That it was the real thing that he said had been divinely intended and everything else was just error around this average. So the average man becomes real and we all become just a cheap approximation of it. It was really weird even at the time. So his contemporaries thought he was a little bit crazy, like we’re obviously not an average person.

But it was so valuable for bureaucracies to manage whole populations for public health, for– we invented insurance off of it. So there were lots of practical benefits to managing society they came with thinking in terms of averages. And you know what? Those were real benefits. I think that we’ve all benefit from it.

But it seeped into the way we started to think about ourselves as people. So we’ve come to see ourselves either on average, like how close we can approximate it, or how far we can get above it.

NICOLE TORRES: Right. So the idea of being better than average or worse than average. And this is problematic.

TODD ROSE: Right.

NICOLE TORRES: So what has changed exactly that makes using averages to understand people no longer useful? Why are we suddenly more focused on the individual?

TODD ROSE: Of course it was always wrong, scientifically, but the value that it had at the societal level, when we had an industrial economy, where basically through a guy named Frederick Taylor, we had basically standardized almost the entire workplace, and we’ve kind of deskilled the jobs as best we could. And you just wanted a standardized worker to plug into the system.

And off of that approach, we built education systems that were meant to create standardized outputs for the economy. At no point in that process did we care to know about individuals. Frederick Taylor, the father of scientific management, flat out said in the past man was first, now the system must be first. So it was an explicit choice to focus on efficiency of the system. But nowadays, if you think about the standard jobs, they’re gone. Right?

NICOLE TORRES: Right.

TODD ROSE: We’re a knowledge economy. We’re a creativity economy. We need innovation from all aspect of these different sectors. And so we’re increasingly focused on getting the most out of every individual. And that’s true in the workplace. It’s also true in education where there’s been a rapid shift toward personalization and development of talent rather than just batch processing and sorting.

So in that environment, in our modern society, where we all expect to have a better focus on individuals, this average and ranking approach breaks down miserably.

NICOLE TORRES: Yeah. So I want to talk about when evaluating someone based on averages actually results in bad outcomes. And your book mentions a few big name companies that discovered how doing this, this approach, wasn’t actually useful for building teams and understanding their employees. So you talked about how Google started to rethink its talent recruitment, for example. And you talk about how Deloitte became skeptical of performance rankings, and how Microsoft eventually abandoned its stack ranking system. So what happened in each of these cases and what did these companies find out?

TODD ROSE: So ultimately, what they found was when you think in terms of averages, when you collapse all the complexities of being a person, or thinking about talent, into one dimensional terms– so when you think of averages you see this kind of bell curve, most people are in the middle, and then there’s a few exceptional people up top and some of the people that are dragging down the curve on the bottom. So within an average based system, that one dimensional view it is how you think about people. And it has to be true for averages to work.

But what we found was that when you start looking at talent and performance, it is always, always multidimensional. And that what we call it as jagged, that people are good at some things and not so good others. So the problem is as soon as you try to reduce talent to a single score, you actually lose the essence of what you’re trying to measure.

So in the case of performance reviews, what Deloitte found was that the problem is when you try to give someone a number, one to five, you’re trying to describe what they’ve done into one single number which doesn’t really exist. And so you don’t understand what their true performance really is.

Now I will say, this is what everyone has found. This is what Google found when they tried to use one dimensional rankings like SAT scores and grades to hire. They were just missing out on all kinds of talent. And what I think is important here is these are companies, these aren’t small upstart companies, right?

NICOLE TORRES: Right.

TODD ROSE: These are giants who, actually if you think about it, really win if we keep the system as it is. Google can out pay almost anybody for the best talent. So why would they suddenly abandon one dimensional ways of hiring?

NICOLE TORRES: Right. Why would they?

TODD ROSE: Because it wasn’t working. At the end of the day, they’re obsessed with trying to understand not only, did the person we hire turn out to be as good as we thought? But also following the people they don’t hire, and did they turn out to be better than they thought they would?

And so there was a gentleman name Todd Carlisle who started this initiative at Google. And they did this massive study of like a billion data points on trying to understand what really predicted success at Google. And they had their traditional metrics. They thought GPA, SAT, and the prestige of your alma mater. And those turned out really not to predict much at all. I think grades predicted for the first couple years and then they don’t. And instead, what they found there were a collection of predictors, and there was no one predictor that mattered for every job, so what they ended up doing is a much more customized view of– so if it’s about being a programmer, what are the things that predict, at Google, for being a good programmer? And so at the end of the day, they’re just getting better talent than they would have otherwise.

And with Deloitte, imagine how much time we spend reviewing each other at work. And it better payoff, right? And what I think is really great about Deloitte’s done is it’s one thing to say what doesn’t work. It’s another thing to start going in a direction of something that’s better.

And they’ve really put a lot of effort into thinking about what’s the alternative? And as it stands now, they’ve been working on something that replaces one dimensional rankings with like four basic questions that every manager of every project is asked about the employees. And then you get this really interesting, contextualized picture over time of say, for example, how well you’re performing under different projects with different teams. So that rather than just saying, you’re a three, whatever that means, I can really understand, look with this kind of manager on this kind of project with this kind of team you really thrive. So rather than trying to rank you, we’re going to find the place in which we can get the most of your talent.

NICOLE TORRES: So those two things, understanding that talent is jagged and that context is important, is two of your three principles of individuality, which I do want to get to, but the big question I had when I started your book was how do we evaluate people without using averages? So wouldn’t it too messy or too unmanageable if teachers, if managers, had to focus on the individual every time instead of where this individual fits in the distribution? How do you measure progress without some benchmark or baseline to compare things to?

TODD ROSE: Sure. So I think there’s two parts that. One is, of course, you want some benchmark to compare to, right? Actually, it turns out comparing them to other people isn’t necessarily the best way to do things. There is a time for that, obviously. At some point you have to hire an employee and that means hiring this employee instead of these other potential employees. So it’s not that you can’t ever compare, it’s that whenever you collapse to very narrow, one dimensional metrics that basically benchmark against average, it will mask the things you actually are trying to understand.

So what we focus on is knowing what are the multiple dimensions that you really need to be measuring at the same time. And then, as you already mentioned, the context aspect which is it doesn’t work to say here’s some generic dimensions that are predictive everywhere. It’s just not like that. And so for a company, who cares if SAT predicts something on average in the population, right? If it’s not predictive at your company, it’s a useful thing to apply. And so really just focusing on more dimensions, but without going crazy, without getting too crazy, the dimensions that matter, the minimum set, and taking context into account.

It’s not that you’re not going to compare people sometimes, it’s just that you’re going to compare them on the right kinds of things.

NICOLE TORRES: Right. And is there a point where it does get too messy? You know, because we do have a tendency to use a one dimensional scale to think about complex human traits because that’s easy and that’s efficient, and it’s easy to wrap our heads around. But how do we decide where the cutoff point is? How many actual dimensions matter for something verses are irrelevant for the thing we’re trying to measure?

TODD ROSE: So that is now the holy grail of everything we look for now in terms of going down this path of in HR and other things. I’ll give you a parallel here which is when the Air Force is designing things, like for say a cockpit or a flight suit, they keep a database of 147, I believe, dimensions of body size. If you had to keep track of all those every time you wanted to like design a cockpit, that would just overwhelming, right?

NICOLE TORRES: Right.

TODD ROSE: Except for it turns out, you want to design a cockpit, it only takes 10 dimensions, but you’ve got to know which dimension those are. And if you design a flight suit, it’s different. And so what we’re looking at now is to say, OK, there’s lots of dimensions, you could look at but there’s a diminishing return pretty quickly. And so there’s this balance between– we know for sure it’s never just one dimensional. So if you’re using one dimensional metrics, it’s quite certain that you’re actually failing at finding talent. But in terms of looking in your context and saying, what are those critical dimensions? And usually, the truth is it ends up clustering out a handful of them. But there’s no getting around the fact that HR folks and talent people actually have to figure out what the dimensions are the matter most for their company.

NICOLE TORRES: And you had mentioned already the second principle of individuality that you talk about in your book, which is the traits are a myth and we need to focus on context. And the example you give, that you’ve touched on already, is that this happens with job descriptions. We call for key qualifications and skills, but then we don’t focus specifically on the context of the job. So what is wrong with this? And what would be a way to fix this kind of problem?

TODD ROSE: So I’ll step back and say the trait we’re missing seems provocative, because that’s what we talk about, except for in the sciences, this is exactly what we find. So if you take personality, for example, we often think oh, I’m an extrovert. Well that’s nice to say, and personally I would think that I’m an extrovert. It feels like intuitively true. But when we actually take in context, seriously, what we found is first of all, when we try to use these traits, they are terrible at predicting behavior, like really bad at predicting the behaviors that they should predict. And so people always said, well human beings are messy. Well that’s a very good answer, right? We could be a little better than just we’re messy.

And so there’s a scientist named Yuichi Shoda out of University of Washington who actually figured out the answer which was we are really stable in our behaviors and personality and performance in the way that it changes in different contexts. So for example, I am extroverted around people I know really well, but I’m pretty introverted if it’s people I don’t know. So it’s what we call if-then signatures. If this in the context, then this is the kind of behavior to expect. And it turns out when you think of talent and behavior and personality from the if-then approach, it’s actually really predictive of how someone will perform or behave.

So when you shift that over time thinking about how you hire people, if we say look, here’s the job description, must be a good communicator. Well there’s no such thing as that. And what it means to be good communicator is very, very different if you’re a customer service representative or you’re an accountant. And so what’s funny– and this seems so unbelievably intuitive, but we haven’t done it that much– is when you look at people like Lou Adler, The Lou Adler group who pioneered what’s called performance based hiring, their whole secret is don’t use these generic job profiles, define the contextualized performance that you’re looking for and then find people that actually have those kind of performances. And they crush it. I mean they are so much better than the traditional approach to hiring.

But of course, if I say you must have 10 years of experience, why? Where is suddenly magically 10 years of experience means you’re going to be good at the job? So really focusing on the contextualized performance first, and then finding people whose performance profiles fit that.

NICOLE TORRES: Do companies that do that, are they proven to be more successful than companies that don’t?

TODD ROSE: Yeah. So I was really surprised, frankly, at the success rate of these kind of approaches. So my view is you would think when there’s money to be made that people would immediately shift over to these approaches, but The Lou Adler Group has done a great job and they’ve got plenty of good evidence showing, and validation studies showing, how much more effective they are. And they’ve worked with thousands and thousands of companies and hiring managers. So then, for me, I thought well, why isn’t this more widely adopted? And I think there’s two big barriers to it. One is just a mindset. If you continue to rely on this way of thinking about people, it’s hard to see the value of performance based hiring. But the second thing is most of our HR infrastructure, and the technology we’ve developed, reinforce this old way of thinking.

So I think where you’re going to see the biggest breakthroughs are in start ups and stuff like that, where they are free from the legacy of this old way of thinking and they can use new technologies to focus contextualized performance.

NICOLE TORRES: So I do want to talk about the third principle, which focuses on pathways. And I think the title of the chapter was “How We All Walk the Road Less Traveled.” And you took an unusual and, certainly, not average path to end up where you are today. I was wondering if your background factored into how you thought about this, how people’s career trajectories are different and that there’s a problem with trying to standardize them?

TODD ROSE: Yeah, for sure. I mean I think most scientists or most professionals– the thing you end up doing for your career, usually a mix of intellectual curiosity and then some kind of personal motivation. So I think for me, obviously my background is having dropped out of high school and then being successful, working my way back through college while I was married and working full time–

NICOLE TORRES: With two kids.

TODD ROSE: With two kids, and being on welfare, and going to Harvard for my doctorate, and then becoming a faculty member. The way that I was successful was very, very different than– whether it was on the theory side when I was taking psychology class about how learning and behaviors are supposed to happen, it did not seem to map to how I would be successful at all. But also we give this generic this is how you should progress through school and if you’re not on that path, then you’re behind. And when you get into a career, there’s this normal trajectory, supposedly. And we do this all the time with succession planning. And we’re like OK, well if you’re not at this point, then you must not be great.

And frankly, I think what we’ve ended up doing is, first of all we know now from if you just career pathways, this idea that there’s a career path is just nonsense. If we go back to Deloitte, they have a whole approach on the lattice structure of careers realizing that there’s multiple pathways, always, to even the highest the positions in the company. And we’ve seen this when we study careers of scientists. And this is a basic principle that there’s always more than one path to an outcome.

But if that’s true, I think what we’ve done is we’ve limited talent pools. And in particular, I think for example, in corporate careers, it’s really hard to have a child. And if you’re not keeping up with this normal pathway, then you’re not getting the kind of promotions because we’re assuming only the people that are keeping up are worthy of promotion.

So I think that back in the day when we had a standardized economy, where there were only a certain number of good jobs and everyone gets slotted to where they’re supposed to be, maybe you could put up with the failures of this kind of approach, maybe the trade offs were even worth it. But right now, they’re just not, where we’re starved for talent. It’s crazy that we’ll spend so much time trying to overpay the best people, but not invest time in discovering the hidden talents all over the place that were there for the taking.

NICOLE TORRES: Because they don’t fit some model.

TODD ROSE: Right. A model that’s not actually true to begin with.

NICOLE TORRES: Right. So because of how deeply ingrained this idea is, our reliance on averages, do you think it’s possible for us to leave it behind? Who needs to be convinced for things to actually change?

TODD ROSE: So that’s a great question. And off of this book, I’ve also started a nonprofit called The Center for Individual Opportunity and we have been kind of quiet right now until now we’re starting our public phase, where we’re actually launching a very large social movement campaign. Because our bet is in a democracy and in a market based economy, public demand wins, ultimately. And if you don’t change the public mindset, it doesn’t matter what kind of solutions you build. So our bet is to take these ideas and focus on how we think about ourselves as individuals, like how do we as a people think about ourselves and then extend that to what does it mean for how we create our institutions of opportunity? So our first wave which will happen later this year, is going to be just broad scale really going after the general public, and frankly try to make average a four letter word.

NICOLE TORRES: And when do you move into companies? So how can companies start adopting these principles of individuality?

TODD ROSE: So what we’ve focused on in company work is developing some proof points, which we’ve got some things going on that we haven’t been public about yet. But also shining a spotlight on things in which the companies have been able to win on these ideas. In the book, I mentioned some, and we’re continuing to create more spotlights on others, and showing the language for it and helping people have models to see.

I talked about Costco or Morningstar. It’s not that you want your company just to be exactly like that, but if you can see enough examples and start to understand what aspects of those success stories work for your company. So we think basically there’s two things. One is you’ve got to change the public mindset. And then you’ve got to put enough models in people’s hands to show off the power of the new idea.

NICOLE TORRES: Well, great. Thank you. I think that’s about all the time we have today. Todd, thanks so much for joining us.

TODD ROSE: Thank you.

NICOLE TORRES: That was Todd Rose and his book is called The End of Average. For more, go to hbr.org.