In Head‑Hunting, Big Data May Not Be Such a Big Deal

This interview with Laszlo Bock, senior vice president of people operations at Google, was conducted and condensed by Adam Bryant.

Q. How is Big Data being used more in the leadership and management field?

A. I think there’s been a fairly recent confluence of the ability to crunch lots of data at fairly low cost, venture capital investments that support new businesses in this field, and changes in what people expect. Leadership is a perennially difficult, immeasurable problem, so suddenly people are saying, “Maybe I can measure some piece of it.”

Part of the challenge with leadership is that it’s very driven by gut instinct in most cases — and even worse, everyone thinks they’re really good at it. The reality is that very few people are.

Years ago, we did a study to determine whether anyone at Google is particularly good at hiring. We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship. It’s a complete random mess, except for one guy who was highly predictive because he only interviewed people for a very specialized area, where he happened to be the world’s leading expert.

Q. What else has Google done in this field?

A. I have to preface the answer by saying that when we look at any data related to our people, we treat the data with great respect. Typically, we give people an option to participate in anything either confidentially or anonymously. The lesson for anyone looking at this space is that you need to construct this really powerful tent of trust in the people gathering the data and how they use it.

We’ve done some interesting things to figure out how many job candidates we should be interviewing for each position, who are better interviewers than others and what kind of attributes tend to predict success at Google. On the leadership side, we’re looking at what makes people successful leaders and how can we cultivate that.

We’re also observing people working together in different groups and have found that the average team size of any group at Google is about six people. So we’re trying to figure out which teams perform well and which don’t. Is it because of the type of people? Is it because of the number of people? Is it because of how they work together? Is there something in the dynamic? We don’t know what we’re going to discover.

Q. Other insights from the studies you’ve already done?

A. On the hiring side, we found that brainteasers are a complete waste of time. How many golf balls can you fit into an airplane? How many gas stations in Manhattan? A complete waste of time. They don’t predict anything. They serve primarily to make the interviewer feel smart.

Instead, what works well are structured behavioral interviews, where you have a consistent rubric for how you assess people, rather than having each interviewer just make stuff up.

Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.” The interesting thing about the behavioral interview is that when you ask somebody to speak to their own experience, and you drill into that, you get two kinds of information. One is you get to see how they actually interacted in a real-world situation, and the valuable “meta” information you get about the candidate is a sense of what they consider to be difficult.

On the leadership side, we’ve found that leadership is a more ambiguous and amorphous set of characteristics than the work we did on the attributes of good management, which are more of a checklist and actionable.

We found that, for leaders, it’s important that people know you are consistent and fair in how you think about making decisions and that there’s an element of predictability. If a leader is consistent, people on their teams experience tremendous freedom, because then they know that within certain parameters, they can do whatever they want. If your manager is all over the place, you’re never going to know what you can do, and you’re going to experience it as very restrictive.

Q. Other examples?

A. Twice a year, anybody who has a manager is surveyed on the manager’s qualities. We call it an upward feedback survey. We collect data for everyone in the company who’s a manager on how well they’re doing on anywhere between 12 and 18 different factors. We then share that with the manager, and we track improvement across the whole company. Over the last three years, we’ve significantly improved the quality of people management at Google, measured by how happy people are with their managers.

We’ve actually made it harder to be a bad manager. If you go back to somebody and say, “Look, you’re an eighth-percentile people manager at Google. This is what people say.” They might say, “Well, you know, I’m actually better than that.” And then I’ll say, “That’s how you feel. But these are the facts that people are reporting about how they experience you.”

You don’t actually have to do that much more. Because for most people, just knowing that information causes them to change their conduct. One of the applications of Big Data is giving people the facts, and getting them to understand that their own decision-making is not perfect. And that in itself causes them to change their behavior.

Q. What are some things that the managers are ranked on?

A. Some of them are very straightforward — the manager treats me with respect, the manager gives me clear goals, the manager shares information, the manager treats the entire team fairly. These are fundamental things that turn out to be really important in making people feel excited and happy and wanting to go the extra mile for you.

Q. Other insights from the data you’ve gathered about Google employees?

A. One of the things we’ve seen from all our data crunching is that G.P.A.’s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there’s a slight correlation. Google famously used to ask everyone for a transcript and G.P.A.’s and test scores, but we don’t anymore, unless you’re just a few years out of school. We found that they don’t predict anything.

What’s interesting is the proportion of people without any college education at Google has increased over time as well. So we have teams where you have 14 percent of the team made up of people who’ve never gone to college.

Q. Can you elaborate a bit more on the lack of correlation?

A. After two or three years, your ability to perform at Google is completely unrelated to how you performed when you were in school, because the skills you required in college are very different. You’re also fundamentally a different person. You learn and grow, you think about things differently.

Another reason is that I think academic environments are artificial environments. People who succeed there are sort of finely trained, they’re conditioned to succeed in that environment. One of my own frustrations when I was in college and grad school is that you knew the professor was looking for a specific answer. You could figure that out, but it’s much more interesting to solve problems where there isn’t an obvious answer. You want people who like figuring out stuff where there is no obvious answer.

Q. Any crystal-ball thoughts about how Big Data will be used in the future?

A. When you start doing studies in these areas, Big Data — when applied to leadership — has tremendous potential to uncover the 10 universal things we should all be doing. But there are also things that are specifically true only about your organization, and the people you have and the unique situation you’re in at that point in time. I think this will be a constraint to how big the data can get because it will always require an element of human insight.

In terms of leadership, success is very dependent on the context. What works at Google or G.E. or Goldman Sachs is not going to be the right answer for everyone. I don’t think you’ll ever replace human judgment and human inspiration and creativity because, at the end of the day, you need to be asking questions like, O.K., the system says this. Is this really what we want to do? Is that the right thing?

This interview was edited and condensed.