To my great interest I read recently that Google has this year abandoned the abstract brain-teaser type interview questions which have help earn them a reputation as one of most difficult companies to interview for worldwide.
For those unfamiliar with the topic, previous interview questions include such gems as:
- How many golf balls can fit in a school bus?
- How many piano tuners are there in the entire world?
- How many vacuum’s are made per year in USA?
Mind-bending questions such as these are not uncommon in recruiting, particularly among consultancies, investment banks, and technology companies like (until recently) Google. The problem is, as a predictor of job success, these type of questions are not particularly helpful. Or, as Lazslo Bock, senior Vice President of people operations Google puts it “a complete waste of time.”
“… They (brain teasers) don’t predict anything. They serve primarily to make the interviewer feel smart. Instead, what works well are structured behavioural interviews, where you have a consistent rubric for how you assess people, rather than having each interviewer just make stuff up.”
To many, the conclusion that brain-teasers are not a useful indicator of job success will come as no surprise. Though a person may be good at coming up with a creative solution to an abstract problem under pressure, it is not a given that they will face similar pressures in the workplace – fast thinking is not the same as smart thinking.
So if brain teasers are not the answer, then what is?
Interestingly (and although I have written previously about the numerous biases that can challenge their validity), research indicates that interviews remain one of the better indicators of job success, albeit only when certain conditions are met.
Firstly, the use of a standardised structured interview process is critical. Historically, job interviews are quite unstructured. That is, there are few standardised questions or scoring criteria for evaluating different candidates. A structured interview by comparison is standardised such that the same questions are asked in the same way and in the same order for all candidates.
Unsurprisingly, unstructured interviews have been demonstrated to have very low reliability and validity due to (among other things), interviewer bias, misalignment of decision criteria and hiring standards between interviewers, and disagreement on evaluation criteria. Structured interviews in contrast have quite high validity as a predictor of job performance (more than 100% increase in validity in some instances!).
In addition to following a structured format, interview questions should be behaviourally focused. Comparisons of question formats have found that behavioural-based questions (e.g. tell me about a time that you did X) are more valid than other types of questions such as future-oriented situational questions (e.g. what would you do if X?).
Finally, interviews should be combined with other selection methods. One meta analysis of 85 years of personnel selection methods found that along with general mental ability, and performing a work sample test, a structured interview had one of the highest validities as a predictor of job performance. Additionally, the validity of the structured interview method was even further increased when combined with tests of general mental ability, which yielded a 23% increase in validity over structured interviews alone.
It is a credit to Google that they have identified insufficiencies in their selection process and made moves to improve it. Other organisations could learn a lot from this example, with the benefit of avoiding bad hiring decisions that can be extremely costly.
Latest posts by Andrew Comensoli (see all)
- Behavioural Economics in Action: ScootBiz Case Study - September 16, 2015
- How Passive Leadership is Destroying Your Culture - July 3, 2015
- The People Dimension of Strategy Execution - December 9, 2014