Last modified: Monday, February 27, 2012
Study: Long-held view of 'bell curve' in performance measurement proven flawed
Study debunks myth of 'normal distribution' in performance, suggests overhaul of theory and practice in all elements of workforce management
FOR IMMEDIATE RELEASE
Feb. 27, 2012
BLOOMINGTON, Ind. -- The dreaded bell curve that has haunted generations of students with seemingly pre-ordained grades has also migrated into business as the standard for assessing employee performance. But it now turns out -- revealed in an expansive, first-of-its-kind study -- that individual performance unfolds not on a bell curve, but on a "power-law" distribution, with a few elite performers driving most output and an equally small group tied to damaging, unethical or criminal activity.
This turns on its head nearly a half-century of plotting performance evaluations on a bell curve, or "normal distribution," in which equal numbers of people fall on either side of the mean. Researchers from Indiana University's Kelley School of Business predict that the findings could force a wholesale re-evaluation of every facet related to recruitment, retention and performance of individual workers, from pre-employment testing to leadership development.
"How organizations hire, maintain and assess their workforce has been built on the idea of normality in performance, which we now know is, in many cases, a complete myth," said author Herman Aguinis, professor of organizational behavior and human resources at Kelley. "If, as our results suggest, a small, elite group is responsible for most of a company's output and success, then it's critical to identify its members early and manage, train and compensate them differently from colleagues. This will require a fundamental shift in mindset and entirely new management tools."
According to Aguinis and his co-author, Ernest O'Boyle of Longwood University (soon to join the University of Iowa), the entrenched notion of normality -- notably in performance evaluations that force managers to assign only numeric or category ratings -- is detrimental to individuals, the group and the larger organization. They suspected that any group, regardless of size or industry, would show a pattern with a few elite performers ("the best") dominating the many ("the rest").
The researchers amassed a database of more than 600,000 individuals and conducted separate studies applying normal and power-law distributions to assess performers in four carefully chosen fields:
- Academics in 50 disciplines, based on publishing frequency in the most pre-eminent discipline-specific journals.
- Entertainers, such as actors, musicians and writers, and the number of prestigious awards, nominations or distinctions received.
- Politicians in 10 nations and election/re-election results.
- Collegiate and professional athletes looking at the most individualized measures available, such as the number of home runs, receptions in team sports and total wins in individual sports.
"We saw a clear and consistent power-law distribution unfold in each study, regardless of how narrowly or broadly we analyzed the data," said Aguinis, who also is director of the Institute for Global Organizational Effectiveness at Kelley. "For example, with the athletes we could look at performance within leagues, within teams or specific positions, but the shape of the distribution was constant."
Aguinis and O'Boyle believed that the power-law distribution would also identify outliers at the other end of the performance spectrum -- those likely to engage in unethical or illegal behavior. "Counterproductive work behaviors" are often covert and thus challenging to assess, so they again used sports samples, examining such elements as yellow cards in soccer and first-base errors in baseball to find negative performance largely attributable to an individual. Here too, the results conformed to the power law.
"All five of our studies suggest that organizational success depends on tending to the few who fall at the 'tails' of this distribution, rather than worrying too much about the productivity of the 'necessary many' in the middle," Aguinis said.
Broad implications for society, but challenges abound
Aguinis noted that the power-law approach has applications in groups of all types and sizes, including governments, nonprofits, education systems and corporations. However, changing theory and practice will be challenging, due partly to deeply entrenched notions of fairness and equality in society and business. Further, it could pose difficult ethical dilemmas, because it requires taking care of the "superstars" first in the context of treating everyone fairly.
"Dedicating extra effort, time or money toward a handful of employees will seem anathema to managers or human-resource professionals accustomed to thinking in terms of parity, what's best for all or most, or what's applicable to a set pay grade or position," Aguinis said. "Similarly, an educator may struggle with the notion of investing more time in top students -- who would bring up overall class scores -- rather than trying to improve everyone a little bit, which won't move the needle."
Aguinis cautions that top performers can also exhibit personality traits such as narcissism and selfishness that will have a terrible impact on an organization. "When we recommend looking for 'superstars,' we mean those who don't just do their jobs well, but also create a good work environment and enhance productivity for the entire team, unit and organization," he said.
The bottom line, according to Aguinis, is that everything about individual performance has to be re-evaluated so managers can identify and go after lead performers.
"These people will be desirable to outside firms, so success means thinking about excellence and improvement all the time, talking with top performers continuously to find out what they need to grow and advance," he said. "Rating them once a year, based on a bell curve, will send top performers -- and profits -- right out the door."
The paper, "The Best and the Rest: Revisiting the Norm of Normality of Individual Performance," is published in the spring 2012 issue of Personnel Psychology.
