Last modified: Thursday, August 25, 2011
IU math professor uncovers flaws in highly publicized 'obesity is contagious' study
FOR IMMEDIATE RELEASE
Aug. 25, 2011
BLOOMINGTON, Ind. -- An Indiana University Bloomington mathematician's recently published critique of a highly publicized study on obesity being socially contagious is now also garnering international attention.
Mathematics professor Russell Lyons' research, "The Spread of Evidence-Poor Medicine via Flawed Social-Network Analysis," published in the journal Statistics, Politics, and Policy, questions the conclusions made by Nicholas Christakis of Harvard and James Fowler of the University of California, San Diego, in their highly publicized 2007 paper "The Spread of Obesity in a Large Social Network over 32 Years" that appeared in the New England Journal of Medicine.
In their original paper, Christakis and Fowler claimed to have provided evidence of a "three-degrees-of-influence rule of social contagion" within networks such as families and friend groups where obesity characteristics could be transmitted socially. They went on to include other personal characteristics -- divorce, smoking and even loneliness -- in their social contagion theory, and their research took off in the media, accompanied by publication of their 2009 book "Connected" and appearances on the shows "Good Morning America" and "The Colbert Report." Christakis built on his research to form a company, MedNetworks, which proposes to help pharmaceutical companies get doctors to prescribe more of their drugs.
When Lyons reviewed the evidence, he found not only a lack of statistical significance in the findings, but also that both the researchers and the reviewers did not realize that the statistical procedures Christakis and Fowler had used were inapplicable. Their methods, Lyons found, were fundamentally flawed.
"Their studies even provided some evidence against the existence of social network transmission," Lyons said.
Lyons is not arguing that social contagion, or peer pressure, doesn't exist: He agrees that people influence each other. Rather, Lyons shows that the research did not support the paper's contention that one could measure how much people influence one another, whether the people are one, two or three steps out in the network of family or friends.
"The problem is that their methods were deeply flawed from bottom to top: The models used to analyze the sparse data contradict the data and the conclusions, and the method used to estimate the dubious models does not apply," he said. "The statistical significance tests that were applied to the questionable estimates do not show the differences they have proposed."
Even if the effects could be measured, the differences could have been created by the "birds of a feather" factor known as homophily, where there is no social multiplier or contagion to address, but rather people simply associate with others like themselves. There could also be shared environmental factors that influence outcomes.
"Their wrongly proposed differences, even if they were correct, do not distinguish among homophily, environment and induction," he said. "And associations at a distance are better explained by homophily than by induction."
In the past 90 days, The New York Times , The Boston Globe , Slate and the Irish Medical Times , among numerous other media, have written about Lyons' critique. The Strategy Research Initiative, a group that includes scholars from Harvard, Duke, Dartmouth, Columbia and Yale, now lists Lyons' research as an "Exemplary Paper" in the area of Theoretical Critiques of Empirical Work.
The first researchers to question the correlation between health outcomes to social network effects proposed by Christakis and Fowler were Ethan Cohen-Cole, now at the University of Maryland, and Jason Fletcher, a professor in the Yale University School of Public Health. They argued in 2008 that the study did not adequately control for homophily or environmental influences.
Among those who have since joined Lyons in questioning the results are Columbia University professor of statistics Andrew Gelman and Oxford University professor of statistics in social sciences Tom Snijders. Other critiques have come from Hans Noel, Georgetown University, and Brendan Nyhan, Dartmouth, and from Carnegie Mellon University statistics professors Cosma Shalizi and Andrew Thomas.
Lyons said his critique has not only brought to light problems with well-publicized studies related to human health, but it has also allowed him the opportunity to voice a broader criticism of how statistical modeling is misused, of the role of peer review in academia, and about the missing place for critique in scientific literature.
Both of the leading, prestigious journals that published research by Christakis and Fowler -- the New England Journal of Medicine and BMJ (formerly British Medical Journal) -- rejected Lyons' critique, the first declining to give a reason and the second saying the work would be better placed in a specialist journal. Rejections then came from three other leading journals on the grounds that they had not published the original research. A statistics review journal rejected Lyons' paper on the basis that the original research of Christakis and Fowler was itself not sufficiently important.
As for the status of statistical modeling, how it is reviewed in journals, and its present state in academia, Lyons cites a 1998 account from Doug Altman, the current senior statistics editor at BMJ, to make the point: "The main reason for the plethora of statistical errors is that the majority of statistical analyses are performed by people with an inadequate understanding of statistical methods. They are then peer reviewed by people who are generally no more knowledgeable. Sadly, much research may benefit researchers rather more than patients, especially when it is carried out primarily as a ridiculous career necessity." Lyons' paper also cites prominent social scientists to make the same points regarding their fields.
Lyons, also an adjunct professor in the IU College of Arts and Sciences' Department of Statistics, said he teaches his students how to find the hidden assumptions and flaws in statistical studies.
"I tried to get high-profile journals interested in my critique, telling them about the wide interest the topic had, but to no avail," he recalled. "I don't think I got any special treatment; these were simply their policies. In fact, that's one of my main points: From flawed statistics in research to poor-quality review to difficulty in publishing critiques, this is all too common. And that's why I think these are important points, ones that I emphasize to my students when I teach statistics. They need to be wary."
To speak with Lyons, please contact Steve Chaplin, University Communications, at 812-856-1896 or stjchap@indiana.edu.
