Last modified: Thursday, February 14, 2013
IU's Kafadar takes ongoing effort to improve 'science' of forensic science to AAAS annual meeting
Statistician has co-authored federal reports on forensics, anthrax investigation
FOR IMMEDIATE RELEASE
Feb. 14, 2013
"With the exception of nuclear DNA analysis, however, no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source."
BLOOMINGTON, Ind. -- Such was the synopsis on the state of forensic science -- fingerprint and bullet analysis, hair and fiber evidence, blood stain and tire track analysis -- from a 2009 report from the National Academies of Science that Indiana University Rudy Professor of Statistics Karen Kafadar helped produce.
On Saturday in Boston during the 2013 annual meeting of the American Association for the Advancement of Science, four years later and with no implementation of any of the report's recommendations for improving the "science" of forensic science, Kafadar and colleagues from the National Institute of Justice and the National Academies will collectively ask, "Will there be real change?"
In its 2009 report, the National Academies of Science Committee reviewed published research on various forms of forensic evidence, including fingerprint analysis and other pattern and impression evidence like shoe prints, tire tracks and tool marks; hair and fiber evidence; paint and coating evidence; and arson, explosive, bite mark and blood stain pattern evidence. In all cases, it was unable to find comprehensive studies that addressed performance with high levels of confidence. And as a statistician, high confidence levels are what Kafadar and anyone else trained in using the scientific method depend upon for accuracy.
"That report found that the published studies of the accuracy and reliability of most forensic methods, other than nuclear DNA analysis, failed to meet the stringent criteria seen in other scientific disciplines," Kafadar said. "The scientific method is very general, and the principles of science apply to all branches, irrespective of the specific application, be it physics, chemistry or biology. If the scientific method has not been applied, then scientists from no branch can defend the results."
In the symposium presentation, titled "Critical Role of Statistics in Development and Validation of Forensic Methods," Kafadar not only identifies the continuing needs and challenges facing forensic science, but she also outlines a research agenda -- one that includes the involvement of statisticians -- that addresses the ways studies can be designed to better quantify the capabilities of forensic methodologies. She has been doing research on problems related to forensic science for a decade now.
"Only by better understanding both the strengths and the limitations of the methods can the totality of the evidence in a particular case be better evaluated," she noted.
Forensic science needs to develop a series of quality thresholds for evidence, to construct objective analysis steps, to design validation studies for each process, and to design reliability studies that reflect repeatability, sensitivity and specificity. One way to begin that process would be to look at examples of how statistics has already been used successfully in forensics, most specifically in the area of DNA analysis, and unsuccessfully, as in comparative bullet lead analysis, which the FBI later discontinued following the National Research Council's report in 2004. Kafadar co-authored that report also, along with the 2011 National Research Council's review of the anthrax investigation.
"Statisticians need to be involved in assessing the validity of the approaches to analyzing forensic evidence," Kafadar said. "And the methods and standards in forensic science need to be developed by scientists, not by law enforcement."
One example that will be highlighted during the AAAS presentation will be calling for an analysis of fingerprint identification methods similar to the type of rigorous study and analysis that DNA analysis went through before its use in forensics.
Fingerprint analysis uses an analysis, comparison, evaluation and verification, or ACE-V, methodology for which three of the four phases have been acknowledged as subjective in some degree. And the fourth phase, verification, is not conducted by an independent second reviewer, but rather by one who may be aware of the first conclusion.
In an upcoming paper, "Statistical Issues in Assessing Forensic Evidence," being prepared for publication in International Statistical Review, Kafadar notes, "Unlike DNA analysis, which was studied and evaluated in research laboratories all over the world, no extensive and comprehensive studies of performance of the latent fingerprint process or ACE-V methodology (that quantify the effects of multiple sources known to affect variability in performance), nor the error rates from the individual steps in the process, have been conducted."
If the federal government were to follow the National Academies of Science report's key recommendation, that a National Institute of Forensic Science be established and that it not be part of the Department of Justice, Kafadar said the future could bode well for young statisticians, a group already performing well in the technology job market.
"We would still have a long way to go," she said. "Statisticians need to be trained in design and process control, there needs to be a heightened sense of cultural awareness for the need for improvement in forensic science, and there is always the challenge of finding funding for internships."
For more information about this research or to speak with Kafadar, please contact Steve Chaplin, IU Communications, at 812-856-1896 or stjchap@iu.edu.
You can also read more about Indiana University's involvement at the 2013 annual meeting of the American Association for the Advancement of Science.
