The New Old Razzle Dazzle
November 20, 2012I'm Seth, and I'm a skeptic. I think skepticism should be an important part of how the public views the legal system. It's no secret that the American public is tragically illiterate when it comes to science. As long as the science does not explicitly mention evolution or climate change, the public is generally willing to defer to expert scientific authority without understanding the details. We watch a lot of primetime television dramas that feature sexy scientific gadgets solving crimes and demonstrating beyond any reasonable doubt in the viewer's mind that the suspect is guilty (or not).
In the industry they call this the "CSI Effect." Among fans of the musical Chicago, we call it the Old Razzle Dazzle. Every good skeptic knows about this stuff. We know science is the best way to discover empirical facts about the world, like what happened, who did what, why something looks the way it does. We also know that not everything with "science" in the name is scientific (see: "creation science"), and that confidence in legitimate scientific results ranges depending on a variety of factors. So, as long as one steers clear of evolution and climate change, it is quite easy to abuse scientific authority to ease the task of convincing people of something.
In the words of Ron Burgundy, "It's science."
In the courtroom, it is the New Old Razzle Dazzle. Attorneys use the authority of science, obfuscate the evidence and confuse the jury.
Billy would have done better to get his forensic science certificate.
In the Casey Anthony trial, as in many trials, we saw a phenomenon called the Dueling Experts. Each legal team gathers a group of experts, who then give conflicting expert testimony. The jury will likely believe the expert that supports whatever intuition they had before hearing the Dueling Expert testimonies. Unfortunately, judges like to expedite the trial process wherever possible, so they often depend on certificates and whatnot when ruling someone an expert. So, rather than expertise being evaluated by scientific peers, it is evaluated by a judge, based on formal credentials in areas like "forensic odontology" or "certified forensic examiner." It is not meant as an insult when I say that if judges are experts in anything, it is the interpretation of the law, and not the application of the scientific method to criminal investigations. This is not to say they are stupid, but to say that they aren't the most qualified people to judge expertise in matters scientific. That's what the peer review process is for.
Maggie, my third year law student girlfriend, tells me that the only case she can think of where expert testimony gets thrown out involves a genius law clerk who read an appeals case and informed the judges that the expert testimony was inherently misleading, and the prosecutor relied on fundamentally incorrect mathematical assumptions (People vs. Collins). Other than that, judges are extremely hesitant to cast doubt on a professed expert's credentials, because the judges feel they lack the expertise to do so. It is a bit curious that judges doubt their ability to assess expert credentials when it involves denying the status of expertise, but have no hesitation in granting the status to anyone with a certificate. I think the judges should err in the other direction, so that only the real deal stand a chance of making it to the stand. That way, even if a true expert is denied "expert" status, he or she is still likely able to defend assertions competently under cross examination. They are used to that sort of thing from years of brutal, demoralizing peer review.
And I'm not merely some raving lunatic trying to stir things up. I stumbled upon the problems of forensic science when I watched a PBS Frontline episode titled, "The Real CSI." It was a real eye-opener. For example, fingerprint analysis: totally subjective. What, you thought you could just scan the fingerprint into a computer and it would search all the databases and magically come up with a match? Ha, good joke, Television!
From Strengthening the Forensic Sciences: A Path Forward, a report put out by the National Academy of Sciences: "Although some Automated Fingerprint Identification Systems (AFIS) permit fully automated identification of fingerprint records related to criminal history (e.g., for screening job applicants), the assessment of latent prints from crime scenes is based largely on human interpretation. Note that the ACE-V method does not specify particular measurements or a standard test protocol, and examiners must make subjective assessments throughout." (pg. 139)
"Individualization," the process of matching a fingerprint to a single person, is the task of a human looking for 6 to 9 points of similarity, where similarity is measured by the person's best judgment. How good is that judgment?
Read this: "In 1995, the Collaborative Testing Service (CTS) administered a proficiency test that, for the first time, was "designed, assembled, and reviewed" by the International Association for Identification (IAI).The results were disappointing. Four suspect cards with prints of all ten fingers were provided together with seven latents. Of 156 people taking the test, only 68 (44%) correctly classified all seven latents. Overall, the tests contained a total of 48 incorrect identifications. David Grieve, the editor of the Journal of Forensic Identification, describes the reaction of the forensic community to the results of the CTS test as ranging from ‘shock to disbelief.'" (From the article "Fingerprint Evidence (PDF)," by Sandy Zabell, published in the Journal of Law and Policy.)
I bet you've heard that no two people have the same fingerprints. Do you know how many scientific experiments have confirmed that? Not two. Not one. Zero. Did you also hear that no two fingerprints, not even from the same person's same finger, mere minutes between printings, are exactly the same? I bet you didn't hear that, either, but it at least has scientific backing. From that, there is good reason to think no two people have the same prints, but it does not provide the same level as confidence, does it?
And that is for Fingerprint Analysis, one of the two pillars of forensic science, behind only DNA analysis! What can we expect for the lower, less scientifically vetted practices, like forensic odontology (forensic dentistry), blood spatter analysis (sorry, Dexter...), ballistics, and that strange scent analysis ordeal we witnessed in the Anthony trial? The rule should be: It can't be used as scientific authority in a court case unless it has been thoroughly vetted by the relevant scientific disciplines.
The walls run red with subjectivity!
The only way to combat this woo, this razzle dazzle, is to raise the public's awareness of it with some good old fashion skepticism. What would Sagan do? He'd hear the expert witness say "We can conclude with certainty, based on forensic science x", and give them the old,
About the Author: Seth KurtenbachSeth Kurtenbach is pursuing his PhD in computer science at the University of Missouri. His current research focuses on the application of formal logic to questions about knowledge and rationality. He has his Master's degree in philosophy from the University of Missouri, and is growing an epic beard in order to maintain his philosophical powers. You can email Seth at Seth.Kurtenbach@gmail.com or follow him on Twitter: @SJKur.
#1 DYLAN CASPARI (Guest) on Wednesday November 21, 2012 at 10:37pm
Nice job on this. So would you ultimately say that our court systems should move away from calling upon “expert testimony” at all? Or that judges should simply stop trying to expedite the trial process?
On a side note, I was hoping that you could expound a little on one of your statements at the beginning: “It’s no secret that the American public is tragically illiterate when it comes to science.”
Could you qualify this statement a little for me? In what way do you feel that the American public is scientifically illiterate? Do you include yourself in this category? Why or why not?
#2 SethKurtenbach on Sunday November 25, 2012 at 10:06am
Hey, thanks Dylan.
I am not against expert testimony or efficient trials. The problem, in my mind, is the vetting of experts and of forensic science disciplines. At the moment, providing expert testimony is a lucrative industry for individuals, and several certificate mills have popped up that require only a few hours of online test taking and a couple hundred dollars in exchange for a certification. Surely this does not constitute expertise. I ultimately agree with the NAS report that there should be a federal institution regulating the use of forensic science expert witnesses in trials which works in conjunction with academia.
Regarding American scientific illiteracy: http://www.sciencedaily.com/releases/2007/02/070218134322.htm and this is a relatively positive finding, in that we are doing slightly better than Japan and Europe, with only a 70% science illiteracy rate. A brief Googling will provide numerous other studies, with similarly bleak results.
I think I am less scientifically literate than I'd like to be, but more scientifically literate than a good chunk of the American population.
Thanks for the questions.