Statistics: Why Figures Don’t Lie, But Liars Figure…
More and more, figures and statistical information finds it way into litigation, both criminal and civil. At some point in your career as an attorney you will need to understand what can and cannot be accomplished in utilizing statistics. Most laypersons and attorneys are ill-equipped to handle such information. Oftentimes experts can find refuge in statistics which may or may not be truly relevant to the legal issue you are confronting. As Mark Twain (a/k/a Samuel Clemons) famously noted:
“Figures don’t lie, but liars figure.”
Another often quoted quip is:
“There are three kinds of lies: lies, damn lies and statistics.”
In litigation, you will often hear someone argue that the odds of being injured in a particular fashion are so low that a jury should not compensate them. However, there is a real risk in engaging in such post hoc analysis. How would you feel for example, if the State of Indiana came into court refusing to pay the Lotto Jackpot on your winning ticket by arguing that you could not have won it because the odds of winning are one in seven million.
Another way to point this same principle out, is the fallacy of using statistics to explain away a plaintiff’s untimely and unexpected demise:
“Your honor and ladies and gentlemen of the jury. Research has established that 90% of individuals involved in similar accidents survive. Accordingly, we must conclude that in spite of the evidence of lack of respiration, heartbeat, and brain wave activity, and in spite of the unfortunate burial of the decedent, in my expert opinion I conclude that he did not really die, and therefore the plaintiff estate cannot recover.”
Even though this sort of logic is flawed to its core, such arguments regularly find their way into our justice system… sometimes with disastrous effect. In order to spot such problems, you need to read about statistics, understand their limitations and how they can be misused. In this regard I would recommend the following reading:
1. Trial by Mathematics: Precision and Ritual in the Legal Process by Laurence Tribe, Harvard Law Review, 1971. This is an informative law review article addressing this topic. Mr. Tribe was the law clerk who assisted a California justice in writing a seminal opinion in this area. The court reversed a criminal conviction where a prosecutor improperly used statistical arguments in a robbery case involving a multi-racial couple. People v. Collins, 438 P.2d 33, 36-37 (Cal. 1968). Tribe had a math degree from Harvard in addition to his J.D. This opinion is often cited by courts as a prime example of how statistics and “scientific” evidence can be misused and down right dangerous to the pursuit of justice.
2. Naked Statistics by Charles Wheelan. The author strips away the arcane and technical details and focuses on the underlying thinking that drives statistical analysis. The author also clarifies key concepts such as inference, correlation, and regression analysis, reveals how biased or careless parties can manipulate or misrepresent data. Memorable examples of problems with statistics are discussed as well.
3. Calculated Risks: How to Know When Numbers Deceive You by Gerd Gigerenzer. This book does exactly what the title infers, it shows you in a concrete fashion how faulty thinking leads to people drawing incorrect conclusions from statistics and data. One of the problems discussed in the book is the famous dilemma presented by the Monty Hall Let’s Make a Deal Problem.
4. Math on Trial: How Numbers Get Used and Abused in the Courtroom by Leila Schneps. This book reviews the facts and outcomes of ten trials spanning from the nineteenth century to the present day, in which mathematical arguments were used, abused and disastrously misused resulting in unjust outcomes.
5. Reference Manual on Scientific Evidence prepared by the Federal Judicial Center. This is a free handbook that covers a number of areas of science that regularly appear in federal courtrooms. This manual is utilized by the federal judiciary as a reference book and covers both the law and science underlying a number of disciplines including epidemiology which is statistically based. This is must reading for any trial attorney who is going to take on an expert in a courtroom. This manual is regularly updated as well.
6. A Systematic Approach to Clinical Determinations of Causation in Symptomatic Spinal Disk Injury Following Motor Vehicle Crash Trauma by Michael D. Freeman, PhD, MPH, DC, Christopher J. Centeno, MD, and Sean S. Kohles, PhD. is an article which critically examines the misuse of data and pseudo-science to undermine claims of personal injury in motor vehicle accidents by defense “experts” and studies conducted in this area of litigation. This article provides an excellent survey and critic of the literature dealing with medical causation in motor vehicle collisions.
This list of reading should be both interesting and informative to the trial attorney confronted with the use of statistics. Just as it is helpful to “think like a lawyer”, it is equally useful to “think like a statistician”. Remember, numbers don’t lie, but liars figure…
Posted on April 28, 2014, in Evidence, Trial Advocacy and tagged analogies, Calculated Risks, Expert Witness, Laurence Tribe, Mark Twain, Math on Trial, Mathematics on Trial, Michael Freeman, Monty Hall problem, Naked Statistics, People v. Collins, Reference manual on Scientific Evidence, Samuel Clemons, statistics. Bookmark the permalink. Leave a comment.