The Impact of Medina's Marginal Balancing on DNA Databases and Sample Retention

The opinion of the Supreme Court of Vermont in State v. Medina is a first. This is not because it strikes down a law requiring all individuals merely charged with felonies to submit to DNA sampling. Other state courts have held that state laws providing for DNA collection before conviction are unconstitutional. However, they reasoned that the prohibition on unreasonable searches and seizures in the Fourth Amendment to the U.S. Constitution compelled such action. The Vermont court relied entirely on a provision of the Vermont Constitution (Article 11).

Article 11 proclaims “That the people have a right to hold themselves, their houses, papers, and possessions, free from search or seizure.”  However, the Vermont Supreme Court has never taken this proclamation literally. Article 11 goes on to recognize that searches or seizures can proceed on the basis of “warrants ...  affording sufficient foundation,” and, as Medina noted, the Vermont Supreme Court has “import[ed]” into Article 11 “the ‘reasonableness’ criterion of the Fourth Amendment.”

But “reasonableness” does not usually entail a fact-specific balancing of individual and government interests. Indeed, this June, the Supreme Court disapproved of warrantless seizures and searches of cellphones notwithstanding the argument that they were just “incident to arrest.” In Riley v. California, the Court reiterated the rule that “[i]n the absence of a warrant, a search is reasonable only if it falls within a specific exception to the warrant requirement.”

So too, the Vermont court declared that under Article 11, “[w]arrantless searches are thus per se unreasonable” although “sometimes permitted.” And it pointed to its opinion in State v. Martin, a 2008 case upholding a post-conviction DNA database, on the theory “that using DNA to determine who committed a past crime is fulfilling an ordinary law enforcement purpose” but using it to link the same person to future crimes is a kind of “special need” for which a warrant is unnecessary as long as the state’s interests outweigh the convicted offender’s.

This is weird. Why should police have to get a warrant to search you when they suspect you of a past crime, but not when they think you might commit one in the future? Nevertheless, the Medina court used this distinction to apply a balancing test. That test, in the hands of Justice John Dooley and two justices who joined his opinion, had some unusual or questionable features.

First, it was a marginal balancing test. That is, the court asked how much extra future crime-fighting power taking DNA before conviction achieved. That is a sensible question for the legislature to weigh before enacting a DNA-on-arrest law (and to study later by having law enforcement agencies gather data on the efficacy of the law). But laws are often compromises, and it is almost always possible to envision less drastic alternatives. For example, a DNA-after-conviction law might include provisions for destroying most samples after the profiles are recorded or for deleting the profiles after a certain period of time. It might be restricted to adult offenders or to a smaller set of offenses. A strict less-drastic-alterative test could put the courts in the business of tweaking legislation. The U.S. Supreme Court has done so for the sake of protecting First Amendment rights but not Fourth Amendment ones.

Second, the Vermont balancing test considered some but not all of the state’s interests. Solving past crimes—through matches to DNA from old rape kits, for example—did not count. If one is going to balance at all, why not put all the relevant interests on the scale? To be sure, the standard interest in solving old crimes is not enough to overcome the individual interests in the security of the person in the absence of a well founded warrant. But that hardly means that it must be ignored when this categorical rule does not apply because the balance of all the state and individual interests may be different. 1/

Third, the majority used the fact that the statute required “expungement of the sample and profile when the adjudication is completed without a conviction of a qualifying crime” as a mark against it. According to these three justices, the legislature’s adoption of this less drastic alternative meant that it determined that it had less need for the samples from unconvicted suspects than from convicted ones. Does this mean that if the legislature re-enacts the law without providing for expungement, the case for upholding the law will be stronger? 2/

Fourth, on the individual side of the balance, Vermont’s justices believed that “each defendant’s privacy interest is greater [than it would be after a conviction] because he or she has not been convicted.” This is a mainstream view, but it seems like a category mistake. Why does your interest in personal security or privacy deserve less weight because you have been convicted of a crime? Certainly, the conviction can subject you to punishment—fines or imprisonment. But this is not because you have a lesser interest in retaining your money and your liberty than anyone else. The significance of the conviction is that the state’s general interests in deterrence, rehabilitation, or retribution carry more weight as applied to known offenders.

Fifth, the court rests its decision on the theory that “[t]he privacy interest of the preconviction defendant is greater than the interest of one who has been convicted because a preconviction defendant has a presumption of innocence.” But the legal presumption of innocence is an evidentiary doctrine that requires the state to prove guilt without relying on the fact that the defendant has been charged with the crime as if it indicates guilt. 3/  It has no bearing on the extent of any privacy interest. As we just noted, a conviction intensifies or focuses the state interests, permitting it to invade some privacy interests that it otherwise could not. Therefore, although the fact of a conviction is an important consideration in a reasonableness inquiry, the “presumption” adds nothing to the balancing test.

Sixth, Medina distinguished fingerprinting of arrestees from DNA sampling on two grounds. The majority thought that unlike DNA, “fingerprints can show only identification, and they have limited functionality in solving old cases.” These claims ignore the fact that fingerprint patterns are not without medical significance and that they routinely solve old cases. 4/ Just last year, the FBI’s criminal fingerprint database received hundreds of thousands of queries for possible matches to latent prints. Nonetheless, it is surely true that DNA samples contain far more socially and personally significant information than fingerprints. The response of most courts, and of course the U.S. Supreme Court in King, has been that statutory protections against accessing that kind of information are sufficient to satisfy a balancing test.

The Medina court disagreed. It stated that “[w]hile current law limits use of the sample, that law can be amended to allow greater use; the retention of the DNA sample suggests that expanded use is possible in the future.” But almost anything is possible in the future. Courts do not normally strike down otherwise constitutional laws because they might be amended in a manner that would make them unconstitutional.

If Vermont follows this unusual approach to constitutional analysis, its entire DNA database law might be unconstitutional under Article 11. Despite its effort to distinguish Martin, the earlier case on convicted-offender DNA collection, Medina observes that “Martin is silent on the storage of the sample, which is retained by the State, apparently indefinitely. ... Neither the statutory purposes, nor the State’s asserted justifications for the law, provide any rationale for retaining the DNA sample once the profile has been created.”

The majority apparently did not think that indefinite sample retention was so grave a risk to privacy as to invalidate the convicted-offender database system, but given the court's understanding of DNA databases, a stringent version of marginal balancing should have this effect. After all, if there is "no rationale" to retain samples indefinitely, and if the risk that the legislature will amend the law to permit the state to mine the samples for sensitive personal information is a weighty concern, there is no state interest to justify sample retention. Conversely, could the legislature revive the part of its law that the court struck down by providing for prompt destruction of those samples? At that point, what basis would the court have to insist that DNA identification is dramatically different from fingerprint identification?

Notes

1. See, e.g., David H. Kaye, Why So Contrived? DNA Databases After Maryland v. King, 104 J. Crim. L. & Criminology (forthcoming August 2014), early draft available at http://ssrn.com/abstract=2376467.

2. Cf. David H. Kaye, Drawing Lines: Unrelated Probable Cause as a Prerequisite to Early DNA Collection, 91 N.C. L. Rev. Addendum 1 (2012), http://ssrn.com/abstract=2064074.

3. David H. Kaye, A Fourth Amendment Theory for Arrestee DNA and Other Biometric Databases, 15 U. Pa. J. Const. L. 1095 (2013), available at http://ssrn.com/abstract=2043259; David H. Kaye, Associational Privacy, the Presumption of Innocence, and “Corruption of Blood” as Constitutional Metaphors in the Debate on “Familial Searching,” Amer. Crim. L. Rev. (Nov. 26, 2012), http://ssrn.com/abstract=2269047

Earlier Posting on State v. Medina

State v. Medina: Can the Vermont Supreme Court Overrule the U.S. Supreme Court on DNA Databases? July 11, 2014, http://for-sci-law-now.blogspot.com/2014/07/state-v-medina-can-vermont-supreme.html

State v. Medina: Can the Vermont Supreme Court Overrule the U.S. Supreme Court on DNA Databases?

Today the Vermont Supreme Court struck down as unconstitutional Vermont’s law requiring individuals charged with any felony to submit to DNA sampling. The court reached this conclusion in a set of consolidated cases led by State v. Medina.

This outcome might seem surprising following the decision of the U.S. Supreme Court in Maryland v. King, 133 S. Ct. 1958 (2013) (discussed in many previous postings). After all, in King, the Court reversed the decision of Maryland’s highest court that a similar Maryland statute infringed an arrestee’s right to be free from unreasonable searches and seizures.

The Vermont court faces no risk of reversal, however, for its opinion relies strictly on the protection against searches and seizures in the Vermont Constitution. Article 11 of this state constitution provides

That the people have a right to hold themselves, their houses, papers, and possessions, free from search or seizure; and therefore warrants, without oath or affirmation first made, affording sufficient foundation for them, and whereby any officer or messenger may be commanded or required to search suspected places, or to seize any person or persons, his, her or their property, not particularly described, are contrary to that right, and ought not to be granted.

The U.S. Supreme Court is only supreme when it comes to federal law. A state normally is free to adopt a statute or to interpret its constitution so as to confer greater rights on its citizens that those recognized for everyone in the federal Constitution.  The Fourth Amendment to the U.S. Constitution declares that

The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

Thus, Maryland v. King merely held that the Maryland Court of Appeal erred in construing the Fourth Amendment to bar DNA sampling soon after arrests for certain felonies (because, given the state and individual interests affected, the DNA sampling was “reasonable”). Even though some of the same words appear in Vermont's Article 11, the Vermont Supreme Court has the power to interpret them differently.

Indeed, the Maryland Court of Appeal could turn around and invalidate the Maryland law notwithstanding the Supreme Court's opinion in King. Article 26 of the Maryland Constitution of 1867 specifies

That all warrants, without oath or affirmation, to search suspected places, or to seize any person or property, are grievous and oppressive; and all general warrants to search suspected places, or to apprehend suspected persons, without naming or describing the place, or the person in special, are illegal, and ought not to be granted.

A new arrestee in Maryland might argue that Maryland's statute requiring the seizure of DNA without an oath or affirmation setting forth probable cause and approved by a magistrate is tantamount to a proscribed “general warrant” or “grievous and oppressive.” Of course, this is essentially the argument that the Supreme Court rejected for the Fourth Amendment, but a state court can embrace such reasoning as applied to its state constitution.

And that is what the Vermont Supreme Court did today, by a vote of 3–2.

Coming up: Analysis of the reasoning in Medina.

McDaniel v. Brown: Whose Error?

The DNA analyst who miscomputed the siblingship probability and accepted the prosecutor's transposition of her random match probability in McDaniel v. Brown (discussed in the previous posting) now leads the Washoe County Sheriff’s Office Forensic Science Division. After the Ninth Circuit's ruling, she explained to a television news reporter that "the prosecutor ... led her through a complicated mathematical journey aimed at reducing Brown's guilt to a percentage." The mathematical journey, as recorded in the trial transcript (left panel), follows. Annotations are in the right panel.

Q [by the prosecutor]: Now, for my benefit, we're looking at a one in 3 million statistic, is there another way to show that statistic? In other words, what -- let's say 100 percent -- what is the likelihood that the DNA found in the panties is the same as the DNA found in the defendant's blood? A: Paternity testing uses percentages. Q: Okay. A: Not the way forensics likes to look at it. We prefer the one in 3 million.

The probability of paternity in parentage testing is a posterior probability computed via Bayes' rule with hypothetical prior odds of 1. It is no more (or less) a percentage than is any other fractional quantity.

Q: I understand that, but for just another way to look at it, what would that percentage be? A: It would be 99.99967 percent. That's what -- MR. SMITH: May I go to the blackboard, Judge? THE COURT: Well, we'll pull it out for you and you want her to write on it? MR. SMITH: No, I'd like to write on it. THE COURT: Well, I don't think you're a witness. I'm not going to let you write on it now. MR. SMITH: All right. THE COURT: If you want her to write on it, she can write on it. By MR. SMITH (continuing): Q: Okay. If -- Ms. Romero, if you'd write it down, please. A: Okay. THE COURT: Is there a -- could you make the decimal a little bigger for us older -- By MR. SMITH (continuing): Q: So, okay. So, if you would do, also, put 100 percent on top of that, if you would, please, with the corresponding number of zeros after the decimal point. And if you would please, then, subtract the lower number from the higher number. A: I don't think this is right. Just a minute. Q: All right. So, if you put a little line under the 99 there and a minus -- subtraction indication. All right. All right. Thank you. If you'd just take -- resume the stand.

Q: So, the -- would it be fair to say, based on that that the chances that the DNA found in the panties -- the semen in the panties -- and the blood sample, the likelihood that it is not Troy Brown would be .000033? A: I'd prefer to refer to it as the one in 3 million. Q: All right. But from a mathematical standpoint, would that be inaccurate? A: Repeat the question, please. Q: Would it be fair, then, to say that with that mathematical calculation there, that the likelihood that the DNA extracted from the semen in the panties and the DNA extracted from the blood that the likelihood that it's not Troy Brown, that it's not a match is .000033?

MR. LOCKIE: Your Honor, I'm going to object on relevance. The witness is testifying that it's not scientifically valid in her opinion. So it's not relevant. THE COURT: Well, I don't know that -- MR. LOCKIE: That's just a subtraction problem. THE COURT: Let's go back. I don't think that's what she said. I don't think that's what she said. Let's go back a step and find out. I don't think that's what she said.

According to the TV news reporter, "Romero found herself agreeing with the district attorney's math, but not in how he was applying it. She says the judge stepped in to clarify the matter, but it was the exchange between her and the prosecutor that was pulled out of the transcript and became the basis for Brown's appeal."

By MR. SMITH (continuing): Q: Now, I understand that — and what I'm trying to do is make this into a percentage where I can understand it. And so I recognize that as far as your testing, you would prefer to have it as a one in 3 million, but just as another way of looking at it, would it be inaccurate to state it that way? A: It's not inaccurate, no. Q: All right. Then in response to my question, would the likelihood that the semen from the DNA found in the panties and the blood from Troy Brown, that it's not the same, would it be -- the chances that they are not a match would be .000033? A: Yes. That's the way the math comes out. Q: All right.

THE COURT: Let's make sure. It's the same thing -- it's the same math just expressed differently. Is that correct? THE WITNESS: Yes. Exactly, your Honor.

Is this how the judge clarified the matter -- by having the witness unequivocally confirm that a DNA type that occurs with a frequency of one in three million in the general population is mathematically the same as a probability of innocence of .00003?

McDaniel v. Brown: Prosecutorial and Expert Misstatements of Probabilities Do Not Justify Postconviction Relief — At Least Not Here and Not Now

Originally posted to Double Helix Law, Jan. 18, 2010 (http://www.personal.psu.edu/dhk3/blogs/DoubleHelixLaw/2010/01/mcdaniel-v-brown-prosecutorial-and-expert-misstatements-of-probabilities-do-not-justify-postconvicti.html) as explained here.

In The Double Helix and the Law of Evidence (pp. 173-176), I briefly described the Ninth Circuit Court of Appeals’ muddled opinion upholding a writ of habeas corpus in Brown v. Farwell, 525 F. 3d 787 (9th Cir. 2008). Much has happened since then. First, the Supreme Court granted a writ of certiorari to review whether the Ninth Circuit used the correct legal standard and whether it should have considered a letter written by a geneticist at the behest of defense counsel eleven years after Brown’s trial. Second, the Court received a slew of briefs, including one on defendant’s behalf from “20 Scholars of Forensic Evidence.” Third, after scheduling oral argument, the Court decided that it could dispose of the case on the briefs alone. Finally, on January 11, 2010, the Court issued its unanimous per curiam opinion (sub nom. McDaniel v. Brown).

The case arose from the brutal rape in 1994 of a nine-year-old girl in Nevada. A jury convicted Troy Brown on evidence that included a DNA profile (at VNTR loci) that had an estimated population frequency of 1 in 3 million. On redirect examination, however, the prosecutor induced its DNA analyst, Renee Romero, to accept his mischaracterization of this number as the probability that someone unrelated to the defendant was the source of the rapist’s profile. In addition, Ms. Romero testified that the probability of a VNTR match to “the very next child” of the same parents would only be 1/6500 when the actual probability is less than 1/1024. (She did not mention other tests she had done that would have brought the probability closer to her figure. See False, But Highly Persuasive: How Wrong Were the Probability Estimates in McDaniel v. Brown?, 108 Mich. L. Rev. First Impressions 1 (2009).) Defense counsel neither objected to nor corrected her testimony even though the legal and scientific literature at the time made it indisputable that the prosecution was misconstruing the 1/3,000,000 figure and that the 1/6500 figure was miscomputed.

After losing various appeals and state postconviction petitions, Brown argued for the first time in federal court that the probabilities were incorrectly computed or interpreted and that without the DNA evidence, no reasonable juror could have found him guilty beyond a reasonable doubt. He also argued that trial counsel’s representation of him was so poor as to amount to ineffective assistance of counsel. The district court agreed with both claims. The Ninth Circuit affirmed on the ground that without the “false and highly misleading” DNA evidence, there was insufficient evidence for the conviction and hence a violation of due process under Jackson v. Virginia, 443 U. S. 307 (1979). It did not reach the question of effective assistance of counsel.

The Supreme Court reasoned that the Jackson claim fails because Jackson merely holds that when the evidence against the defendant — whether or not properly admitted according the rules of evidence or the constitution — is insufficient, then, as a matter of due process of law, the conviction cannot stand. In Brown, however, there was “no suggestion that the evidence adduced at trial was insufficient to convict unless some of it was excluded … thus dispos[ing] of [the] Jackson claim.”

As explained in The Double Helix, the more applicable due process claim is that the misstatements about probabilities rendered the conviction fundamentally unfair. The Court barely discussed this “DNA due process” claim, as Brown denominated it. Instead, it insisted that “[r]espondent has forfeited this claim, which he makes for the very first time in his brief on the merits in this Court.”

Although the Court thus avoided the colorable due process issue posed by the admission of the DNA evidence, it is hard to see how Brown could have prevailed even on that belated claim. Admitting the mischaracterized random-match probability may well have been plain error, but it did not rise to the level of a due process violation. See False, But Highly Persuasive, supra. The error regarding the probability of a random match to a sibling is sufficiently technical as not to amount to plain error, let alone constitutional error. Moreover, the prosecution could have produced a correctly computed sibling-match probability close to 1/6500. See id. Therefore, the trial court’s failure to exclude the statistics — to which defendant did not object — hardly seems like the type of error that rendered his trial fundamentally unfair.

In any event, having determined that Jackson was of no assistance to Brown and that Brown had “forfeited” the better due process claim, the Supreme Court remanded the case to the Ninth Circuit to consider whether Brown’s trial counsel had performed so dismally as to deprive him of due process of law. McDaniel v. Brown is thus a narrow, procedural holding regarding the scope of federal habeas corpus claims of insufficient evidence.

Even as to the procedural issue, however, the per curiam opinion raised the hackles of two Justices. The Scholar’s Brief importuned the Court to seize the opportunity to condemn misinterpretations of DNA evidence at trial. It argued that the DNA analyst’s testimony was badly flawed and assured that the Court that the defense expert was entirely correct. (The author of the brief, Bill Thompson, is a University of California-Irvine colleague of the letter’s author, Larry Mueller; the two have been called part of the “combine from Irvine.”) Apparently, the brief did not persuade the Justices that Mueller’s computation of the sibling-match probability was correct. In the Court’s jaundiced eyes, the letter’s “claim that [the state's expert] used faulty assumptions and underestimated the probability of a DNA match between brothers indicates that two experts do not agree with one another, not that [the state's] estimates were unreliable.” Yet, as the one scientific authority cited in the Court’s opinion — a 1996 Report of the National Academy of Sciences report — indicates, Romero plainly transposed the random-match probability and she miscomputed the sibling-match probability — even on her own assumptions! See False, But Highly Persuasive, supra. On these matters, the Scholar’s Brief was correct. Rather than acknowledge this fact, however, the Court only treated Mueller’s two criticisms as hypothetically true. Accepting them solely for the sake of argument, the Court observed that they did not justify exclusion of the DNA evidence in its entirety.

Given the Court’s interpretation of the Jackson claim, however, this discussion of the probabilities is superfluous. If Jackson only means that when the totality of the evidence — admissible or otherwise — is sufficient for conviction, why talk about whether the admissible evidence alone is sufficient? Was the Court giving some credence to the possibility that a modified Jackson claim would be tenable? That is, could due process require a federal habeas court to excise unfounded exaggerations and then to determine whether the reduced corpus of evidence could permit a reasonable juror to convict?

In a concurring opinion, Justices Thomas and Scalia took the Court to task for considering the implications of the Mueller letter on the admissibility of the DNA evidence. Unequivocally rejecting any possibility of a modified Jackson standard like the one that the Ninth Circuit entertained and then misapplied, these Justices

disagree[d] with the Court’s decision to complicate its analysis with an extensive discussion of the Mueller Report. … [T]he report’s attacks on the State’s DNA testimony were not part of the trial evidence and have no place in the Jackson inquiry. … [E]ven if the report had completely undermined the DNA evidence … the panel still would have erred in considering the report to resolve respondent’s Jackson claim. The reason, as the Court reaffirms, is that Jackson claims must be decided solely on the evidence adduced at trial.

The concurring Justices are correct in describing the per curiam opinion’s analysis of the Mueller letter as dicta. But the reason is not that the letter itself was “not part of the trial evidence.” It is that Brown raised the pseudo-Jackson claim rather than the straightforward due process claim about unfair exaggeration in the presentation of DNA evidence. Even if defense counsel had never presented Mueller’s letter to the trial court, he could have relied solely on sources subject to judicial notice to argue on appeal, in state postconviction proceedings, and then again in the federal habeas court proceedings that the trial judge’s failure to correct the prosecution’s mistakes sua sponte deprived him of due process. But Brown did not make this “DNA due process claim” in state court, and the prosecution’s indisputable errors are not relevant to the claim that he did make.

The Death of the Double Helix Law Blog

I have decided to terminate the Double Helix Law blog. Some of it was duplicated in this blog anyway, and for what they are worth, I'll be copying all the other postings that I consider interesting here. There is no particular schedule for this transfer, but if you notice seemingly old news about DNA evidence and the law appearing here, you'll know why.

False Confessions, True Confessions, and the Q Factor

EXHIBIT A Steven A. Drizin and Richard A. Leo, The Problem of False Confessions in the Post-DNA World, 82 N.C. L. Rev. 891, 930-31 (2004) In this Article, we analyze 125 recent cases of proven interrogation-induced false confessions (i.e., cases in which indisputably innocent individuals confessed to crimes they did not commit) and how these cases were treated by officials in the criminal justice system. ... This is ... the largest cohort of interrogation-induced false confession cases ever identified and studied in the research literature. ... [T]here is no existing universe or database of police interrogation cases from which researchers would be able to draw a random sample. Moreover, even if such a database existed, it would make as much sense for someone studying the causes and consequences of interrogation-induced false confessions to randomly sample police interrogations as it would for someone studying the causes and consequences of lung cancer to randomly sample healthy individuals. The appropriate universe of cases for this study is interrogation-induced false confessions, not all police interrogations.

EXHIBIT B Neil A. Accortt et al., Chronic Disease Mortality in a Cohort of Smokeless Tobacco Users, 156 Am. J. Epidemiology 730 (2002) Our aims are to assess the potential increased mortality risks associated with smokeless tobacco use in a representative sample of the US population and to evaluate the effects of the combined use of smokeless tobacco and cigarette smoking on mortality from chronic diseases. ... We used data from ... a national probability sample of the noninstitutionalized US population ... . After adjustment for age, race, gender, and poverty status, ... [f]emale smokeless tobacco users experienced an increase in mortality from all cancers ... of borderline significance. ... Male exclusive smokeless tobacco users did not experience significant increases in mortality for any type of cancer considered. ... [T]he lung cancer mortality among combined users was nearly twice that of exclusive smokers. ... .

The study of false confessions described in Exhibit A is important and influential. There is much to be said for detailed, historical studies of miscarriages of justice. Analyzing what went wrong in a series of cases can suggest possible reforms to prevent or abate these outcomes in the future.

But by design, the study is not capable of determining the causes of any of the confessions of the 125 DNA-exonerated men. It is comparable to a series of medical case reports rather than to an epidemiological study, such as the one in Exhibit B, which is designed to detect a possible risk factor or cause of cancer. Physicians may report on unusual and interesting cases in which a disease follows an exposure to some factor, such as lung cancer after years of smoking. But without data on the incidence of lung cancer among nonsmokers (or relevant experiments on the effects of smoking), how can one conclude from the study that smoking causes (or even is risk factor for) cancer? A case series offers almost nothing in the way of proof of causation. The persuasive evidence that smoking causes cancer came from large case-control and cohort studies in several populations—not from the earlier case series.

Confronted with defendants' efforts to introduce the testimony of psychological experts on the factors found to be present in DNA exonerations, prosecutors have harped on this point. They have argued that although factors such as physical isolation and extended interrogation sessions certainly can contribute to false confessions, they may also contribute to true confessions, and there is no proof that they are more likely to contribute to a false confession than a true one. In other words, a factor may be relevant to voluntariness—whether the defendant’s will was overborne by unfair pressure—but not be evidence of falsity.

Several courts have agreed that the proffered risk factors are not adequately linked to false as opposed to true confessions.  One federal district court rebuffed defendant’s risk-factor testimony as “nothing more than guesswork: coercive interrogation techniques may lead to false confessions but also produce true confessions, and such techniques were used in this case ... .” United States v. Deuman, 892 F.Supp.2d 881, 888 (W.D. Mich. 2012). The Michigan Supreme Court ruled that the research was methodologically flawed for “failing to compare true and false confessions and identify factors that contribute to false confessions but not true confessions.” People v. Kowalski, 821 N.W.2d 14, 22 (Mich. 2012). Likewise, the Supreme Judicial Court of Massachusetts found it telling that the expert “acknowledged that no studies have been conducted comparing the prevalence of these factors among false confessions to either ‘true’ confessions or all confessions” and “that the studies based on proven false confessions that have been used to identify these relevant factors were based on a sample size of approximately 150 to 200 proven false confessions.” Commonwealth v. Hoose, 5 N.E.3d 843, 861 (Mass. 2014).

These criticisms, I believe, are valid—but only as applied to a possible effort to define a false-confession “profile” or to conclude that when the putative risk factors are present, a confession is false. Defense experts in these cases do not pretend that they can diagnose whether a confession is true or false.  They simply want to present information that they are convinced will help the jury make this judgment. And their beliefs about causes and risk factors for false confessions are based on a richer set of studies than just the case reports.

Consequently, the opinions summarily dismissing all false-confession research as scientifically flawed because of the lack of a comparison group in studies like the one in Exhibit A are too hasty. Furthermore, the testimony could serve a function other than telling the jury which factors are relevant to concluding that a confession is false (for which data on causation or correlation is essential). Suppose that research reveals a factor—I will call it factor Q—that makes virtually every innocent suspect confess. Moreover, although there are no data directly on point, it is highly plausible that Q would have the same effect on guilty suspects. Because the probability of a confession conditional on Q is very large for both innocent and guilty suspects, the relative risk is close to 1. Q is not indicative of whether a given confession is true or false.

Nonetheless, informing the jury of the factor and its impact on innocent defendants could be helpful (if the jury would not otherwise recognize the presence and significance of Q). The point of this testimony would not be to prove that Q has probative value in distinguishing between true and false confessions. It would be show that the confession in the case lacks the probative value that the untutored jury might attribute to it.

Rather than issuing sweeping denunciations of the entire body of psychological research into false confessions as methodologically sterile, courts should focus on whether the expert testimony exceeds the bounds of valid generalizations from the research and how much the particular testimony really can assist the jury. Are there powerful factors like Q that the jury would not recognize? I cannot say that I know the answer, but it is a fair question.

Acknowledgments: Thanks to Debbie Davis and Tom Workman for correspondence on the logic of "risk factors" for false confessions. A longer discussion of the admissibility of expert testimony on false confessions will appear in the 2015 Cumulative Supplement to The New Wigmore: A Treatise on Evidence.

More on the Mistakes in “Forensic Science Isn’t Science”

In "Flawed Journalism on Flawed Forensics in Slate Magazine" I referred to "other inaccuracies in the article" by Mark Joseph Stern entitled “Forensic Isn’t Science.” Here are some excerpts from the article with some thoughts.

Behind the myriad technical defects of modern forensics lie two extremely basic scientific problems. The first is a pretty clear case of cognitive bias: A startling number of forensics analysts are told by prosecutors what they think the result of any given test will be. This isn’t mere prosecutorial mischief; analysts often ask for as much information about the case as possible—including the identity of the suspect—claiming it helps them know what to look for. Even the most upright analyst is liable to be subconsciously swayed when she already has a conclusion in mind. Yet few forensics labs follow the typical blind experiment model to eliminate bias. Instead, they reenact a small-scale version of Inception, in which analysts are unconsciously convinced of their conclusion before their experiment even begins.

Few psychologists would propose that expectancy effects will cause errors in interpretation in every experiment. The role of cognitive bias in science and inference generally is quite complicated. There is no typical “blind experiment model.” The physicists who announced the discovery of the Higgs boson or those who believed they detected the marks of gravitational waves in the cosmic background radiation had no such model. In much of science, experimenters know what they are looking for; fortunately, some results are not ambiguous and not subject to subtle misinterpretation. There is the joke that if your experiment needs statistics, you should do a better experiment.

That said, when interpretations are more malleable—as is often the case in many forensic disciplines—various methods are available to minimize the chance of this source of error. One is “sequential unmasking” to protect forensic analysts from unconscious (or conscious) bias that could lead them to misconstrue their data when exposed to information that they do not need to know. There is rarely, if ever, an excuse for not using methods like these. But their absence does not make a solid latent fingerprint match or a matching pair of clean, single-source electropherograms, for example, into small-scale versions of Inception.

Without a government agency overseeing the field, forensic analysts had no incentive to subject their tests to stricter scrutiny. Groups such as the Innocence Project have continually put pressure on the Department of Justice—which almost certainly should have supervised crime labs from the start—to regulate forensics. But until recently, no agency has been willing to wade into the decentralized mess that hundreds of labs across the country had unintentionally created.

When and where was the start of forensic science? Ancient China? Renaissance Europe? Should the U.S. Department of Justice have been supervising the Los Angeles Police Department when it founded the first crime laboratory in the United States in 1923? The DOJ has had enough trouble with the FBI laboratory, whose blunders led to reports from the DOJ’s Office of the Inspector General and which has turned to the National Research Council for advice on more than one occasion. The 2009 report of a committee of the National Research Council had much better ideas for improving the practice of forensic science in the United States. Its recommendation of a new agency entirely outside of the Department of Justice for setting standards and funding research, however, gained little political traction. The current National Commission on Forensic Science is a distorted, toothless, and temporary version of the idea.

In 2009, a National Academy of Sciences committee embarked on a long-overdue quest to study typical forensics analyses with an appropriate level of scientific scrutiny—and the results were deeply chilling.

The committee did not undertake a “scientific” study. It engaged in a policy-oriented review of the state of forensic science without applying any particularly scientific methods. (This is not a criticism of the committee. NRC committees generally collect and review relevant literature and views rather than undertake scientific research of their own.) This committee's quest did not begin in 2009. That is when it ended. Congress voted to fund the study in 2005.

Aside from DNA analysis, not a single forensic practice held up to rigorous inspection. The committee condemned common methods of fingerprint and hair analysis, questioning their accuracy, consistent application, and general validity. Bite-mark analysis—frequently employed in rape and murder cases, including capital cases—was subject to special scorn; the committee questioned whether bite marks could ever be used to positively identify a perpetrator. Ballistics and handwriting analysis, the committee noted, are also based on tenuous and largely untested science.

The report is far more nuanced (some might say conflicted) than this. Here are some excerpts:

"The chemical foundations for the analysis of controlled substances are sound, and there exists an adequate understanding of the uncertainties and potential errors. SWGDRUG has established a fairly complete set of recommended practices." P. 135.

"Historically, friction ridge analysis has served as a valuable tool, both to identify the guilty and to exclude the innocent. Because of the amount of detail available in friction ridges, it seems plausible that a careful comparison of two impressions can accurately discern whether or not they had a common source. Although there is limited information about the accuracy and reliability of friction ridge analyses, claims that these analyses have zero error rates are not scientifically plausible." P. 142.

"Toolmark and firearms analysis suffers from the same limitations discussed above for impression evidence. Because not enough is known about the variabilities among individual tools and guns, we are not able to specify how many points of similarity are necessary for a given level of confidence in the result. Sufficient studies have not been done to understand the reliability and repeatability of the methods. The committee agrees that class characteristics are helpful in narrowing the pool of tools that may have left a distinctive mark. Individual patterns from manufacture or from wear might, in some cases, be distinctive enough to suggest one particular source, but additional studies should be performed to make the process of individualization more precise and repeatable." P. 154.

"Forensic hair examiners generally recognize that various physical characteristics of hairs can be identified and are sufficiently different among individuals that they can be useful in including, or excluding, certain persons from the pool of possible sources of the hair. The results of analyses from hair comparisons typically are accepted as class associations; that is, a conclusion of a 'match' means only that the hair could have come from any person whose hair exhibited—within some levels of measurement uncertainties—the same microscopic characteristics, but it cannot uniquely identify one person. However, this information might be sufficiently useful to 'narrow the pool' by excluding certain persons as sources of the hair." P. 160.

"The scientific basis for handwriting comparisons needs to be strengthened. Recent studies have increased our understanding of the individuality and consistency of handwriting and computer studies and suggest that there may be a scientific basis for handwriting comparison, at least in the absence of intentional obfuscation or forgery. Although there has been only limited research to quantify the reliability and replicability of the practices used by trained document examiners, the committee agrees that there may be some value in handwriting analysis."

"Analysis of inks and paper, being based on well-understood chemistry, presumably rests on a firmer scientific foundation. However, the committee did not receive input on these fairly specialized methods and cannot offer a definitive view regarding the soundness of these methods or of their execution in practice." Pp. 166-67

"As is the case with fiber evidence, analysis of paints and coatings is based on a solid foundation of chemistry to enable class identification." P. 170

"The scientific foundations exist to support the analysis of explosions, because such analysis is based primarily on well-established chemistry." P. 172

"Despite the inherent weaknesses involved in bite mark comparison, it is reasonable to assume that the process can sometimes reliably exclude suspects. Although the methods of collection of bite mark evidence are relatively noncontroversial, there is considerable dispute about the value and reliability of the collected data for interpretation." P. 176

"Scientific studies support some aspects of bloodstain pattern analysis. One can tell, for example, if the blood spattered quickly or slowly, but some experts extrapolate far beyond what can be supported." P. 178

Hardly a ringing endorsement of all police lab techniques, but neither is the report an outright rejection of all or even most techniques now in use.

The report amounted to a searing condemnation of the current practice of forensics and an ominous warning that death row may be filled with innocents.

According to an NRC press release issued in February, 2009, “[t]he report offers no judgment about past convictions or pending cases, and it offers no view as to whether the courts should reassess cases that already have been tried.” Such language may be a compromise among the disparate committee members. But to derive the conclusion that death row is “filled with innocents” even partly from the actual contents of the report, one would have to consider the deficiencies identified in the system, the extent to which these deficiencies generated the evidence used in capital cases, and the other evidence in those cases. Other research is far more helpful in evaluating the prevalence of false convictions.

Given the flimsy foundation upon which the field of forensics is based, you might wonder why judges still allow it into the courtroom.

As the 2009 NRC Committee explained, there is no single "field of forensics." Rather,

"Wide variability exists across forensic science disciplines with regard to techniques, methodologies, reliability, error rates, reporting, underlying research, general acceptability, and the educational background of its practitioners. Some of the forensic science disciplines are laboratory based (e.g., nuclear and mitochondrial DNA analysis, toxicology, and drug analysis); others are based on expert interpretation of observed patterns (e.g., fingerprints, writing samples, toolmarks, bite marks, and specimens such as fibers, hair, and fire debris). Some methods result in class evidence and some in the identification of a specific individual—with the associated uncertainties. The level of scientific development and evaluation varies substantially among the forensic science disciplines." P. 182.

The courts have been lax in responding to overblown testimony in some fields and to those techniques that lack proof of their fundamental precepts.

In 1993, the Supreme Court announced a new test, dubbed the "Daubert standard," to help federal judges determine what scientific evidence is reliable enough to be introduced at trial. The Daubert standard ... wound up frustrating judges and scientists alike. As one dissenter griped, the new test essentially turned judges into "amateur scientists," forced to sift through competing theories to determine what is truly scientific and what is not.

Blaming the persistence of the admissibility of the most dubious forensic disciplines on Daubert is strange. Daubert's standard did not spring into existence fully formed, like Athena from the brow of Zeus. A similar standard was in place in a number of jurisdictions. As the The New Wigmore: A Treatise on Evidence shows, the Court borrowed from these cases. A smaller point to note is that there were not one, but two partial dissenters (who concurred in the unanimous judgment). Chief Justice Rehnquist and Justice Stephens objected to the majority’s proffering "general observations" about scientific validity, and they did not complain about the ones the Mr. Stern points to as an explanation for the persistence of questionable forensic "science."

Even more puzzlingly, the new standards called for judges to ask "whether [the technique] has attracted widespread acceptance within a relevant scientific community"—which, as a frustrated federal judge pointed out, required judges to play referee between "vigorous and sincere disagreements" about "the very cutting edge of scientific research, where fact meets theory and certainty dissolves into probability."

That’s Chief Judge Alex Kozinski of the Ninth Circuit Court of Appeals writing on remand in Daubert itself. Judge Kozinski could not possibly be objecting to the Supreme Court's opinion on the ground that "widespread acceptance within a relevant scientific community" is an impenetrable standard. Quite the opposite. He applied that very standard in his previous opinion in the case and was bemoaning what he called the "brave new world" that the Court ushered in as it vacated his opinion. A recent survey of judges found that 96% of the (disappointingly small) fraction responding deemed the general scientific acceptance to be helpful -- more than any other commonly used factor in judging the validity of scientific evidence.

American jurors today expect a constant parade of forensic evidence during trials. They also refuse to believe that this evidence might ever be faulty. Lawyers call this the CSI effect, after the popular procedural that portrays forensics as the ultimate truth in crime investigation. [¶] “Once a jury hears something scientific, there’s a kind of mythical infallibility to it,” Peter Neufeld, a co-founder of the Innocence Project, told me. “That’s the association when a person in white lab coat takes the witness stand. By that point—once the jury’s heard it—it’s too late to convince them that maybe the science isn’t so infallible.”

Refusal to question scientific evidence is not what most lawyers call the CSI effect. In any event, jury research does not support the idea that jurors inevitably reject attacks on scientific testimony or that the testimony of the first witness in a figurative white coat is unshakeable.

If judges can’t be trusted to keep spurious forensic analysis out of the courtroom, and juries can’t be trusted to disregard it, then how are we going to keep the next Earl Washington off death row? One option would be to permit anybody convicted on the basis of biological evidence to subject that evidence to DNA analysis—which is, after all, the one form of forensics that scientists agree actually works. But in 2009, the Supreme Court ruled that convicts had no such constitutional right, even where they can show a reasonable probability that DNA analysis would prove their innocence. (The ruling was 5–4, with the usual suspects lining up against convicts’ rights.)

This option, which applies applies to a limited set of cases (and hence is no general solution) is not foreclosed by District Attorney for the Third Judicial District v. Osborne, 129 S.Ct. 2308 (2009). If there is a minimally plausible claim of actual innocence after conviction, let’s allow such testing by statute. Of course, it would be better to thoroughly test potential DNA evidence (when it is relevant) before trial—something that Osborne's trial counsel declined to request, fearing that it would only strengthen the prosecution’s case.

Until lab technicians follow some uniform guidelines and abandon the dubious techniques glamorized on shows like CSI, forensic science will barely qualify as a science at all. As a recent investigation by Chemical & Engineering News revealed, little progress has been made in the five years since the National Academy of Sciences condemned modern forensic techniques.

Again, as the NRC committee stressed, "forensic science" is not a single, uniform discipline. Since the report, funding has increased, some guidelines have been revised, and significant research has appeared in some fields. Still, the pace resembles that of global warming. It is coming, notwithstanding resistance described in earlier years on this blog.

As for the "investigation by Chemical & Engineering News," the latest I saw from that publication was an article in a May 12, 2014 issue with a map showing selected instances of examiner misconduct dating back to 1993 and indicating that only five states require laboratory accreditation. No effort was made to ascertain how many labs are still operating without accreditation. With no apparent literature review, The article simply asserted that

[I]n the years since [2009], little has been done to shore up the discipline’s scientific base or to make sure that its methods don’t result in wrongful convictions. Quality standards for forensic laboratories remain inconsistent. And funding to implement improvements is scarce. [¶] While politicians and government workers debate changes that could help, fraudsters like forensic chemist Annie Dookhan keep operating in the system. No reform could stop a criminal intent on doing wrong, but a better system might have shown warning signs sooner. And it likely would have prevented some of the larger, systemic problems at the Massachusetts forensics lab where Dookhan worked.

I must be missing the real investigation that the C&E News writers conducted.