Who Is Nelson Acosta-Roque? (Part III)

As we have seen, the fingerprint analyst who testified about a match between the two ten-print cards from the man named Acosta-Roque and the two from the man named Pecheca-Aromboles said next to nothing about her examination in the particular case. Anthony Capece, Assistant Chief Counsel for DHS, only asked only the following, very general questions, about the methodology:

Q. Okay now, how do you — if you can answer this, how do you actually identify someone by means of fingerprints?
A. I have two impressions that I'm comparing, I use — I personally use two fingerprint magnifiers and two pointers is what we call them, and I'll have a glass over each impression, and I use the pointer to what I call walk through the ridges and count my points.
Q. Okay now, is there a set number of points that needs to compare in order for you to be able to say that this is the same person?
A. There is no established number. I normally would not want to sign off on anything with less than eight, but there is no set established policy for minimum.
Q. Okay, is there an amount that's required by criminal courts?
A. No, there's not.
Q. Okay, so eight would be acceptable in criminal courts then?
A. Yes. Yes, if you sign off on it, you're saying that that's the same person.

It is impossible to discern from the answers to these poorly framed questions how complete the comparisons among the four cards were. They could have been exceedingly thorough or they could have been disturbingly cursory. The fingerprint analyst, Susan Blei, has yet to respond to emails seeking clarification

The amicus brief (2012) of "scientists and scholars" offers the following guidance:

Although the record is not altogether clear on this point, it is possible that the examiner based this conclusion on eight corresponding ridge details from a single finger. Amici are aware of no research that would support concluding from eight corresponding details that “this is the same person.” Indeed, recent research suggests that while some comparisons showing eight corresponding characteristics may well have significant probative value, the probability of observing such number of characteristics in agreement between fingerprints from unrelated individuals is not negligible.²⁸ The scientific data collected to date clearly support the general value of fingerprint characteristics for constituting very strong evidence in favor of a same-origin hypothesis, but are in significant contradiction with the claims made by the expert during her testimony.

I wonder if, like the examiner’s testimony, this description of "scientific data" is subject to a charge of overclaiming. Just what do the data show about “the probability of observing such number of characteristics in agreement between fingerprints from unrelated individuals” and the probative value of most eight-point matches? Indeed, how would one measure the probability of a false eight-point match and the probative value of an eight-point match?

An experiment could investigate how often fingerprint examiners detect eight (or more) matching points among impressions of the same fingers (mates) as opposed to different fingers (non-mates). This experiment could provide an estimate of the probability of an eight-point match for different fingers and show that it "is not negligible” (if that is the case). Moreover, if eight-point matches turned out to be not much more common among pairs of impressions from the same finger than among pairs from different fingers, an eight-point match would not give strong support to the hypothesis that the marks come from mates. Thus, the ratio of the probability of a match with a mate to the probability with a non-mate—a quantity known as the “likelihood ratio” (LR)—could tell us whether the probative value of such matches is high or low.

The amicus brief cites no data on eight-point matches of human examiners. The footnote accompanying the claim that the probability of eight-point marches from different fingers is “not negligible” cites three studies. The researchers computed quantities akin to a likelihood ratio according to an algorithm and model of distortion in impressions for different numbers of points (even if a human would not call them matches). This is great research, but it does not investigate the properties of an eight-point matching rule as employed by fingerprint examiners—at least not directly.

Still, it might be suggestive, which is all that the brief claims. So what do the papers reveal about the probabilities for eight-point non-mates versus eight-point mates? Are only “some” likelihood ratios large? Are probabilities of non-mate matches “not negligible”? The first paper (Neumann et al. 2006) is confined to three minutiae. Hence, it is largely beside the point.

The second paper (Neumann et al. 2007) reports that computed LRs with 8 minutiae rarely exceeded 1 for prints from different sources (non-mates). The percentages of LRs in this range in different types of patterns are 1.2%, 0.5% 0.3%, 0.1% (Table 3). In contrast, the LRs exceeded 1 for prints from the same fingers (mates) 99.6% of the time (Figure 11). Of course, an LR only slightly greater than 1 has little probative value, but the LRs for mates exceed 10,000 nearly 90% of the time, judging from Figure 11.

In the third paper (Neumann et al. 2012), the separation in the distributions of the LRs for eight minutiae as between mates and non-mates was even sharper. The LRs almost always exceeded 1,000 for mates, almost never reached this level for non-mates, and never reached 10,000 for non-mates. The text accompanying Figure 3 refers to only a “few occasions when LR(H_d) [the LR for non-mates] exceed[ed] 1" and suggests that even this small number is inflated “when it is borne in mind that the prints have been retrieved as the closest among 600 million.” Later, the paper states that “very few instances of LR(H_d) > 1 were observed” and “the probability of misleading evidence ... is very low.” Moreover, the “wider range of features [used] by examiners would make their examination more discriminating.”

Thus, the research seems more supportive of the value of an eight-point match than the brief's suggestion that significant probative value is merely a sometimes thing for such a match. “Significant probative value” seems to characterize not just “some comparisons” but the vast majority of them. Likewise, the research indicates that different fingers match at eight points at a low rate. There could be some false eight-point matches, but this hardly means that an eight-point matching rule would produce a major proportion of false positives. (And even a small proportion would be of concern.)

So is it true that “[t]he scientific data collected to date are in significant contradiction with the claims made by the expert during her testimony”?  Well, the answer is both yes and no. The research actually seems to support the examiner’s good feelings about eight matching minutiae when making a positive identification. At the same time, the studies do limit how positive a knowledgeable examiner can be about an eight-point identification. Two of the studies suggest that the examiner’s degree of confidence in the claim that no one else in the world has a finger that could produce eight or more similar minutiae was unfounded.

Of course, for the reference in the testimony to eight points of comparison to have much bearing on the specific identification in issue, the principal examiner would have had to have made her identifications among four ten-print cards using only one of the ten fingers and stopping upon finding eight corresponding points in the impressions of that one finger. In addition, the second verifying examiner also would have had to have used only eight features—and the very same ones at that. The amicus brief treats this as a possibility worth considering because the record does not “affirmatively” show otherwise. At the same time, the brief recognizes the distinct possibility “that the examiner relied upon a much greater number of corresponding details than the figure of eight mentioned in the record” (but never mentioned as having been used in the case itself).

The brief's response to this obvious possibility is curious. The scientists and scholars state that “[i]f this is the case, however, there is no information available regarding how the forensic expert witness adjusted her assessment of the probative value of the evidence in light of this greater amount of discriminating information.”

But why would an expert who believes that there is zero probability of a false match for 8 points have to “adjust her assessment” when there are, say, 32 points? Suppose I assert that (1) the probability that all the air molecules around my mouth and nose will happen to move away and stay away for one second is essentially zero, and (2) the chance of a temporary vacuum lasting two seconds also is virtually zero. The second statement is neither false nor misleading because of the first statement. Similarly, it makes little sense to demand that a fingerprint examiner who would give an opinion that the chance of false match is negligible for eight features of the sort seen here has to “adjust[] her assessment” when there are more such features. Naturally, her confidence would increase with more data pointing to the same conclusion. Once again, the real problem with the testimony is not the examiner’s personal rule of thumb. The problem lies with the protestation of 100% confidence in the first place.

What can an examiner who is sensitive to the growing base of scientific research into her craft do to explain the striking similarities in a pair of fingerprints? Here is a hypothetical dialogue:

Q. What did you find when you compared the prints?
A. I observed that the overall patterns and the minutiae were very similar. The prints had the same general appearance and the same detailed features.
Q. And what does that signify?
A. Well, all these similarities strongly support the conclusion that the marks come from the same finger rather than from two different fingers.
Q. How strongly do they support the fact they are from the same finger?
A. I cannot give you a simple number, but patterns like these are much more likely to arise from prints of one and the same finger than from impressions of different fingers.
Q. How certain are you that they are from the same finger?
A. I did not say they came from the same finger. All I can tell you is that they have the definite appearance of coming from the same finger. They do not look like a pair of marks coming from different fingers.
Q. How certain are you of this?
A. I am very, very certain

References

Brief of Scientists and Scholars of Fingerprint Identification as Amici Curiae in Support of Petitioner and in Favor of Reversal, Acosta-Roque v. Holder, No. 11-70705 (9th Cir., Mar. 8, 2012).

Cédric Neumann et al., Computation of Likelihood Ratios in Fingerprint Identification for Configurations of Three Minutiae, 51 J. Forensic Sci. 1 (2006).

Cédric Neumann et al., Computation of Likelihood Ratios in Fingerprint Identification for Configurations of Any Number of Minutiæ. 52 J. Forensic Sciences 54 (2007)

Cédric Neumann et al., Quantifying the Weight of Evidence from a Forensic Fingerprint Comparison: A New Paradigm, 175 J. Royal Stat. Soc’y: Series A 371 (2012).

Who Is Nelson Acosta-Roque? (Part II)

Back in March, Who Is Nelson Acosta-Roque? (Part I) described the pending appeal of an Alaska waiter being deported because the government believes he is really Victor Antonio Pecheca-Aromboles — a previously deported Pennsylvania cocaine dealer from the Dominican Republic who changed his name and returned to the U.S. The Department of Justice resolved the question of the true identity on the basis of “trust-me testimony.” Below are some key questions and answers in the hearing in 2010, along with some thoughts about "100% certain" testimony. Susan R. Blei, supervisor of Alaska’s Criminal Records and Identification Bureau, is testifying over the telephone:

Q. [W]hat is your occupation?
A. I'm a fingerprint analyst and I supervise the criminal records and identification bureau. * * *
Q. Okay now, how do you — if you can answer this, how do you actually identify someone by means of fingerprints?
A. I have two impressions that I'm comparing, I use — I personally use two fingerprint magnifiers and two pointers is what we call them, and I'll have a glass over each impression, and I use the pointer to what I call walk through the ridges and count my points. * * *

Comment: That’s it? Just count the number of Level 2 details in common, with no concern for which ones are common and which ones are rare? How many common points were there in this case? How many fingers were compared? The lawyer for the Department of Homeland Security and the immigration judge never thought to ask. And neither did Mr. Acosta-Roque, who was representing himself. You would think that Ms. Blei knows, but she has not responded to my emails.

Q. What is the likelihood of two people possessing the same fingerprints?
A. That won't happen. No two people have the same fingerprints.
Q. Okay, is that — and how — I mean, how do you know that? Is there a scientific study?
A. There have been scientific studies, yes.
Q. So even identical twins wouldn't have the same fingerprints?
A. No, they would not.
* * *

Comment: These answers attracted the scorn of the 39 scholars and scientists, whose amicus brief stated that “The record does not report what studies the expert was referring to, and amici have found no study that proves that ‘No two people have the same fingerprints.’ Amici are skeptical that it would be possible to even conduct a study that proved that proposition." [1]

Impossible to prove? Well, it cannot be proved directly, by a census of all fingerprints that exist, that have existed, and that will exist [2], but studies of identical twins show that their fingerprints are distinguishable, and statistical analyses purport to show very small probabilities of duplicating a fingerprint pattern. For example, the NIST report, released in February, observes that “[o]ne study reports that in 100,000 randomly chosen fingerprints of exemplar quality, there is only a 10^–14 probability that some pair of them will match in regard to both minutiae and ridge shape.” [3] Of course, there are more than 100,000 fingerprints in the world’s population, but the same generative model estimates the probability of at least one match among 10 billion to be less than 0.0001. [4]

To be sure, the fact that some scientific studies purport to establish the uniqueness of complete fingerprints does not conclusively prove the point. But it does make the statement that “[t]here have been scientific studies, yes,” understandable as a response to the lawyer’s question.

The problem with the expert’s answer, then, is not that it refers to a fact outside the domain of potential scientific knowledge (which is always tentative in the sense of being subject to revision) [5].  The problem is that it fails to inform the judge that the expert's belief can be and has been disputed. The assertion that "there have been scientific studies, yes" may be the literal truth, but it is one-sided, and an expert forensic scientist or technician has some responsibility to tell the whole truth. The NIST report explains:

During a trial, counsel’s questioning frames the expert’s presentation. The expert may not simply decide what information to discuss but must answer counsel’s questions. This can create some tension between the goal of being complete and the need to be responsive. In resolving this tension, “ethical considerations and professional standards properly place a number of constraints on the expert’s behavior.” One such constraint is “a requirement of candor. While an expert is ordinarily under no legal obligation to volunteer information, professional ethics may compel this ... when the expert believes that withholding information will change dramatically the picture that his … analyses, properly understood, convey.” Thus, the “guiding principles” proposed by the American Society of Crime Laboratory Directors/Laboratory Accreditation Board (ASCLD/LAB) advise forensic scientists to “attempt to qualify their responses while testifying when asked a question with the requirement that a simple ‘yes’ or ‘no’ answer be given, if answering ‘yes’ or ‘no’ would be misleading to the judge or the jury.” [3, p. 115 (notes omitted)]

In Acosta-Roque, it would not have been hard for the expert to answer, “Yes, there have been scientific studies that bear on the question of whether every person has distinguishable fingerprints. Although the conclusion is accepted in most forensic science textbooks and fingerprint identification manuals, some commentators have questioned it.”

The amicus brief also chastises the expert for not listing specific studies, stating that “forensic expert witnesses have an ethical obligation to specify to courts the studies they rely upon in forming their conclusions and should refrain from the dubious practice of invoking ‘studies’ without specifying what those ‘studies’ are.” However, experts should not be expected to cite studies on every general proposition that is part of their reasoning. Certainly, an expert should not try to bolster testimony with uncalled for references to unnamed “studies”; however, it is fair to answer a lawyer’s unanticipated question, “Is there a scientific study?” in the affirmative even without being able to name the study that the expert knows exists.

Here, the expert could have replied, “Yes, there have been scientific studies that bear on the question of whether every person has distinguishable fingerprints. Because I did not know you would be asking about these, I do not have a list of them at hand.”

Q. Were you able to obtain the Pennsylvania fingerprints?
A. Yes, I was.
* * *
Q. And did you compare all four sets of prints?
A. Yes, I did.
Q. And what was your conclusion upon comparing the prints?
A. They were made by one and the same individual.

Comment: This is source-attribution testimony for ten-print cards. It is based on the theory that an examiner can distinguish every possible pair of prints from the same individual’s ten fingers from every possible pair of ten-print cards from different individuals. The recent reports of the National Academy of Sciences and the NIST Expert Working Group emphasized in the amicus brief do not focus on this claim. Rather, they concern latent print identification, in which far less information is available. In that context, the NIST report concludes that “[g]iven the current state of scientific and professional knowledge, . . . it is best to avoid [source-attribution] testimony based on the theory of global general uniqueness.” [3, p. 138] The same point could be made for ten-print identifications. There are viable alternatives to source attributions on the part of the testifying expert and to reliance on the difficult-to-prove “theory of global general uniqueness.”

Q. Okay, now how did you go about comparing these sets of prints?
A. I began with the Pennsylvania prints and I compared them one at a time to each of the other cards.
Q. And why did you use the Pennsylvania prints?
A. They were the ones that had been requested. You could do it in any order. That was just the way I did it.
Q. Okay, and you determined that all four matched up to the same person?
A. Yes, they did.
Q. And did you have — was there any sort of back-up check done on this?
A. I did have another employee do comparisons on the very same cards, and that employee came to the same conclusion.
* * *

Comment: The record is skimpy about the verification. Was it blind? Appellate counsel for Acosta-Roque asserted that “Neither the initial analysis nor the verification were blind,” [6], but how does he know what the first examiner told the second? The shallow hearing reveals nothing other than that “another employee” looked at the “same cards and came to the same conclusion.”

Q. And so what are the chances that this could — that the fingerprints from Mr. Acosta-Roque maybe don't match Mr. Victor Arumboles?
A. They do match.
Q. Okay, so I mean, can you state — I mean, what are they — is there a percentage? Can you quantify it, or ...
A. No. No, I'm 100 percent sure the prints are the same and made by the same individual.
* * *

Comments: Despite the inarticulate questions, the witness professes 100% subjective certainty in her source attribution. The amicus brief denigrates all claims of certainty as irresponsible because human error is possible:

Claims of absolute certainty go beyond mere expressions of the degree of an expert’s confidence, and are, in essence, an unsupported form of bolstering the expert’s own testimony. In addition to failing to recognize the possibility of an erroneous match, they also fail to recognize that there might be other errors that could occur, such as mislabeling of a record within a database, or a misattribution of a source print to the incorrect individual. No human endeavor is always utterly error-free, and assertions of 100 percent certainty in this context are both misleading and scientifically unwarranted.

I know what the brief is driving at, but its rationale for eschewing "assertions of 100 certainty" is unconvincing. The analyst attributed the four prints she examined to the same source — "the prints [were] made by the same individual." This conclusion does not turn on how the exemplar prints were labelled.

As for the charge of "bolstering," a claim of total certainty is no less an “expression of the degree of an expert’s confidence” than is a claim of near certainty. Any assertion of a high degree of confidence could be overstated because of an inadequate perception of the risk of human error. Any assertion of great confidence “gives a boost to” a more tentatively stated  conclusion that all the fingerprint impressions on the four cards come from the same set of ten fingers. There is nothing wrong with this type of "bolstering" if (1) the asserted degree of confidence is warranted, and (2) the expert acknowledges that, like all statements about the world, it is subject to some degree of doubt. An expert who testifies that “I am as sure about this as I am about my own name or the fact that the earth orbits the sun” also could be mistaken. Degree-of-confidence testimony is misleading only if it is misunderstood as a statement that the objective probability that the same fingers are represented on all the cards is the specified percentage.

That said, it would be best for experts to avoid 100% even as an expression of subjective probability. From a Bayesian perspective, this level of certainty corresponds to accepting infinite odds on a bet that the match is false. Would Ms. Blie be willing to accept eternal damnation and unbearable torture if she were proved wrong? Total certitude means that no contrary evidence, no matter how convincing could change her mind. Thus, the use of 100% smacks of dogma or advocacy rather than a neutral statement of the evidence. Most likely, this extreme probability is what the amicus brief means by “bolstering.”

The more fundamental and general issue, however, is whether an assertion of a number bordering on 100% for the subjective certainty of identity is “scientifically unwarranted.” Science certainly permits a witness to report probabilities conditioned on explicit assumptions. Do scientific studies permit an examiner to assert that, having extensively analyzed sets of ten-print cards, she is (nearly) 100% confident that the only plausible hypothesis to explain the comparable patterns is a common source? If so, was her examination sufficient to warrant such a claim? Without more information on the actual examination than the government and the expert deigned to provide, the last question cannot be answered.

Q. Okay, and you concluded that they're one and the same to the exclusion of all others?
A. Yes, sir.
Q. Okay and I just — one last time, how certain are you about your conclusion that these prints belong to the same individual?
A. I'm 100 percent certain.

Comment: Objection — asked and answered.

This is not the end of the testimony. I'll pick up the thread and wind it up soon with a discussion of the amici's criticism of the expert's desire to have at least eight matching points before making a source attribution.

References

1. Brief of Scientists and Scholars of Fingerprint Identification as Amici Curiae in Support of Petitioner and in Favor of Reversal, Acosta-Roque v. Holder, No. 11-70705 (9th Cir. Mar. 8, 2012).

2. Michael Saks & Jonathan J. Koehler, The Individualization Fallacy in Forensic Science, 61 Vanderbilt L. Rev. 199 (2008).

3. NIST Expert Working Group on Human Factors in Latent Print Analysis, Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach: The Report of the Expert Working Group on Human Factors in Latent Print Analysis 82 (David H. Kaye ed., 2012). The report adds that “[w]ithout the assumption of independence, however, the computed probability could be orders of magnitude higher.” Id.

4. Su, C. and S. Srihari. “Generative Models for Fingerprint Individuality Using Ridge Models.” in Proceedings 19th International Conference on Pattern Recognition, Tampa, Florida, Dec. 8–11, 2008 (tbl. 3).

5. David H. Kaye, Probability, Individualization, and Uniqueness in Forensic Science Evidence: Listening to the Academies, 75 Brooklyn L. Rev. 1163 (2010).

6. Caleb Mason, Scientific Validity of Fingerprint Comparison: New Case and Amicus Brief (Mar. 5, 2012), http://prawfsblawg.blogs.com/prawfsblawg/2012/03/scientific-validity-of-fingerprint-comparison-new-case-and-amicus-brief.html.

Voice Stress Analysis and the Investigation of Trayvon Martin's Death

Today’s New York Times has a lengthy article [1] on the flaws in the Sanford, Florida, police investigation of the shooting of black teenager, Trayvon Martin, by the 28-year-old neighborhood watch volunteer, George Zimmerman. When the 16-day investigation did not produce any charges, Florida’s governor responded to a national outcry by appointing an aggressive prosecutor. Now indicted, Zimmerman continues to maintain that he shot Martin in self-defense.

Tucked away at the end of the article is the single sentence: “The police conducted a lie-detection procedure, known as voice stress analysis, on Mr. Zimmerman that [sic] he passed.” A voice-stress test? By “a small city police department that does not even have a homicide unit and typically deals with three or four murder cases a year”? [1]

Yes, law enforcement agencies across the country have invested millions of dollars in voice stress analysis (VSA) software programs—despite a widely known lack of evidence to show that they work. For example, after conducting a field study of jail inmates using urinalysis to check their statements about whether they had used specific drugs, a University of Oklahoma researcher wrote that “two of the most popular VSA programs in use by police departments across the country are no better than flipping a coin when it comes to detecting deception regarding recent drug use [2]." 1/

The response of the National Institute for Truth Verification—the company that bills itself as “the world leader in voice stress analysis”—is instructive. The company’s website insists that

the vast majority of VSA studies funded by pro-polygraph elements of the US Government were significantly flawed. One of the many flaws of these studies . . . was that they lacked real-life consequence and thus lacked jeopardy. . . . [C]onsequence and jeopardy found in “high stakes lies” are required to accurately and consistently detect deception. [¶] . . . VSA research conducted by the University of Florida, and a second study conducted by researchers from the University of Oklahoma, both utilized “low stakes lies” in an attempt to measure the results of various VSA instruments. [3]

Another webpage on validity lists many studies, but the descriptions indicate that they merely demonstrate that VSA can detect stress and anxiety. [4] The question, as with the polygraph, is whether an examiner can ascertain the cause of the stress. Thus, the defense of VSA seems be this: we have no scientifically respectable body of proof showing that VSA is highly sensitive and specific in detecting deception, but, then again, nobody has proven to our satisfaction that it does not work for this purpose.

Despite the inability to validate VSA, some police love it. A detective in the Sex Crimes Section of the Metropolitan Nashville Police Department explained that:

We purchased 10 CVSA's and trained 20 examiners and in my opinion, the instruments and training have been one of the greatest assets our department has ever acquired. Not only has it helped us solve many crimes from major thefts to homicides, but has also helped expose false reports from victims, thus saving our department many man-hours of investigation. [5]

Considering that the fundamental question about VSA is whether it can distinguish between stress caused by intentional deception and stress caused other factors—think about the rape victim who has to describe the events to the police—this kind of screening is a little frightening. It helped the Sanford police and prosecutor, though. Or did it?

Notes

1. The comparison to a coin may be misleading. Consider Table 12 in the Oklahoma study. Kelly R. Damphousseat et al., Assessing the Validity of Voice Stress Analysis Tools in a Jail Setting, Mar. 31, 2007, at 53 (NCJRS doc. no. 219031, available at https://www.ncjrs.gov/pdffiles1/nij/grants/219031.pdf). It indicates that 87 subjects tested positive for cocaine use within the past 72 hours. Of these, 40 deceptively stated that they had not used cocaine in this period. However, VSA programs only correctly indicated deception for eight of the 40. This is a sensitivity of 20%. Given an individual lying about recent cocaine use, the programs had only one chance in five of recognizing the deception. However, Their specificity was much better. Of the 47 respondents who were not deceptive, the programs correctly classified 42 (89%) as truthful. A coin flip, on the other hand, would have a specificity and sensitivity of 50%. See, e.g., David H. Kaye, The Validity of Tests: Caveant Omnes, 27 Jurimetrics J. 349 (1987).

References

1. Serge F. Kovaleski, Trayvon Martin Case Shadowed by Series of Police Missteps, N.Y. Times, May 16, 2012

2. Kelly R. Damphousse, Voice Stress Analysis: Only 15 Percent of Lies About Drug Use Detected in Field Test, NIJ Journal No. 259, Mar. 17, 2008

3. National Institute for Truth Verification, Research Casts Doubt on US Government-Funded VSA Studies

4. National Institute for Truth Verification, Studies Validating Voice Stress Analysis

5. Kent McAllister, quoted in National Institute for Truth Verification, Law Enforcement Reviews

ASCLD Meets Frontline

On April 17, PBS aired a Frontline documentary on “The Real CSI.” Within the week, the president of the American Society of Crime Lab Directors (ASCLD), Jill Spriggs, gave the following assessment to her fellow crime lab directors:

In the last week, published reports dusted off the same old forensic cases from the past and regurgitated negative information from years ago in order to provide doubt in the minds of the public and the court. I don’t know about you but some of my neighbors watched the Frontline documentary and had many questions. “Were latent prints really not a validated science?” “Is latent print analysis not what I thought it is?” “Were bite marks really a product of bad forensic science?” “How could anyone rely on bite marks?” “And, how many employees use certificate mills to obtain a forensic science certification?” My answer—You can be confident about latent print evidence. Latent prints are a validated science. Of all of the millions of fingerprint samples in the databases throughout the world, no two people have ever matched the same fingerprint. Bite marks are not an accredited crime laboratory discipline and “no” we don’t use forensic science certification mills to certify our analysts.

Watching the documentary, I shared her sense of boring familiarity. What’s new here? Same faces, same criticisms. But Ms. Spriggs’ rejoinder likewise is outdated in the logic it uses to shrug off the criticisms. Frontline summarized real problems with the way the American criminal justice system produces and consumes forensic science evidence. Although ASCLD is not responsible for these problems, let’s look more carefully at the three matters that Ms. Spriggs mentions.

1. Credible Credentials

In case after case, witnesses bolster their credentials as forensic experts with credentials from the American College of Forensic Examiners Institute—the organization depicted as a diploma mill on the Frontline show (and elsewhere). Courts often take the bait. For example, in People v. McWhorter, 212 P.3d 692 (Cal. 2009), the California Supreme Court juxtaposed a defense expert’s seemingly strong credentials—he “was certified by an organization known as the American Board of Recorded Evidence; and was a member of the American College of Forensic Examiners” with his puzzling inability to “identify the computer program he used to enhance or ‘electronically emboss’ the image in question [and to] satisfactorily explain the full nature of the process he used to create it.” Id. at 726.

It is appropriate to note that “we [all government crime labs?] don’t use forensic science certification mills to certify our analysts.” But what about the rest of the forensic science community—and the courts? Some prosecutors seem to have no compunction about presenting their witnesses as qualified at least in part because they are “a life fellow for The American College of Forensic Examiners” or some such thing. Chavarria v. State, 307 S.W.3d 386, 387 (Tex. Ct. App. 2009).

Judges and lawyers need to learn which organizations have meaningful standards and which do not. If there were less demand for dubious credentials, expert witnesses—government employees and private consultants alike—would have less incentive to pad their CVs with such credentials.

2. “Bite marks are not an accredited crime laboratory discipline”

Disavowing bite-mark evidence because it comes from outside the public laboratory is not responsive to the question, “Were bite marks really a product of bad forensic science?” Does ASCLD believe that forensic science is limited to the reports of crime laboratories? That forensic odontologists are not accredited? The American Board of Forensic Odontology "was organized in 1976 under the auspices of the National Institute of Justice" and "is accredited by the Forensic Specialties Accreditation Board (FSAB) as a forensic specialty offering board certification to qualified forensic dentists." If the public cannot trust this field's accredited expertise, why should it trust "an accredited crime laboratory discipline"? (There are a number of possible answers, but Ms. Spriggs does not provide them.)

3. “You can be confident about latent print evidence. Latent prints are a validated science.”

Latent prints often contain valuable information for ascertaining the origin of the print. However, the Frontline interviewees pointed to the lack of objective criteria for deciding when prints do or do not come from the same source and the danger that unconscious bias could influence these judgments. These problems notwithstanding, there are scientifically sound studies suggesting that latent fingerprint examiners can get the correct answers most of the time. The Frontline show could be faulted for failing to acknowledge this research while publicizing the NIJ’s refusal to fund a rigorous, comprehensive audit of actual laboratory work. Such a study should be done to complement the recently acquired experimental data, but why would Frontline omit these findings from its televised and online materials?

Yet, Ms. Spriggs does the same thing. Rather than refer to the scientific research that is responsive to the calls for better validation, she repeats the same old story: We can have great confidence in the work of latent print analysts because “[o]f all of the millions of fingerprint samples in the databases throughout the world, no two people have ever matched the same fingerprint.”

What is wrong with this proof of validity? First, whether any individuals have the same fingerprint tells us virtually nothing about the ability of analysts to compare prints accurately. In the Frontline show, Jennifer Mnookin made this point when she explained that the task confronting a latent print analyst is to discern when a partial print comes from a particular person’s finger. That task is distinct from deciding whether pairs of full prints come from different fingers.

Second, just because there are millions of prints on file, it hardly follows that “the millions of fingerprint samples in the databases throughout the world” match. The FBI alone has over 71 million prints in its database. To verify that no pair of these exemplar prints match would require approximately 2.5 x 10¹⁵ (2.5 quadrillion) comparisons of pairs of 10 prints. It would take an examiner, working at the incredible pace of one comparison per digit per second, some 800 million years to complete this task. When the FBI’s contractor, Lockheed Martin, tried to prove the non-existence of matching prints in the database using an automated matcher, it limited itself to a mere 51,000 prints and failed to establish uniqueness.

Ms. Spriggs importuned her fellow crime lab directors to “speak out on these issues. . . . Don’t wait another day! Get started!” Indeed, there are important things to be said.

Earprints Leave Their Mark in Germany

Der Spiegel recently reported that "earprints . . . are as useful to the police as finger prints. A burglar in Germany made the mistake of pressing his ear to front doors to check if anyone was home. The unique prints have allowed the police to pin 96 burglaries on him" [1].

The article makes no mention of the flap over earprints in England [2]. In 1998, a small-time burglar named Mark Dallagher made legal history when he became the first man to be convicted of murder by earprint evidence [3]. A jury found him guilty of murdering 94-year-old Dorothy Woods after Cornelis van der Lugt, a Dutch earprint expert, said he was "absolutely convinced" that Dallagher had made the prints found on the window of Miss Wood's house [3]. Eventually, however, DNA was recovered from the earprints, and it did not match Dallagher's. Following "an appeal, a retrial and a fresh police investigation the Crown said it had 'anxieties' about the case and was offering no evidence against Dallagher'" [3]. Thus, he was acquitted of the murder in 2004 [3].

Despite the bad press the Dallagher case generated, "earprints" are not without value in criminal investigations. Obviously, earmarks vary across individuals. Of course, so do repeated impressions of the same ear, but studies indicate that there is discriminating value in the marks [4-7]. Interestingly, in light of Dallagher, another study suggests that DNA recovered from earmarks can produce false exclusions. When DNA in "60 earprints collected from three healthy adult volunteers under controlled laboratory conditions" was analyzed, "high levels of non-donor alleles [were] observed" [8]. (This is from the abstract of the study. I have not had time to read the body of the paper or check for follow-up work.)

It seems clear, however, that reports like the one in Der Spiegel unconditionally praising the power of "unique prints" cannot be reconciled with the research to date. Fortunately, the German police in the case against the 33-year-old Macedonian linked to some 100 robberies have more to go on than the man's ears. A Hamburg police spokeswoman also referred to fingerprints and DNA [9].

References

1. German Police Identify Burglar by His Earprints: Ninety-Six Break-Ins Solved, Spiegel Online Int'l, April 30, 2012, http://www.spiegel.de/international/zeitgeist/0,1518,830659,00.html
2. David Bamber, Prisoners to Appeal as Unique 'Earprint' Evidence Is Discredited, The Telegraph, Dec 2, 2001
3. Sean O'Neill, Expert Evidence Flaws Clear 'Earprint Killer', Telegraph, Jan. 23, 2004
4. Cornelis Van Der Lugt, Andrew Thean, Lynn Meijerman, & George J. R. Maat, Earprints, in Forensic Human Identification: An Introduction 73-84 (Sue Black & Tim Thompson eds. 2006)
5. Lynn Meijerman, Andrew Thean & George Maat, 1 Forensic Science, Medicine, and Pathology 247-256 (2005)
6. Ivo Alberink & Arnout Ruifrok, Performance of the FearID Earprint Identification System, 166 Forensic Sci. Int'l 145–154 (2007)
7. Ivo Alberink & Arnout Ruifrok, Repeatability and Reproducibility of Earprint Acquisition, 53 J. Forensic Sci. 325-330 (2008)
8. E.A. Graham, V.L. Bowyer, V.J. Martin & G.N. Rutty, Investigation into the Usefulness of DNA Profiling of Earprints, 47 Sci. & Justice 155-159 (2007)
9. Andy Eckardt, Earprints Allow German Cops to Nab Alleged Serial Burglar, World News on MSNBC.com, Apr. 30, 2012

Acknowledgments

Thanks to Ira Ellman and ASU student Seth Reeker for the link to the Der Spiegel article. Cross-posted to the Double Helix Law blog.

Why Can't Maryland Continue to Check Arrestee DNA Profiles?

In the wake of Maryland’s peculiar decision this week holding its arrestee DNA database law unconstitutional “as applied” (King v. State 2012), the Balitmore Sun reported that “a bank of nearly 16,000 DNA samples collected since January 2009 will remain in limbo. The samples were run through a database of cold-case evidence for matches” (Wenger, 2012). Presumably, this means that 16,000 DNA samples in the state’s possession (1) come from arrestees who have not been convicted (at least, not yet); (2) these samples already have yielded DNA profiles (digital records of DNA features used in making identifications); (3) the profiles are in a computer database that allows them to be searched against the profiles recovered from crime scenes or victims (in hot and cold cases, new ones and old ones, alike); but (4) the state has chosen to stop using the existing arrestee database. Thus, the Sun quotes a state police spokesman as saying that "The DNA database will remain inactive and the searching of any information in it ceased today" (Ibid.).

But why? Let’s assume that the King case was correctly decided — something that is not apparent from the reasoning given in the opinion and the cases cited there (Kaye, in press). It seems clear that trawling the crime-scene database for a match to an arrestee is not a search under the Fourth Amendment. Trawling and retrawling does not invade bodily integrity; neither does it reveal facts about the individual’s genome that are not already known. These are the only two interests that the Maryland judges in the majority provided as the basis for their opinion. The only interest that the database trawl implicates is the interest in not being tied to a crime-scene. Under rulings of the U.S. Supreme Court, a suspect’s bare desire not to be linked to a crime does not create a “reasonable expectation of privacy” (e.g., United States v. Jacobsen 1984). Government action that invades no reasonable expectation — in this case, retrawling — does not constitute a “search or seizure.” It is outside the scope of the Fourth Amendment.

But even if retrawling cannot constitute a search — a conclusion that has clear support in federal case law (Boroian v. Mueller 2010; Kaye 2011) — what about the illegality (according to King) in acquiring the DNA before the suspect has been convicted?  Does not the use of the illegally acquired and profiled DNA taint any hit made in the later trawl?  Is not the hit “the fruit of the poisonous tree,” as the courts call it, that must be excluded as evidence in a criminal case?

The answer could be “yes but no.” Yes, the trawl is the fruit and normally would have to be excluded from the state’s case in chief. Here, however, the illegal search (and its fruits) need not be excluded as evidence under the Supreme Court’s holding in Illinois v. Krull, 480 U.S. 340 (1987). Krull holds that the exclusionary rule does not apply when police conduct a warrantless, suspicionless, administrative search pursuant to a state law that later is found to be unconstitutional. That is precisely the situation here. The only “search” was the acquisition of the DNA for databasing, and it was conducted in good faith on the basis of a statute that seemed to be constitutional.

But surely this is too clever a legal argument. The good-faith exception supports (1) the acquisition of the DNA sample; (2) the laboratory analysis; and (3) all trawls conducted before the opinion in King. (So does the Supreme Court’s approach to the retroactivity of constitutional decisions.) This means that the lower courts on remand should uphold King’s original conviction, but retrawling after the court has ruled that the samples and profiles should not have been acquired goes beyond the situation in Krull. In that case, the police did no more than turn over the evidence they collected in good faith to the prosecution.

Thus, Maryland’s decision to keep the arrestee DNA database under wraps (at least until the Supreme Court rules on the practice or the Maryland court reconsiders the issue at some later time) is probably correct. But King should have no effect on “12 pending cases across Maryland that depend on such DNA evidence” or on the “65 arrests and . . . 34 convictions” (Wenger 2012) already obtained in good faith. Even if the five judges in the majority in King are right about the constitutional status of the law, the exclusionary rule does not apply to this previously acquired evidence.

References

• United States v. Jacobsen, 466 U.S. 109 (1984)
• Boroian v. Mueller, 616 F.3d 60 (1st Cir. 2010)
• King v. State, No. 68, 2012 WL 1392636 (Md. Apr. 24, 2012)
• David H. Kaye, A Fourth Amendment Theory for Arrestee DNA and Other Biometric Databases, 15 U. Pa. J. Const. L (in press)
• David H. Kaye, DNA Database Trawls and the Definition of a Search in Boroian v. Miller, 97 Va. L. Rev. in Brief 41 (2011)
• Yvonne Wenger, Md. Authorities Stop Post-arrest DNA Collection: Sampling Halted Pending Potential Supreme Court Appeal, Baltimore Sun, Apr. 27, 2012

Maryland's Highest Court's Opinion on Arrestee DNA Is an Outlier

Maryland's highest court is its Court of Appeals. Two days ago, in King v. State [1], this court became the first supreme court of any state to hold that taking a DNA sample from an arrestee is unconstitutional. But you would not know this from the court's opinion.

Instead, the Maryland court purports to follow "the Minnesota Supreme Court in C.T.L." Considering that the Minnesota Supreme Court did not decide C.T.L. and that it has yet to consider the routine practice of taking DNA prior to arrest [2], this is no small feat.

The majority opinion in King, penned by Judge Glenn T. Harrell, Jr., contains additional gaffes. It refers to Judge Marjorie Rendell of the U.S. Court of Appeals for the Third Circuit as a man, and it asserts that "Fourth Amendment analyses" are "more stringent" than "a First Amendment 'rational basis' review" -- whatever that may mean.

Of course, these infelicities do not mean that the opinion is wrong -- although it is long on description and short on analysis. The balancing of state and individual interests that is pivotal to the opinion is less than lucid. We can get a sense of the court's approach from its discussion of the individual interests that the opinion seeks to protect. To begin with, the Maryland court gives more weight than other courts do to the interest in being free from an unwanted but painless and relatively minor intrusion into one's body. Judge Harrell writes that DNA

is collected by swabbing the interior of a cheek (or blood draw or otherwise obtained biological material). While the physical intrusion of a buccal swab is deemed minimal, it remains distinct from a fingerprint. We must consider that “[t]he importance of informed, detached and deliberate determinations of the issue whether or not to invade another's body in search of evidence of guilt is indisputable and great.” Schmerber, 384 U.S. at 770.

The puzzle here is that, if the physical intrusion is indeed minimal, why is it of “great importance” to have a “detached . . . determination” in the form of a judicial warrant? It cannot be the peculiar notion that laboratory analysis to produce an identifying profile is a separate search requiring a warrant. If that were so, the laboratory analysis of “abandoned” DNA of a suspect also would require a warrant. This might be a reasonable position—but the King court does not retreat from State v. Williamson, 993 A.2d 626 (Md. 2010). There, the police recovered and then analyzed DNA from a drinking cup given to a suspect at a police station, and the Maryland Court Appeals flatly rejected the separate-search argument. Furthermore, if the “physical invasion” aspect of the DNA collection were of such great importance, the state could avoid the impact of King by changing the method for collecting the DNA. Instead of a buccal swab, the arrestee could be asked to place his fingers on a sticky pad to which some cells would adhere.

The interest that actually seems to be driving the opinion is not that the arrestee is compelled open his mouth so that some cells can be scraped from the inside of his cheek. It is, in the King court's words, the possibility that “the vast genetic treasure map that remains in the DNA sample retained by the State” will be read or released in violation of state law. But the opinion utterly fails to address whether the state’s possession of that unread map (the physical sample kept under lock and key) unreasonably interferes with a defined privacy interest. And even if it does, could not Maryland acquire the identification profiles -- data that are nothing like "[a] person's entire genetic makeup and history" -- and then destroy the physical sample to satisfy the court's oddly applied balancing test?

Despite its problems, both superficial and fundamental, King is not devoid of all redeeming value. For example, the court correctly distinguishes between the use of a biometric identifier for identification and its use of it for intelligence purposes. But the opinion sheds no new light on the constitutional issue and casts some grotesque shadows. Readers seeking a deeper analysis will have to look elsewhere [3].

References

1. King v. State, No. 68, 2012 WL 1392636 (Md. Apr. 24, 2012)
2. In re Welfare of C.T.L., 722 N.W.2d 484 (Minn. Ct. App. 2006)
3. David H. Kaye, A Fourth Amendment Theory for Arrestee DNA and Other Biometric Databases, University of Pennsylvania Journal of Constitutional Law, Vol. 15 (in press)

Postscript

Alan Lazerow pointed out that the court's opinion does not have the superficial flaws that caused my jaw to drop and pen to move. Thankfully, the opinion as now posted on Maryland's website and Westlaw have these blemishes removed.

Cross-posted to the Double Helix Law Blog.