Sunday, 8 February 2015

"Remarkably Accurate": The Miami-Dade Police Study of Latent Fingerprint Identification (Pt. 1)

A week ago (Feb. 2, 2015), the Justice Department issued a press release entitled "Fingerprint Examiners Found to Have Very Low Error Rates." According to the Department:
A large-scale study of the accuracy and reliability of decisions made by latent fingerprint examiners found that examiners make extremely few errors. Even when examiners did not get an independent second opinion about the decisions, they were remarkably accurate. But when decisions were verified by an independent reviewer, examiners had a 0% false positive, or incorrect identification, rate and a 3% false negative, or missed identification, rate. ... “The results from the Miami-Dade team address the accuracy, reliability, and validity in the forensic science disciplines, ...” said Gerald LaPorte, Director of NIJ’s Office of Investigative and Forensic Sciences.
Inasmuch as the researchers -- latent print examiners and a police commander in the Miami Dade Police Department 1/ -- only studied the performance of 109 latent print examiners, it is not clear how many forensic science disciplines it actually addresses. Nor is it obvious what "validity" means (beyond "accuracy") in this one activity.

But let's put press releases to the side and look into the study itself. The authors assert that
The foundation of latent fingerprint identification is that friction ridge skin is unique and persistent. Through the examination of all of the qualitative and quantitative features available in friction ridge skin, impressions can be positively identified or excluded to the individual that produced it. 2/
This study does next to nothing to validate this foundation. The premise of uniqueness is very difficult to validate, and this study is limited to "80 latent prints with varying quantity and quality of information from [a grand total of] ten known sources." 3/ But, to its credit, the research does tell us about the ability of one large group of examiners to correctly and reliably pair these particular latent prints to the more complete known prints of the fingers that generated them. Let's see how much it reveals in this regard.

The Test Set

As for the prints used in the experiment, "[a] panel of three International of Association (IAI) certified latent print examiners independently examined and compared the 320 latent prints to the known standards and scored each latent print and subsequent comparison to their known standard according to a rating scale that was designed and used for this research; 80 were selected as the final latent prints to be used for testing purposes." 4/ The purpose of the three independent examinations was to rate the latent-known pairs on a difficulty scale "in order to present the participants with a broad range of latent print examinations that were representative of actual casework." 5/ Although the researchers may well have succeeded in fashioning a test set with pairs of varying difficulty, the report does not explain how they knew that this set was "representative of actual casework" and that "[t]he test sets utilized in this study were similar to the work that participants perform on a daily basis." 6/ Neither did they report how consistently the three uber-experts gauged the difficulty of the pairs.

The Examiners Who Were Tested

It seems that readers of the Miami-Dade report must take on faith the assertion that the test set is "representative of actual casework." In contrast, it is plain that the test subjects are not representative of all caseworkers. Rather than seek a random sample of all practicing latent print examiners -- which would be a difficult undertaking -- the researchers chose a convenience sample. Only "[l]atent print examiners in the United States who were an active member [sic] of the IAI received an email invitation from the MDPD FSB inviting them to participate in this study." 7/ Inasmuch as IAI certification is a mark of distinction, the sampling frame diverges from the population of all examiners. Departing from good statistical practice, the report does not state how large the nonresponse rate for IAI-certified invitees was. If it was high (as seems probable), the sample of examiners is likely to be a biased sample of all IAI-certified examiners.

In addition to soliciting participation from IAI-certified examiners, "[a]pplications were also made available to any qualified latent print examiner, regardless of affiliation with a professional organization." 8/ How this was done is not explained, but in the end, 55% of the subjects were not IAI-certified. 9/

Of course, these features of the sampling method do not deprive the study of all value. The experiment shows what a set of motivated examiners (volunteers) with high representation from IAI-certified examiners achieved when they (1) knew that their performance would be used in a report on the capabilities of their profession, (2) had an unspecified period of time to work, and (3) may not have always worked alone on the test materials. In the next posting on the study, I will describe these results.

Notes

  1. The only description of the authors in the report is on the title page, which identifies them as Igor Pacheco, CLPE (MDPD), Brian Cerchiai, CTPE (MDPD), and Stephanie Stoiloff, MS (MDPD)." The International Association for Identification lists the first two authors as certified latent print examiners as of Dec. 4, 2014. Mr. Cerchiai is also a, IAI certified tenprint examiner. The third author is a senior police bureau commander in the Forensic Services Bureau of the Miami-Dade Police Department (MDPD). In July 2012, she testified before the Senate Judiciary Committee on behalf of the International Association of Chiefs of Police that "[f]orensic science is not the floundering profession that some may portray it to be."
  2. Igor Pacheco, Brian Cerchiai & Stephanie Stoiloff, Miami-Dade Research Study for the Reliability of the ACE-V Process: Accuracy & Precision in Latent Fingerprint Examinations, Final Technical Report, Award No. 2010-DN-BX-K268, Dec. 2014 (abstract).
  3. Id. The latent prints were not just from fingers. Some were palm prints.
  4. Id. at 24.
  5. Id. at 27.
  6. Id. at 35.
  7. Id. at 34.
  8. Id. at 35.
  9. Id. at 51.
Related Postings
  • Reports on studies in mainstream journals can be found on this blog under the labels "fingerprint" and "error."

Posted by at No comments:
Labels: ,

Saturday, 31 January 2015

Justice Department Reverses Decision on the Mandate of the National Commission on Forensic Science

The Justice Department reversed its position on the National Commission on Forensic Science's authority to recommend expansive criminal pretrial discovery of the opinions and information held by forensic scientists and criminalists who might testify for either the prosecution or the defense. The full remarks of Deputy Attorney General Sally Q. Yates and Judge Jed Rakoff, whose indignant resignation two days earlier, may have prompted her to reconsider the Department's position, are set out in an Appendix. So are statements from the Commissioners who spoke about the resignation.

According to Washington Post investigative reporter Spencer Hsu, an anonymous source in the Justice Department who "was not authorized to discuss the issue" "said the initial decision that pretrial evidence discovery rules were beyond the commission’s scope was made by Yates's predecessor, James Cole, before his departure Jan. 8." The short article does not explain why the Department did not convey Deputy AG Cole's decision to the judge or (it seems) anyone else on the Commission's Subcommittee on Reporting and Testimony until the eve of the Commission's fifth meeting.

The new decision led Judge Rakoff to rejoin the Commission and to continue as co-chair of the reporting and testimony subcommittee. With the discovery issue within its recognized purview, the subcommittee is free to submit a proposal for the full Commission to consider. In the end, the strongest proposal the Commission can make is a recommendation to the Attorney General.

References
Appendix

The following are excerpts from the transcript of the second day of the January 29-30 Commission meeting in Washington, D.C. I have edited them to correct misspellings, remove redundancies, etc. The full transcript is (or at least was) at http://www.captionedtext.com/client/ViewTranscript.aspx?EventId=2541437&ParticipantId=14469494-359b-4882-ad09-d1f77ab69e5e.

[Acting Deputy Attorney General Sally Q. Yates:] First, let me thank all of you for being so flexible today to adjust your schedule so that Judge Jed Rakoff and I could join you together this afternoon. I would like to welcome Judge Jed Rakoff back to the Commission. Late yesterday afternoon, Judge Jed Rakoff and I got to speak at great length, and I was able to hear his concerns about the scope of the responsibilities of the Commission. I will tell you that I believe that reasonable people can differ on whether or not discovery is within the scope of this Commission's charter. I was able to hear Judge Jed Rakoff 's views that he believes it is very much in the scope. I also heard from the Attorney General that the [view] that it is not in the scope was also reasonable. What I heard yesterday afternoon was a game changer. I learned that since the inception of this subcommittee that the subcommittee has been operating with the understanding that discovery was within the scope of the subcommittee's charter. And the subcommittee has been working almost for a year now on discovery and has been working very hard and has been very thoughtful in its approach to these issues. And so it seemed to me that given that it is at least arguable, I know you feel it is more than arguable, but if it is at least arguable that this is in the scope of the charter, and given that the subcommittee has openly been doing this for almost a year with everyone knowing about it and working on the discovery, it seemed fair to me that under the circumstances that this Commission should have an opportunity to hear the subcommittee's views on that issue and that this Commission could make its determination as to what information should be provided to the Attorney General.

And so that is the way that I would like for us to proceed. In connection with that, I have asked Andrew Goldsmith, who some of you may know who is the Department's national discovery coordinator, to be available to the subcommittee if the subcommittee so chooses, to be able to get his perspective on what the impact of what if some of these occasions [recommendations?] might be on the department's practices. Andrew is here with me today and available to talk with any of you -- not just today but when the subcommittee meets going forward.

I don't want us to be in this position again. And so I have directed my staff to get together with the various subcommittees here to make sure we all have a complete understanding of what is on the agenda for each of the subcommittees. And if the Department of Justice or any of you as Commissioners have a problem or believe that any of that is outside the scope of the charter of this Commission, then you should go ahead and say something now rather than waiting until after the subcommittee has done much of its work. That is something that we are going to be doing going forward, and I hope that we can work with you in that regard.

On the discovery issue, this is obviously a critically important issue to the Department. We take very seriously our obligation to ensure that defendants receive a fair trial, and they can only get that fair trial when they receive the appropriate discovery. And so we look forward to hearing whatever information the Commission wants to give us on your perspective on that issue. And so without further droning on, I will turn it over to Judge Jed Rakoff.

[United States District Judge Jed Rakoff:] I am glad to be back. I know you have been working very hard all of these last two days. I tried my best to get out of it, but I didn't fully succeed. [Laughter] It was close. [Laughter]

I wanted to thank Deputy Yates for having the open-mindedness to reconsider and reverse the decision previously made. Like the deputies from the old West, I have learned that she is a straight shooter, and it has been a real pleasure having the chance to interact with her over the last two days. The bad side for you folks is you are stuck with me again. Bear with me as best you can. What I look forward to is having this Commission now give the fullest consideration to discovery views, discovery recommendations if there are such. These are matters of great importance, but they need to be discussed on the merits. Now thanks to Deputy Yates, we will have the ability to discuss the merits. I am also glad that we now have Mr. Goldsmith on board as well to help us with that.

My subcommittee will be at least available telephonically and have the reviews available for the next meeting. This is one area. I feel very strongly about this area. I don't want to minimize that this wonderful Commission has so many important areas that it is working on, and so for me, it is a great pleasure for me to rejoin this Commission. I thank Deputy Yates for making this possible. [Laughter]

[Commissioner (John Fudenberg?):] Dr. Mae and Deputy Yates, thank you for your time and support. I'd like to begin saying something briefly that doesn't have to do with our subcommittee. It's important to be said. I wanted Judge Rakoff to be here. [Note: By this point, Judge Rakoff had left: "With some embarrassment, when I didn't think I would be attending this meeting, I scheduled some things in New York ... I'm going to have to get a plane back, so I apologize.] I don't know him well enough to insult him, so I'm trying not to. I am personally embarrassed by what has happened the last couple of days. I think if we have disagreement, whether or not we are working on something for a year or something for a week, I think we should be mature about it, and talk to each other about it, and have open communication rather than packing up quitting, and I don't think that's appropriate. I think we as Commissioners should take the time and take a deep breath and talk issues through rather than having, I understand, newspaper articles today, which I think is embarrassing to this Commission, and I'm sure embarrassing to the Department. It was disappointing, and I think we should rise above those petty issues, and we should as a Commission commit to each other that we are going to try to work through issues before we start quitting. I don't know that he went to the media about it, but somebody did. I think as a member of the Commission, I want to apologize because I'm embarrassed for that.
[Commissioner ?:] Given that John talked about the judge, I'd like to go on record that I supported Judge Rakoff's action in resigning at the committee. I was the chair of the legal group that worked for months and months and went through 10 different drafts that we shared with everybody. I think it would have been (?) resignations next week if this had [not been] resolved as well as (?)
[Commissioner ?:] I also have tremendous respect for what Judge Rakoff did. I'm confident none of us know all the communications that were delivered from the Commission to both your predecessor and to you, and I'm sure we don't all the communications that went back and forth, but I'm confident there were sources of misunderstanding, and I'm really impressed both sides came back together to address that, and I trust both sides did that in good faith.

It's not uncalled for (?) stand up for (?) what the judge did. Whether I would've done it or not, it was an act of principle. But having said that, I am truly committed. I so appreciate the commitment to go forward to respect our charter and our independence. I just want to say thanks.

[Commissioner (Hon. Barbara Hervey?):] I want to echo some of their comments. I respect your right to have an opinion. I don't appreciate an apology on behalf of the Commission because I am sure that everyone has their thoughts and reasons and beliefs. On behalf of the judge, he is not here to defend himself, I think he had some very principled ideas. I am just grateful that all of you would sit down and maturely discuss all of these decisions and come to some conclusions that were helpful to all of us. I appreciate that.
Posted by at No comments:
Labels:

Thursday, 29 January 2015

"A Bump in the Road" for the National Commission on Forensic Science

Yesterday, U.S. District Court judge Jed Rakoff resigned from the National Commission on Forensic Science (NCFS). His resignation letter, placed on the web by the Washington Post, begins as follows:

Last evening, January 27, 2015, I was telephonically informed that the Deputy Attorney General of the U.S. Department of Justice has decided that the subject of pre-trial forensic discovery — i.e., the extent to which information regarding forensic science experts and their data, opinions, methodologies, etc., should be disclosed before they testify in court — is beyond the “scope” of the Commission’s business and therefore cannot properly be the subject of Commission reports or discussions in any respect. Because I believe that this unilateral decision ... reflects a determination by the Department of Justice to place strategic advantage over a search for the truth — I have decided to resign from the Commission, effective immediately.

According to Judge Rakoff,

The notion that improved discovery, going beyond what is minimally required by the federal rules of criminal procedure (which were drafted without any consideration of the difficulties unique to forensic science), is somehow outside the scope of the Commission’s work thus runs counter to both the mandate of the Commission’s Charter and the Commission’s overall purpose.

At today's Commission meeting, which is still in progress, several commissioners indicated frustration over this eleventh-hour ruling. Paul Giannelli, who drafted the "views" document for the subcommittee, noted that it went through some seven iterations with no indication from the Justice Department that it was beyond the "jurisdiction" of the NCFS and that the initial written instruction from a Department of Justice liaison stated that the subcommittee should consider “legal issues inherent in reporting and testimony, such as discovery.” (Emphasis added.) Peter Neufeld asked to see the internal memorandum to the Deputy Attorney General and noted the absence of any written statement from the Deputy AG.

My guess is that the topic of access to information from forensic science laboratories will not be deemed off-limits to the Commission but that no recommendations regarding possible amendments to the rules of discovery will be permitted. The Deputy Attorney General is expected to appear before the Commission tomorrow afternoon. The Department of Justice's co-chairman of the Commission referred to the matter as a bump in the road on which the Commission is traveling.

Postscript (added 1 Feb. 2015): My guess was wrong.See Justice Department Reverses Decision on the Mandate of the National Commission on Forensic Science, Jan. 31, 2015.

Reference: Spencer Hsu, U.S. Judge Quits Commission to Protest Justice Department Forensic Science Policy, Wash. Post, Jan. 29, 2015.
    Posted by at No comments:
    Labels:

    Wednesday, 14 January 2015

    A Probability for Dog DNA

    A pet peeve of many statisticians is the transposition fallacy—misconstruing the probability of the data given the hypothesis as the probability of the hypothesis given the data. In general,

    P(data | hyp) ≠ P(hyp | data).

    The term “pet” is particularly apt in the case of yesterday’s PBS “Here and Now” interview on animal DNA analysis in forensic science. The program’s host interviewed WBUR’s Vicki Croke, who has written an engaging and informative account of “Pet CSI: How Dog and Cat DNA Nabs Bad Guys.” She begins with the following case:
    On Sept. 14, 2000, Wayne Shumaker, 58, Corby Myer, 30, and Lynn Ganger, 54—three carpenters building a barn loft at an upscale property near Lakeville, Indiana—were bound and shot execution style during an armed robbery. Less than two years later, the triggerman in the case, Phillip Stroud, was found guilty on all three counts of murder and sentenced to life in prison. The criminal was done in—at least in part—by the dog droppings he had stepped in during the commission of the crime. It turns out that dog feces not only messed up his sneakers, but his defense too. It was a simple mistake that was exploited by the prosecution using some new and very sophisticated science. Samples from Stroud’s sneakers were compared to dog feces at the barn. Through DNA analysis (as they exit, feces snag DNA-carrying epithelial cells from the colon), the specimens turned out to be a perfect match—proof positive that the defendant had been present at the scene of the crime.
    In the interview, Ms. Croke elaborated on "proof positive" as follows:
    [T]he lab needs to calculate probabilities. How common is this particular pattern of DNA in the wider population? In other words, how likely is it that this hair could have come from any other dog or cat than the one linking the criminal to the crime? In the triple murder case we were talking about, the probability that the feces on the suspect’s sneaker came from any other dog than the one at the scene of the crime was one in ten billion!
    Here and Now's webpage thus refers to "a probability test to determine how likely it is that the DNA comes from any other animal in the area."

    These characterizations are fairly typical examples of transposition. The data in the triple-murder case are the pair of DNA profiles that are said to match. The hypothesis is that the source of the sneaker DNA is a different dog. We’ll call this the defense hypothesis, def-hyp. Assuming no laboratory error in profiling ever occurs, the probability of the data—the matching profiles—given that they came from different (and unrelated) dogs is the frequency of the profile in the “wider population.” Let’s assume that one in ten billion is a good estimate of that probability. That is,

    P(data | def-hyp) = 1/10,000,000,000.

    Is “the probability that the [DNA] came from any of other dog” also 1/10,000,000,000? Not exactly. This probability is P(def-hyp | data). According to Bayes' rule, it depends not only on P(data | def-hyp), but also on two additional probabilities. For one thing, we need to know the probability of the data given the prosecution’s hypothesis, P(data | pros-hyp). This probability is 1 (if the lab is certain to declare a match when the two samples really contain the same dog's DNA).

    Another factor to consider is the prior probability of the defense hypothesis, P(def-hyp). How many alternative dogs could have been crossing defendant’s paths in the weeks before the murder? One thousand seems like a lot. If we take the prior odds for the defense hypothesis to be 1,000:1, then the match to the dog doodoo in the barn reduces these odds to 1,000:10,000,000,000 = 1:10,000,000.

    What is the moral? Transposition is wrong, but almost everybody, from journalists to jurors, does it. A DNA match to a random, unrelated dog may be a one in ten-billion event, but it does not follow that the probability that defendant stepped on stuff from an unrelated dog is one is ten billion. That said, if the random-match probability is as infinitesimal as one in ten billion, the probability of the defense hypothesis (about an unrelated dog being the source of a true match) is still doggone small. Transposition should be avoided, but it is not always the most grievous of errors.
    Posted by at No comments:
    Labels: , , ,

    Sunday, 4 January 2015

    Buza Reloaded: California Balancing

    This is the fourth installment on Buza II, the opinion of the California court of appeal that invalidates the state's DNA-on-arrest law. It discusses the part of the opinion that argues that the balance the U.S. Supreme Court struck in Maryland v. King is either flatly wrong or wrong for California. In giving substantial weight to concerns over "familial searching" and the information content of DNA samples, the opinion assumes that it is appropriate to strike down a law that is constitutionally reasonable as currently implemented because future developments might make it unreasonable as then implemented. This premise is highly contestable.

    Formally, the conclusion that California's DNA-BC (Before Conviction) law is unreasonable under the Fourth Amendment as it appears in the California Constitution does not imply that it is unreasonable under the Fourth Amendment as it exists in the U.S. Constitution. California is a sovereign state of the Union, and its courts can read different meanings into the words of its constitution. But many of the reasons the Buza II opinion gives for its conclusion—if correct—also apply to nearly all of the 25 or so DNA-BC laws on the books, and the opinion itself indicates that, in large part, the divergence between Buza II and King emanates from the California judges’ outright disagreement with the Supreme Court's balancing in King.

    To begin with, the California judges complain that King “unjustifiably dismissed concerns about the extent of the personal information contained in DNA samples by limiting ... attention to the profile used in DNA databanks, as currently restricted by statutes and scientific capability.” One might expect that this observation immediately would be followed by the undeniable fact that the entirety of a person’s genome contains some medically significant information that would not otherwise be known, such as predispositions to certain diseases. Testing for these alleles (or for markers for them) would pose significant privacy issues (which is why such testing generally is prohibited without the individual’s consent).

    But the opinion veers off into a superficial discussion about the CODIS profile itself. The problem, according to Buza II, is that the profile can be used not merely to identify an individual whose DNA is taken when he is arrested, but also sometimes can be used to identify a first-degree relative as a likely source (when the arrestee’s DNA is a close mismatch to the crime-scene sample). This “familial searching,” as the court calls it, is a “factor not relevant to identity,” and therefore “present[s] additional privacy concerns.”

    The second part of this statement is true enough. Like a perfect match, a close mismatch is relevant to the identity of the DNA source, but it also reveals that the arrestee could be genetically related to the source of the crime-scene DNA. 1/ Consider the “Grim Sleeper” case of serial rapes and murders in the Los Angeles area, with years of apparent inactivity between some of the attacks. Trawls of the database proved fruitless—until Christopher Franklin was convicted of a felony. His DNA profile did not match the Grim Sleeper’s, but it lined up with it in a manner that would be expected if the two were father and son. This led investigators to Christopher’s father, Lonnie Franklin, Jr. In this way, Lonnie emerged as a suspect only because of his son’s conviction. (His DNA profile was not in the database because his arrests had occurred before California had a database.) Now he stands accused of ten murders.

    People v. Franklin reveals an important fact about kinship trawling. In Franklin, it is difficult to discern the slightest “additional privacy concerns.” That Lonnie was Christopher’s father was a publicly known fact, not a private secret. Furthermore, Lonnie can hardly claim to have a legitimate Fourth Amendment interest in keeping secret the fact that it was his DNA that was found on or around murdered women. 

    Of course, there could be other cases in which the familial relationship between the database inhabitant and the culprit was not known to one or both of the genetically related individuals. In such situations, the claim to a right to keep the genetic relationship secret is more plausible. But the existence of possible cases of this kind does not demonstrate that the occasional legitimate privacy interests that might be affected by the rare, "other-directed" trawls (that look for people outside of the database) outweigh those of the government.

    In particular, for Mark Buza and his relatives to have an additional privacy interest compromised by the arresteee database, at least two conditions would have to be fulfilled. First, California would have to initiate other-directed trawls of its arrestee database. It has never done so, and it cannot do so under the policy its Department of Justice has adopted for such database trawling. This policy confines the other-directed trawling to convicted-offender databases. Second, Mark Buza would have to have publicly unknown first-degree relatives whose DNA profile would be close enough to Mark’s to implicate them in other crimes via a kinship match to Mark’s profile.

    On its face, the first condition suggests that the parts of the opinion discussing “familial searching” are inapposite. Why strike down a law because of what could be but is not? Nonetheless, the Buza court’s sensitivity to the possibility of a change in the state’s DNA-BC practice might be seen as prescient rather than premature. From the outset, an argument against DNA databases has been mission creep. Once the database is established, the state will be tempted to use it for additional and more insidious purposes. To guard against this outcome, the argument goes, society should bind itself to the mast in anticipation of an irresistible siren song.

    There are situations in which this self-disabling strategy is advisable. Indeed, much of the Bill of Rights constrains the majority from doing what seems expedient or appealing in the heat of the political moment. But it is not so clear that a handful of judges should block the democratic decision to allow DNA-BC to be used in acceptable ways that advance law enforcement on the ground that the system might be administered in unacceptable ways at some future time. If and when a jurisdiction combines other-directed trawling and DNA-BC, courts can consider whether that type of trawling is so serious an invasion of privacy as to render it unconstitutional. Cf. United States v. Knotts, 460 U.S. 276 (1983) ("if such dragnet type law enforcement practices as respondent envisions should eventually occur, there will be time enough then to determine whether different constitutional principles may be applicable."). Using the mere possibility of a correctable change in the allowed uses of the DNA data to strike down the collection and otherwise acceptable uses of the data seems Draconian.

    Moreover, relying on future familial searching as a ground for striking down the system as currently implemented is inconsistent with Buza II’s effort to distinguish the Maryland practice. Presiding Justice Kline emphasized the existence of a Maryland statute banning familial searching. But as Chief Judge Alex Kozinski of the U.S. Court of Appeals for the Ninth Judicial Circuit tartly observed in oral argument in Haskell v. Harris (a separate case challenging California DNA-BC law), statutes can be changed too. The logic of Buza II—that databases that are constitutionally reasonable (as currently implemented) but might become unreasonable (as implemented in the future) are constitutionally unreasonable ab initio—would render the Maryland law on DNA-BC unconstitutional.

    Despite these problems, Buza II applies the nip-it-in-the-bud reasoning not only to DNA profiles but also to samples. Displaying little knowledge of behavioral genetics, the court invokes “the pedophile gene” and “the violence gene” that, it imagines, might well be discovered some day. It predicts that “surely law enforcement will seek to mine genetic information for that ‘identification purpose.’” 
    But there is no good reason to believe that the word “identification” as used in DNA-BC laws would permit predictive genetic testing for these behaviors, and the court makes no attempt to explain why such testing could not be condemned as constitutionally unreasonable if and when the time arises.

    My criticism of the court of appeal's reliance on dystopic visions of the future is not based on naive faith in the goodness of police and law enforcement laboratories. Courts need not—and should not—trust law enforcement to exercise perfect self-restraint in investigative methods that easily can be abused. Before approving a DNA database system, they should satisfy themselves that sufficient safeguards against predictable abuses are in place. But if such protections are present, courts should not invalidate a system because the safeguards might be removed or might cease to be effective in the future. In this case, might does not make the decision right.

    Note
    1. Confusingly, the court presents this fact as if it "disproves the King majority’s assumption that 'the CODIS loci come from noncoding parts of the DNA that do not reveal the genetic traits of the arrestee.'" Some noncoding DNA does affect visible traits of an arrestee, but the CODIS loci, as far as current science can tell, do not reveal much about any phenotypes. Because all DNA sequences are inherited, however, including those that King (also confusingly) calls "junk," the ones that vary across individuals, can be used in kinship analysis. In fact, the sequences that do give rise to individual traits often are the best for this purpose because they tend to be extremely variable within populations.
    References
    Closely related postings
    Posted by at No comments:
    Labels: , , , ,

    Thursday, 1 January 2015

    Buza Reloaded: Fourth Amendment Balancing


    This is the third installment on Buza II, the opinion of the California court of appeal that invalidates the state's DNA-on-arrest law. It comments on the court of appeal's treatment of the definition of a "search" and the court's claim that the reasons the Supreme Court gave in Maryland v. King for upholding post-arraignment DNA collection from Alonzo King are "completely inapplicable" to taking DNA from Mark Buza during booking.

    Having described the scope of totality-of-the-circumstances balancing in exceedingly generous terms, Presiding Justice Kline turns to a logically prior Fourth Amendment question—should the DNA collection from Buza be considered a search under the Fourth Amendment? This definitional issue is a condition precedent to reasonableness balancing in that the state has no burden of showing reasonableness if it does not conduct a “search or seizure” within the meaning of the amendment.

    Again, the court reaches the correct conclusion—of course there was a search—but the exposition raises some questions. Presiding Justice Kline wrote that
    Nonconsensual extractions of substances that may be used for DNA profiling are “searches” entitled to the protection of the Fourth Amendment. (King, supra, 133 S.Ct. at p. 1969 [buccal swab]; Schmerber v. California (1966) 384 U.S. 757, 767-771 (Schmerber) [blood]; People v. Robinson, supra, 47 Cal.4th at p. 1119 [blood]; Skinner v. Railway Labor Executives’ Assn. (1989) 489 U.S. 602, 616-617 [breathalyzer and urine sample]; Cupp v. Murphy (1973) 412 U.S. 291, 295 [finger nail scrapings].) The physical intrusion involved in the buccal swab procedure used in the present case has been viewed as minimal. (King, at p. 1969.) The collection of the DNA sample, however, is only the first part of the search authorized by the DNA Act; the second occurs when the DNA sample is analyzed and a profile created for use in state and federal DNA databases. The latter search is the true focus of our analysis. (Brackets in original).
    Most of this paragraph is gratuitous. Why not stop with the irrefragable proposition that King held that buccal swabbing is a search? Every Justice of the U.S. Supreme Court accepted this premise, and not one maintained that there was one search followed by another search. According to the King majority, "using a buccal swab on the inner tissues of a person's cheek in order to obtain DNA samples is a search" simply because "[v]irtually any intrusion into the human body will work an invasion of cherished personal security that is subject to constitutional scrutiny.” 133 S.Ct. at 1969 (internal quotations, citations, and alteration omitted).  “The ... intrusion [although] negligible ... is still a search as the law defines that term.” Id.

    In contrast, whether trawling a database for a DNA match to the DNA profile derived from the sample acquired by the physical intrusion is a “latter search” is an open question. Federal appellate courts have reached different conclusions about it, and the King Court did not discuss it (despite the insistence of King’s counsel at oral argument that DNA analysis and trawling is a search in itself). To be sure, the nature of DNA profiling and trawling is “of central relevance,” King, 133 S.Ct. at 1969—but only on the distinct question of whether the search is reasonable. Id.

    Before turning to that question, the Buza II court expressed some sympathy for Justice Scalia’s “piercing” dissent. This position is at odds with Presiding Justice Kline’s description of the Fourth Amendment as always accommodating balancing, since the entire theory of Justice Scalia’s dissent was that the reasonableness of DNA sampling on arrest is not to be determined by weighing individual and state interests, but by a putative categorical rule against certain suspicionless searches

    Be that as it may, the Buza II court maintained that King’s “reasons for upholding the Maryland law [are] completely inapplicable to California’s.” That conclusion seems precipitous. There is ample room to debate the value of arrestee sampling and the significance of the differences of the Maryland and California statutes. Indeed, I have written that Justice Kennedy’s reasoning in King is “contrived,” but only because a candid approach to totality balancing (or, preferably in my view, toward creating a new categorical “biometric exception” to the general rule against warrantless searches) has to incorporate the power of DNA to solve cases and deter crimes sooner rather than later (or not at all). 1/

    But surely King’s detention-related reasons for DNA testing cannot be dismissed as “completely inapplicable.” For example, Buza II cited the fact that whereas the California law covers all felony arrests, the Maryland law encompasses only arrests for “murder, rape, first-degree assault, kidnaping, arson, sexual assault, and a variety of other serious crimes.” King, 133 S.Ct. at 1967. The supposition that DNA from individuals arrested for nonviolent felonies is less likely to provide a link to an unsolved crime hardly means that the DNA is devoid of investigative value. Car theft is not a violent crime or a burglary, but at least some car thieves leave behind traces of DNA while committing more serious crimes

    Furthermore, even the more sensible reasoning that arrestee DNA is, on average, less valuable to the state for some felonies than for others does not distinguish the demands the states made on the defendants in King and Buza. Mark Buza, after all, was caught committing arson—surely a serious crime. Why is the Buza court addressing the law as applied to a hypothetical person at another time? Under established Fourth Amendment doctrine, Buza was justifiably arrested for arson. As such, he lacks standing to assert the rights of individuals arrested for much less serious offenses. If it was constitutional to have compelled King to give a DNA sample—which is what the Supreme Court held—why is not equally constitutional to have ordered Buza to give a DNA sample? The answer cannot be that the Supreme Court in King was confronted a serious felony arrest. The court of appeal in Buza likewise was confronted with a serious felony arrest.

    Indeed, the Buza opinion itself backs away from the extreme position that King’s reasoning is “completely inapplicable.” The court announces that it is not actually deciding the fundamental question of “whether these differences require a different resolution of the issue from that of the King majority.” It ends its Fourth Amendment analysis with more of a whimper than a bang, dismissing the entire issue: “Our conclusion that the DNA Act is invalid under article I, section 13, of the California Constitution renders it academic whether the Act is also invalid under the Fourth Amendment.” The court is convinced that the differences “significantly alter the weight of the governmental interests and privacy considerations to be balanced in determining constitutionality under the Fourth Amendment,” but it seems less certain that this alteration truly produces a different result.

    To reach a different result, the Buza II court performs its own balancing under an unusually demanding balancing test. The next installment comments on this California balancing.

    Note
    1. David H. Kaye, Why So Contrived? DNA Databases After Maryland v. King, Journal of Criminal Law & Criminology, Vol. 104, No. 3, pp. 535-596, May 2014, available at http://ssrn.com/abstract=2376467
    Closely related postings
    Posted by at No comments:
    Labels: , ,

    Monday, 22 December 2014

    Buza Reloaded: The Fourth Amendment Framework

    This is the second installment of remarks on Buza II, the opinion of the California court of appeal that invalidates the state's DNA-on-arrest law. It questions the opinion's expansive view of when "totality of the circumstances" balancing is appropriate under the Fourth Amendment

    The Buza II court begins its legal analysis with the reasonableness clause of the Fourth Amendment. Omitting the clause that refers specifically to judicial warrants, the opinion tells us that "[t]he Fourth Amendment protects '[t]he right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures' . . . ." However, the court adds that "[s]ubject only to a few specifically established and well-delineated exceptions not applicable here, warrantless searches are per se unreasonable under the Fourth Amendment." Of course, if this were literally true, the case would be over, since California had no judicial warrant to compel Buza to submit to DNA extraction, and there was no "applicable exception." (The search-incident-arrest exception only permits warrantless searches for weapons or evidence that the arrestee might reach.)

    The court of appeal apparently does not believe that there is a true per-se-unreasonable-with-exceptions (PSUWE) rule. Instead, it presents the absence of a warrant as a mere burden-shifting device.The state, it says, "thus bears the burden of showing that the search at issue is reasonable and therefore constitutional. (People v. Williams (1999) 20 Cal.4th 119,127.)" This interpretation cuts the heart out of the PSUWE rule and does not flow from People v. Williams. In Williams, the California Supreme Court wrote that when moving to suppress evidence, "defendants have the burden of (1) asserting the search or seizure was without a warrant, and (2) explaining why it was unreasonable under the circumstances." 83 Cal.Rptr.2d 275, 282 (Cal. 1999) (emphasis added). It seems odd to cite the case for the opposite proposition--that the burden is on the state.

    But this questionable use of precedent is a minor sin. The more disturbing aspect of Buza II's exposition of the Fourth Amendment lurks in its view that a court always can find reasonableness "by examining the totality of the circumstances," which is to say, "by balancing its intrusion on the individual’s Fourth Amendment interests against its promotion of legitimate governmental interests." The U.S. Supreme Court cases cited for this open-ended approach to reasonableness do not dilute the PSUWE rule to this degree.1/

    In fact, if the Buza II court's claim that every warrantless search can be saved by a finding of reasonableness under the circumstances were correct, the Supreme Court would not have had such a difficult time explaining in Maryland v. King why Maryland's pre-conviction DNA law was subject to a balancing test. Rather than announce that "we always balance," Justice Kennedy, writing for the majority, tried to weave together specific features of other cases departing from the PSUWE framework in favor of a direct balancing test. Whether the resulting tapestry is so poorly woven that it is about to unravel has been debated, 2/ and Buza II misapprehends the but the Court itself has not officially abandoned the PSUWE framework. Indeed, soon after King, in holding that a warrantless search of a cellphone during an arrest was impermissible, the Court reiterated that "[i]n the absence of a warrant, a search is reasonable only if it falls within a specific exception to the warrant requirement," Riley v. California, 134 S.Ct. 2473, 2482 (2014), and demanded that "even when a cell phone is seized incident to arrest," id. at 2494, the police had better "get a warrant." Id. at 2495.

    Although the Buza court is thus mistaken in its treatment of the threshold question of when direct, totality balancing is permissible, King does hold that a form of balancing is required for warrantless DNA swabbing. In the next installment, I will consider the court of appeal's effort to dismiss this balancing, as performed by the Supreme Court, as "completely inapplicable."

    [Next installment]

    Notes
    1. See David H. Kaye, On the “Considered Analysis” of DNA Collection Before Conviction, 60 UCLA L. Rev. Disc. 104 (2013); David H. Kaye, Why So Contrived? DNA Databases After Maryland v. King, 104 J. Crim. L. & Criminology 535 (2014).
    2. Compare Erin Murphy, License, Registration, Cheek Swab: DNA Testing and the Divided Court, 127 Harv. L. Rev. 161 (2013), with David H. Kaye, Maryland v. King: Per Se Unreasonableness, the Golden Rule, and the Future of DNA Databases, 127 Harv. L. Rev. Forum 39 (2013).
    Closely related postings
    Posted by at No comments:
    Labels: , ,

    Sunday, 7 December 2014

    Buza Reloaded: Court Shifts Ground But Again Invalidates California’s DNA-on-arrest Law

    Buza I
    For the reasons we have set forth, we conclude that the DNA Act ... unreasonably intrudes on such arrestees' expectation of privacy and is invalid under the Fourth Amendment of the United States Constitution.
    Buza II
    For the reasons we have set forth, we conclude that the DNA Act ... unreasonably intrudes on such arrestees’ expectation of privacy and is invalid under article I, section 13, of the [California] Constitution.

    Nearly seven years ago, Mark Buza set a San Francisco police car on fire to protest what he said was a corrupt government. Spotted running from the burning car, he was promptly arrested and brought to the county jail. There, he engaged in a second act of defiance, refusing to allow police to swab the inside of his cheeks to acquire a DNA sample. (He did, however, acquiesce in fingerprinting and writing a signature.) After his conviction for arson-related crimes as well as the separate offense of refusing to submit to DNA sampling, a court ordered him to give a sample before sentencing. In a third act of resistance, he refused, but when the court issued a further order allowing police to use force, he finally submitted to the swabbing. His DNA profile entered the California database, apparently without incriminating him in any other crimes.

    The California Court of Appeal reversed the conviction for resisting “the seizure of his DNA at a time when he was entitled to the presumption of innocence, and [when] there had been no judicial determination of probable cause to believe he committed the offense for which he was arrested ... .” California's DNA-collection law, the court reasoned, “violated his Fourth Amendment right to be free from unreasonable searches and seizures.” The court expressly declined to consider whether compelling DNA collection before a judicial finding of probable cause violated Buza’s “right to privacy under article I, section 1, of the California Constitution.”

    The California Supreme Court granted review, vacated the appellate court’s judgment, and ordered that court to reconsider its reversal of the DNA-refusal conviction in light of the United States Supreme Court’s decision in Maryland v. King, 133 S.Ct. 1958 (2013).

    In a lengthy opinion issued four days ago, the Court of Appeal, like Buza himself, stuck to its guns. For a second time, it held that the California initiative that mandates DNA sampling on arrest (and that has been upheld by several other courts) is unconstitutional. However, the court shifted ground. Now Presiding Justice Anthony Kline wrote: “we question whether King establishes the validity of the California Act’s application to arrestees under the Fourth Amendment. We base our decision, however, solely upon article I, section 13, of the California Constitution, which in our view undoubtedly prohibits the search and seizure at issue.” (Emphasis added.)

    I’ll leave it to California’s bar, bench, and scholars to debate how and why the Court of Appeal can be so certain that California’s constitution, which essentially reproduces the words of the Fourth Amendment, compels a different result than King might warrant. Here, I want to consider the Court of Appeal's analysis of the outcome that should follow under the U.S. Constitution as construed in King.

    [Next installment]

    Closely related postings
    References
    Posted by at No comments:
    Labels: , ,

    Thursday, 30 October 2014

    Another Disgusting DNA Case: Please Flush!

    A quick update to the disgusting DNA report of last March (The Whooper Stopper). Following up on a report from KOAT-TV in Albuquerque, the Associated Press reported that a man helped himself to $250,000 worth of jewelry from a home while the owners were on vacation and neglected to flush whatever he left behind in the toilet. Is there is a lesson to criminalists here: swab toilet seats and handles to catch more considerate burglars as well? Probably not. Cf. Peter Gill, Misleading DNA Evidence: Reasons for Miscarriages of Justice (2014) (discussing the interpretation of touch DNA).

    Far more offensive is a case reported by the same TV station in 2011. It seems that a 31-year-old Sunflower Market employee offered a woman what he said was a free yogurt sample. Incredibly, it was his semen. After pleading guilty, he was sentenced to a two-year prison term.
    Posted by at No comments:
    Labels: ,

    Tuesday, 7 October 2014

    The Supreme Sound of Silence: Same-Sex Marriage and DNA Databases

    The big news among Supreme Court watchers is the big dog that did not bark in the night — the Court’s denial of petitions for certiorari in seven cases striking down bans on same-sex marriage in Indiana, Wisconsin, Utah, Oklahoma, and Virginia. [1] A denial of a cert petition has no precedential value. It does not mean that the Court approves of the decision below—or that it disapproves of it. It means that, for unstated and often banal reasons (the Court receives some 10,000 petitions a year [4]), no more than three Justices voted in favor of review the decision below. (By convention, it takes four votes to grant the writ that triggers the Court’s review of the case on the merits.)

    The Court watchers are treating the rejection of the writs here as a “tacit win to gay marriage” on the theory that it means that if and when the Court chooses to confront the issue, a majority of states will have sanctioned same-sex marriage, making it more likely that the Court will accept the argument that the Constitution forbids limiting the institution of marriage to couples of the opposite sex. [3]

    This predicted dynamic was evident in the Court’s handling of laws requiring routine DNA collection for law enforcement databases. No appellate court ever struck down a law requiring convicted offenders to provide samples, and for some thirty years, the Court invariably denied petitions for review in those cases. Only after Maryland’s highest court essentially invalidated that state’s law providing for DNA collection on arrest did the Supreme Court step in. By that time, every state had a DNA database for convicted offenders, and a majority had extended them to require pre-conviction DNA sampling. Every state signed an amicus brief urging the Court to uphold the practice. The Court split 5–4 on the constitutionality of pre-conviction DNA testing. Had the states and the federal executive branch not presented so unified a front in favor of expansive DNA collection, the outcome could have been different. [2]

    References
    1. Amy Howe, Today’s Orders: Same-sex Marriage Petitions Denied, SCOTUSblog, Oct. 6, 2014, 10:41 AM, http://www.scotusblog.com/2014/10/todays-orders-same-sex-marriage-petitins-denied/
    2. David H. Kaye, Why So Contrived? DNA Databases After Maryland v. King, 104 J. Crim. L. & Criminology 535 (2014), available at http://ssrn.com/abstract=2376467
    3. Adam Liptak, Supreme Court Delivers Tacit Win to Gay Marriage, N.Y. Times, Oct. 7, 2014, at A1, http://www.nytimes.com/2014/10/07/us/denying-review-justices-clear-way-for-gay-marriage-in-5-states.html
    4. Robert M. Yablon, Justice Sotomayor and the Supreme Court’s Certiorari Process, 123 Yale L.J. F. 551 (2014), http://yalelawjournal.org /forum/justice-sotomayor-and-the-supreme-courts-certiorari-process.html
    Posted by at No comments:
    Labels: ,

    Tuesday, 30 September 2014

    Bayes in Our Times

    Today's New York Times has an article on "a once obscure field known as Bayesian statistics."1/ It is an informative piece by Faye Flam, a science journalist with an uncommonly good grasp of science. But a quantum of confusion infects the effort to contrast "Bayesian statistics" with "the more traditional or 'classical' approach, known as frequentist statistics."

    The article presents the solution to famous Monty Hall problem (known to "classical" probabilists as the three-curtains problem long before its appearance in the TV game show) as especially amenable to "Bayesian statistics." But frequentist thinking works quite well here. In the long run, the strategy of switching beats the strategy of not switching. This is easily proved with classical, objective probabilities.

    Indeed, it is not clear that the Monty Hall problem is even a problem in statistical inference.2/ There are no statistical (sample) data to consider and no sense in which the use of Bayes' rule to solve the probability problem "counter[s] pure objectivity." How, then, do "[t]he two methods approach the same problem[] from different angles"?

    Of course, the Monty Hall problem is nice for illustrating the power of Bayes' rule in working with conditional probabilities. I have used it in this way in my courses, and that may have been the reason it appears in the article. But it does not illustrate the philosophical divide between frequentists and Bayesians.

    To this extent, it is disappointing that the Times (but probably not the author) chose to start the online version of the article with a large photograph of Monty Hall captioned "Bayesian statistics can help solve the Monty Hall problem of winning a car." It would have been equally accurate to report that "Frequentist statistics can help solve the Monty Hall problem of winning a car." But that is is hardly news fit to print.

    Notes

    1.Faye D. Flam, The Odds, Continually Updated, N.Y. Times, Sept. 30, 2014, at D1.

    2. On the distinction between a "problem of statistical inference or, more simply, a statistics problem," and a probability problem, see, for example, Morris H. DeGroot, Probability and Statistics 257 (1975).


    Posted by at No comments:
    Labels: ,

    Sunday, 31 August 2014

    Hazard Ratios and Heart Failure

    Today’s big news in medicine is a new drug, designated LCZ696 by its manufacturer, Novartis. According to the New York Times, LCZ696 “has shown a striking efficacy in prolonging the lives of people with heart failure and could replace what has been the bedrock treatment for more than 20 years.” [1] Specifically, more than 8,400 patients in 47 countries enrolled in a randomized, double-blind experiment in which they received either LCZ696 or an ACE inhibitor called enalapril (in addition to whatever else their doctors prescribed).

    The trial was halted after a median follow-up time of 27 months “ because the boundary for an overwhelming benefit with LCZ696 had been crossed.” [2] “By that point, 21.8 percent of those who received LCZ696 had died from a cardiovascular cause or had been hospitalized for worsening heart failure. That figure was 26.5 percent for those receiving enalapril. That represents a 20 percent relative reduction in risk using a statistical measure called the hazard ratio.” [1]

    This is good news for patients (if the drug receives regulatory approval and performs as expected in practice). But the account in the Times poses a small statistical puzzle. How does the difference between 21.8 and 26.5 percentage points translate into “a 20 percent relative reduction in risk”? The average risk across patients dropped by 26.5 – 21.8 = 4.7 percentage points. This absolute reduction is appreciable, but 4.7 percentage points is not 20% of the original 26.5 percent risk of hospitalizations or deaths in the control group (4.7 / 26.5 = 17.7%). What accounts for the discrepancy?

    The answer lies in the details of a technique known in biostatistics as survival analysis. The statistical technique is not limited to the analysis of death rates. It can be applied to all sorts of situations involving different times to some outcome. The outcome can be the overruling of a Supreme Court case, the firing of a worker, or the exoneration of a prison inmate sentenced to die, to pick a few examples from forensic statistics.

    So what does the 20% “relative reduction in risk” cited in the Times article mean? Well, a hazard function is the probability that if you survive to a given time t (the event in question has not already occurred), you will survive in the next instant. A hazard ratio is the ratio of the hazard in the treatment group to the hazard in the control group at t. The heart failure study used an estimation procedure known as proportional hazards regression, which assumes that the hazard in one group is a constant proportion of the hazard in the other group. Under this assumption, in a clinical trial where death is the endpoint, the hazard ratio indicates the relative likelihood of death in treated versus control subjects at any given point in time.

    Thus, unlike the ordinary relative risk discussed in many court opinions, the “hazard ratio” is not simply the proportion with a disease in an exposed group divided by the proportion in an unexposed group. In the LCZ696 study, the hazard ratio was 0.80, meaning that the probability that a randomly selected patient taking LCZ696 would die from or be hospitalized for heart failure the next day is 80% of the probability for a randomly selected patient taking enalapril. To put it another way, the probability of hospitalization or death tomorrow from heart failure drops by 20% when LCZ696 is substituted for enalapril.

    Yet a third formulation is that the odds that a randomly selected patient treated with LCZ696 will be hospitalized or die sooner than a randomly selected control patient are 0.8 (to 1) — that's 4 to 5, corresponding to a probability of 4/9 = 44%. [3]

    How long either patient can expect to live and avoid hospitalization from heart failure is another story. As one article on hazard ratios explains, “[t]he difference between hazard-based and time-based measures is analogous to the odds of winning a race and the margin of victory.” [3] By itself, the hazard ratio picks the winning horse (probably), but it does not give the number of lengths for its expected success.

    References
    1. Andrew Pollack, New Novartis Drug Effective in Treating Heart Failure, N.Y. Times, Aug.31, 2014, at A4
    2. John J.V. McMurray et al., Angiotensin–Neprilysin Inhibition versus Enalapril in Heart Failure, New Engl. J. Med., Aug. 30, 2014
    3. Spotswood L. Spruance et al., Hazard Ratio in Clinical Trials, 48 Antimicrobial Agents and Chemotherapy 2787 (2004)
    Posted by at No comments:
    Labels: , , ,

    Thursday, 31 July 2014

    The FBI's Worst Hair Days

    An article by Spencer Hsu in yesterday's Washington Post suggests that the FBI lost a tug of war within the Justice Department. In 2012, the Bureau commenced a comprehensive review of the testimony of FBI hair analysts about matches to defendants in criminal cases before 2000. In those pre-DNA-evidence days, microscopic hair comparisons were valuable for seeing whether a suspect could be the source of a hair at a crime scene. (They still are, but the FBI now uses mitochondrial DNA testing to demonstrate a positive association and relies on visual comparison to screen out nonmatching hairs.)

    Clearly, an inclusion—that is, two hairs with a similar set of features—was never definitive. Even hairs from the same individual vary in certain respects. But hairs from the same individual are more likely to "match" than hairs from different individuals. Thus, a careful hair analyst should have reported a negative finding as an exclusion and a positive finding with words like "not excluded," "could have," "consistent with," or "match, but."

    After it became apparent that the FBI’s analysts were not always being this careful, the Department of Justice agreed to “identify[] historical cases for review where a microscopic hair examination conducted by the FBI was among the evidence in a case that resulted in a conviction ... with the goal of reaching final determinations in the coming months.” That was 2012. The Post article reports that in 2013, the FBI stopped the reviews. It started them back up this month, on orders from the Deputy Attorney General.

    The FBI attributes the delay, in part, to “a vigorous debate that occurred within the FBI and DOJ about the appropriate scientific standards we should apply when reviewing FBI lab examiner testimony — many years after the fact.” To get a sense of what this debate might have been about, it may be useful to examine the two specific cases mentioned in the Post article on “forensic errors.”

    The Exoneration of Santae Tribble

    The article includes an imposing photograph of Santae A. Tribble. The caption explains that Tribble, who was convicted in Washington, D.C., at age 17, “spent 28 years in prison based largely on analysis of hairs found at the scene of a taxi driver’s murder in 1978. More advanced DNA testing showed that none of the hairs used as evidence shared Tribble’s genetic profile. A judge has vacated his conviction and dismissed the underlying charges.”

    There is no denying that evidence suggesting that an innocent man is guilty is erroneous, but is it a laboratory error? Some people argue that microscopic hair evidence is unvalidated and because it sometimes incriminates innocent people, it should be inadmissible. But if that is correct, why go through the trouble of reviewing all the cases? The FBI could just send out letters in every case saying that the laboratory no longer stands by the unvalidated testimony its examiners gave.

    Surely there was (and is) some useful information in microscopic hair comparisons. A 2002 FBI study showed that DNA testing confirmed most visual microscopic associations (almost 90%) on a sample of hairs from casework. For a small minority of hair comparisons—as in Mr. Tribble’s case—microscopy produced false positives.The specificity of the technique—like that of drug tests, tests for strep throat, and so many other things—is not 100%.

    Inasmuch as all hair comparisons cannot summarily be dismissed as invalid, what makes the comparison in the Tribble case a departure from the FBI calls “appropriate scientific standards”? An article from the National Association of Criminal Defense Lawyers (NACDL), which is cooperating in the process of reviewing the cases, describes the criteria as follows:
    Error Type 1: The examiner stated or implied that the evidentiary hair could be associated with a specific individual to the exclusion of all others.

    Error Type 2: The examiner assigned to the positive association a statistical weight or probability or provided a likelihood that the questioned hair originated from a particular source, or an opinion as to the likelihood or rareness of the positive association that could lead the jury to believe that valid statistical weight can be assigned to a microscopic hair association.

    Error Type 3: The examiner cites the number of cases or hair analyses worked in the lab and the number of samples from different individuals that could not be distinguished from one another as a predictive value to bolster the conclusion that a hair belongs to a specific individual.
    Which of these errors did the FBI laboratory commit in Mr. Tribble’s case? According to an earlier Post article on the case, “A police dog found a stocking on a sidewalk a block away [from the victim’s body]. Months later, the FBI would report that a single hair inside it matched Tribble’s ‘in all microscopic characteristics.’” Ideally, the analyst would have added that hair from other people also could have matched, or, at the least, defense counsel should have elicited this fact on cross-examination.

    No such significant qualifications or caveats emerged. Instead, according to the Innocence Project, the FBI analyst "testified that one of the hairs from the stocking mask linked Tribble to the crime." The National Registry of Exonerations reports that he "said ... the hair in the stocking came from Tribble." Such testimony seems to be an "Error Type 1," although it is not clear from these descriptions whether the "link" was explicitly "to the exclusion of all others."

    The latter phrase was extremely popular among analysts of impression and patterns (like fingerprints and toolmarks) who believed that their disciple studies characteristics that can exist in their particulars in only one object in the universe. Of course, the words "to the exclusion" are logically redundant. If the analyst believed that "the hair ... came from Tribble," then he must have believed that it did not come from anyone else. But one can believe that a named individual is the source of a trace (because that is the most likely conclusion) without believing it is impossible for anyone else to have been the source (which is, I think, is what "to the exclusion" was supposed to mean).

    Thus, there is an ambiguity in the meaning of an "Error Type 1." How explicit must the analyst be in excluding all other individuals as contributors of the hair? The NACLD's description of the criteria indicates that a literal use of the phrase is not critical. The article illustrates the error with the following, hypothetical testimony:
    I found brown, Caucasian head hairs on two items of clothing, the sports coat, and a pair of slacks that were reported to me as belonging to (the defendant). Now, these hairs matched in every observable microscopic characteristic to that known hair sample of DEC (the decedent) and consistent with having originated from her. In my opinion, based on my experience in the laboratory and having done 16,000 hair examinations, my opinion is that those hairs came from DEC.
    But regardless of whether Tribble's trial testimony included an "Error Type 1" as the FBI has defined the errors, it was excessive. The analyst should have stuck to reporting the results of the comparison and not made a source attribution.

    In addition to the analyst's overstated testimony, the prosecutor came vanishingly close to making the “Error Type 2.” He argued in closing that “There is one chance, perhaps for all we know, in 10 million that it could [be] someone else’s hair.”

    In the end, however, what exonerated Tribble was not the recognition of the hyperbole of the expert and the prosecutor, but the proof from a DNA test that the hair on the stocking probably worn by the actual murderer was not his.

    The Conviction of John Duckett

    The second case of "forensic error" discussed in the Post article is the trial of James Duckett, a former police officer in Florida. The Post article cites this case as an example of “the continued inadequacy of officials’ response.”

    Duckett was convicted and sentenced to death for sexually assaulting, strangling, and drowning an 11-year-old girl. Unlike Tribble, Duckett has not proved actual innocence. Without such proof, even a letter from the FBI disowning some parts of the testimony in the case may not be a get-out-of-jail card.

    The analyst in the case was the now notorious Michael Malone. The Post notes that Malone was "discredited in a 1997 inspector general’s report on misconduct at the FBI lab." This report came about nine years after Duckett's conviction, and Duckett made sure the Florida courts heard about it. At the center of Duckett's latest postconviction motion was a report from an expert who had been hired by the FBI in response to the first OIG report to "review[] many cases—particularly death penalty cases—in which Malone offered expert testimony." This expert was sharply critical of Malone's documentation of his work and the unsupportable "degree of analytical certainty" with which Malone testified about the hairs in Duckett's case.

    Would a speedier review on the FBI's part have made a difference? I doubt it and have juxtaposed some of the Post’s description of the case with the court’s to indicate why.


    Duckett, then a rookie police officer in Mascotte, Fla., was convicted of raping and strangling Teresa McAbee, 11, and dumping her into a lake in 1987.

    ... Malone ... testified at trial that there was a “high degree of probability” that the hair came from Duckett.

    Such testimony is scientifically invalid, according to the parameters of the current FBI review, because it claims to associate a hair with a single person “to the exclusion of all others.”

    The Florida court denied Duckett’s request for a new hearing on Malone’s hair match. The court noted that there was other evidence of Duckett’s guilt and that the FBI had not entirely abandoned visual hair comparison.
    Malone also explained that hair analysis is not as precise as fingerprints for identifying someone. Malone expressly stated that he could not say that a particular hair came from a specific person to the exclusion of anyone else.

    (1) [T]he victim was last seen in Duckett's patrol car; (2) the tire tracks at the murder scene were consistent with those from Duckett's car; (3) no one saw Duckett, the only policeman on duty in Mascotte, from the time he was last seen with the victim until the time he met the victim's mother at the police station; (4) numerous prints of the victim were found on the hood of Duckett's patrol car, although he denied seeing her on the hood; (5) a pubic hair found in the victim's underpants was consistent with Duckett's pubic hair and inconsistent with the others in contact with the victim that evening; and, (6) during a five-month period, Duckett, contrary to department policy, had picked up three young women in his patrol car while on duty and engaged in sexual activity with one and made sexual advances toward the other two.

    Of course, the arguably redeeming parts of Malone's testimony and the state's other evidence of guilt do not condone or excuse any foot dragging by the FBI, but they do indicate the complexities that can arise in untangling the consequences of analysts' overstated testimony.

    References
    Posted by at No comments:
    Labels:

    Tuesday, 29 July 2014

    A Long Shot Pays Off in Long Island

    A family member shouted “we love you” as police took John Bittrolff back to jail. A court in Long Island had just ordered him held without bail on charges of murdering two women over 20 years ago. “Some arrests take a few hours, some days; some take 20 years,” Suffolk County Police Commissioner Edward Webber told reporters.

    If police have the killer, it is a success for “familial searching” — the practice of trawling a database for near misses that are especially likely to arise when the source of traces of DNA recovered from crime scenes or victims are very close relatives of one of the “inhabitants” of the database — convicted offenders or, increasingly, arrestees.

    Mr. Bittrolff’s DNA profile was not in the New York database. (He had been arrested, but not convicted, for assault in 1993.) However, last year, his brother, Timothy, had been required to give a sample of DNA after a conviction for violating protective orders. DNA from semen found inside the bodies of both women pointed to a brother of Timothy as the source of that semen. But the two victims were said to have been prostitutes, and Mr. Bittrolff’s counsel have been quick to note that "having sex does not mean killing."

    In addition to increasing the modest number of possibly successful “outer directed” database trawls, the case is interesting for some procedural twists involving the acquisition of DNA samples. As in the California “grim sleeper” case, police did not initially seek a court order for a sample of their suspect’s DNA to verify that he was indeed associated with the victim’s bodies. Instead, detectives helped themselves to paper bags of garbage left in front of John Bittrolff's house. Among the plastic cups, drink bottles, toothpicks, straws, crawfish heads, cotton swabs, and bandages, they found DNA from his sons, his brother, and his wife (who they trailed until they collected a cigarette butt that she tossed from the window of her truck while driving to work). And, on one paper cup, they found a DNA profile that matched the semen.

    But the police were not satisfied. They arrested wiped DNA from a cup of water John Bittrolff drank after his arrest. And even that was not enough. The assistant district attorney (ADA) then applied for a court order to force the twice-DNA-matched suspect to submit to DNA sampling.

    Defense lawyers objected that a third sample from Mr. Bittrolff was manifestly unnecessary. The ADA’s response was that prosecutors are entitled to a "judicially approved" DNA sample to present to a grand jury. The court issued the order, and that is where the case stands as of now.

    I cannot say that I understand the prosecutor’s reasoning. Unless New York grand jury procedure is very different from the norm, a prosecutor can introduce all manner of evidence without judicial approval. Grand jurors can even rely on unconstitutionally seized evidence without offending the Fourth Amendment.

    Was the ADA looking ahead to the trial? Would he want to avoid having to explain the artifices — the “familial searching,” the personal surveillance of family members, and the garbage pull — that the police used to acquire the earlier samples? He might be able to excise all that from the case with a “judicially approved” sample. In any event, the People will present their evidence to the grand jury on Thursday.

    References

    The information on the case comes from the following media reports:
    I have taken the liberty of using some words in these articles without quotation marks. For a detailed article on the nature and constitutionality of outer-directed DNA database trawling, see David H. Kaye, The Genealogy Detectives: A Constitutional Analysis of “Familial Searching”, 51 Am. Crim. L. Rev. 109 (2013)

    Related Blogs
    Posted by at No comments:
    Labels: ,
    Subscribe to: Posts (Atom)