Welcome to what we expect will be a very interesting and productive discussion of Marcello Di Bello and Collin O’Neill’s “Profile Evidence, Fairness, and the Risks of Mistaken Convictions.” The paper is published in the most recent issue of Ethics, and is available here. David Wasserman has kindly agreed to contribute a critical précis, and it appears immediately below. Please join in the discussion!
[David Wasserman herewith:] “Profile Evidence, Fairness, and the Risks of Mistaken Convictions” takes a fresh approach to a problem that has vexed philosophers and legal scholars for the past 30 years: the permissibility of using certain types of (accurate) statistical evidence to form beliefs or make judgments about an individual, or to impose criminal or civil liability. After briefly sketching the background of this debate, I will describe Di Bello and O’Neil’s novel approach, note its acknowledged limitations, then pose some challenges that I hope will be taken up in the discussion.
First, some stage setting: It’s widely recognized that there is something objectionable in relying on evidence that people with property X have a high probability of being or doing Y (a higher probability than the index population) to conclude that individual A, who possesses X, is or has done Y, where X is anything from membership in a racial or ethnic group to performance of some past act to possession of some genetic variant, and Y is something wrongful or criminal. Such reliance is problematic despite the (stipulated) fact that this evidence is relevant: you are more likely to reach a correct belief, judgment, or verdict about whether X is or has done Y if you take that evidence into account than if you don’t. It isn’t the statistical form of the evidence that is problematic, since it is clearly acceptable to take account of some forms of accurate statistical evidence, e.g., the extremely low probability that anyone besides the defendant left blood with the very same DNA sequence at the crime scene. Finally, it is especially problematic to rely exclusively on such statistical evidence, even when there is no other evidence available and no reason to think that there should be.
Rather than venture a general explanation for the reluctance to rely on such statistical evidence, as other commentators have done, Di Bello and O’Neill focus on a specific type of evidence in a specific context. They offer an original and plausible rationale for the exclusion in criminal trials of “profile evidence”:
In its incriminating form, profile evidence expresses a positive, non-accidental statistical correlation between bearing a certain property and committing a type of crime. When the correlation is reliable and the defendant has the property . . . the addition this evidence . . . would increase, sometimes substantially, the probability that the fact finders should assign to the defendant’s guilt (149).
Their examples of profile evidence include the “properties” of belonging to a specific ethnic group in a certain geographic area; of committing a prior burglary; and of coming from a bad environment; in each, a defendant possessing that property is at least a hundred times more likely to commit the type of crime charged than others in the index population. Recognizing that such evidence may also be objectionable because of the danger that it will be given excessive weight, or bias the fact finder against the defendant, they insist that profile evidence would be objectionable even if the fact finder would use it in an entirely rational way.
Di Bello and O’Neill begin by briefly considering and rejecting explanations for excluding profile evidence based on threats to deterrence and affronts to autonomy and individuality, then give their own account, based on the equal protection of innocent defendants. (Since the rival accounts are already the subject of an extensive critical literature, I’ll skip over the authors’ critique.) Their explanation of the resistance to admitting incriminating profile evidence is intended to cover all and only such evidence, when offered against innocent defendants. To cover all such evidence, the explanation must not be limited to profiles based on membership in disadvantaged or stigmatized groups, since profile evidence is objectionable even when it is based on neutral or positive properties. In covering only profile evidence, their account does not seek to explain the resistance to other forms of statistical evidence, notably evidence of an accidental association of an individual with a transient group, like the spectators at a rodeo, a majority of whose members commit a crime, like gatecrashing. (I’ll have more to say later on this limitation.) Their argument is that profile evidence, and only profile evidence, violates a “right to equal protection – that is, a comparative right belonging to individual defendants not to be exposed to a higher risk of mistaken conviction than other innocent defendants facing comparably serious charges” (155).
They first argue for the noncomparative right of innocent defendants to “due protection” against mistaken conviction. The level of protection that’s due is determined by balancing the importance of avoiding mistaken convictions with the costs of protection, so that it varies with the seriousness of the offense. But it also varies with cost, so that less protection is due to defendants who are more costly to protect for any reason, from high fees for language translators to a high rate of recidivism for the charged offense. Because such differences in the protection due to innocent defendants are obviously unjust, it is necessary to recognize a right to equal protection. Innocent defendants facing comparably serious offenses must not be subject to risks of mistaken conviction that a reasonable person would regard as unequal. But since that right must be formulated well in advance of specific cases, those risks “must be assessed from lawmakers incompletely informed ex ante perspectives” (160). Profile evidence violates this right, because it is “possible for lawmakers to distinguish, in advance of particular trials, between those who fall under a profile positively correlated with a type of crime and those who do not” (164). The right does not extend to case-specific inequalities in the risk of mistaken conviction, which cannot be anticipated from the lawmakers’ perspective. So, the sole spectator to pay for admission to a well-attended rodeo will not be protected against evidence that all but one of the spectators were gate-crashers. The authors suggest that this is not an arbitrary difference: any one of us could be subject to such case-specific risks; unlike profile evidence, they do not burden a class of individuals identifiable in advance.
The authors acknowledge that exculpatory profile evidence also violates equal protection, by giving some innocent defendants a lower risk of mistaken conviction than others. But it does so by reducing the risk of mistaken conviction for those defendants without raising it for others – as long as the factfinders are told not to draw an adverse inference from the absence of such evidence. This more benign violation of equal protection may be justified by the importance of admitting exculpatory evidence; due protection may outweigh equal protection.
Finally, the authors argue that no other form of evidence, statistical or otherwise, poses the same threat to equal protection, even if that evidence is more likely to be brought to bear against some innocent defendants than others. From the ex-ante perspective from which equal protection must be assessed, no group of innocent defendants faces a greater risk of mistaken conviction by a rational fact finder – as opposed to mistaken incrimination — from having, e.g., a common physical appearance or DNA profile similar to the perpetrator’s. Such individuals, who may be identifiable in advance, are more likely than others to be mistaken for the perpetrator of a crime. But they do not face an increased risk of mistaken conviction by a rational fact finder – the “rational” is critical — who will discount eyewitness or trace evidence by the increased likelihood of a false positive. In contrast, a rational fact finder will not discount evidence of a reliable non-accidental correlation between a property of the individual and the type of crime with which she is charged.
Equal protection is a distinct and original rationale for the exclusion of profile evidence, and one with some moral resonance. There is intuitive appeal in the idea that no innocent individual should bear the enduring legal handicap of greater vulnerability to mistaken conviction than other innocent individuals. That handicap seems to deny them equality before the law in one significant respect. I have doubts, though, about the moral weight of a right that requires equal protection only from the risk of mistaken conviction, not mistaken incrimination, and only from rational, not actual, fact finders.
To begin with, I’m not sure how morally problematic it is to subject innocent defendants to an unequal risk of mistaken conviction, apart from its adverse impact on disadvantaged or stigmatized social groups. The authors acknowledge that their “examples may seem to trade on the fact that defendants fitting those profiles already tend to suffer from discrimination,” but they argue that profile evidence “arouses the same intuitive resistance” even when it “does not pick out a socially salient category of defendants” (152, n. 16). However, finding profile evidence that doesn’t do so is instructively difficult, as suggested by the authors’ own example of the “Warrior Gene.” Individuals significantly more likely than the rest of the population to engage in violent, aggressive, and antisocial behavior are likely to be seen as violent, aggressive, and antisocial, so that evidence of their genetic profile might well pick out an already-salient category of defendants.” And even groups that have not suffered from disadvantage and discrimination can quickly acquire negative salience. Consider the invidious stereotypes likely to be evoked by evidence that individuals previously convicted of insider trading are 125 times more likely to engage in insider trading than the general population. My point isn’t that these examples undermine the author’s case for an equal protection right; merely that we have reason to question the source of its intuitive appeal.
I also doubt that the author’s distinction between false incrimination and conviction, though important, can bear the weight they assign it. They argue correctly that a rational factfinder would assign much less probative value to an eyewitness identification of a defendant living in a high crime area than one living in a low crime area, assuming eyewitnesses in the two areas would have the same reliability, so that “admitting eyewitness testimony would not subject defendants who are at higher risk of mistaken incrimination to a higher risk of mistaken conviction” (174). Even if we accept the idealization of the fact finder as completely rational, it’s still the case that innocent young men in high crime areas are far more likely to be misidentified by eyewitnesses to the far more frequent crimes in their areas than young men in low crime areas are to be misidentified by eyewitnesses to the far less frequent crimes in their areas. For this reason, eyewitness testimony creates a higher risk of mistaken conviction for the former than the latter — just because it increases their risk of being brought to trial, where they face, even from fair judges and rational fact finders, some risk of mistaken conviction. They may not face a higher risk of conviction than other innocent defendants, but they face a far higher risk of becoming defendants in the first place.
Moreover, as the authors recognize, non-profile evidence will in fact place some innocent defendants at greater risk of mistaken conviction than others, in the hands of the imperfectly rational fact finders they will face. Those fact finders are unlikely to discount for the greater likelihood that defendants from high crime areas will be mistakenly identified by equally reliable eyewitnesses than will defendants from low crime areas. It is cold comfort to the former defendants to know that they would not face an increased risk of mistaken conviction from ideally rationally fact finders. Unlike Di Bello and O’Neil, the actual rules of evidence governing criminal trials, do take account of the tendency of factfinders to judge irrationally; under the Federal Rules of Evidence (FRE 403) “The court may exclude relevant evidence if its probative value is substantially outweighed by a danger of one or more of the following: unfair prejudice . . . .”
I conclude with questions about the specificity of the authors’ account, which is limited to one type of statistical evidence in one context. What implications, if any, does their account have for fact finding in other contexts, legal and non-legal, and for other types of problematic statistical evidence? Does the right to equal protection extend, suitably modified, to civil trials, to other legal and administrative proceedings, or to informal judgments and beliefs? Is it an advantage of competing accounts of profile evidence that they apply to guilty as well as innocent defendants? That they also explain the resistance to other kinds of statistical evidence, like that based on the kind of accidental association found in the gatecrasher case? I look forward to the discussion.