Judge William Blackstone on the proper orientation toward new ventures:

The general expectation of so numerous and respectable an audience, the novelty, and (I may add) the importance of the duty required from this chair, must unavoidably be productive of great diffidence and apprehensions in him who has the honour to be placed in it. He must be sensible how much will depend upon his conduct in the infancy of a study, which is now first adopted by public academical authority; which has generally been reputed (however unjustly) of a dry and unfruitful nature; and of which the theoretical elementary parts, have hitherto received a very moderate share of cultivation. He cannot but reflect that, if either his plan of instruction be crude and injudicious, or the execution of it lame and superficial, it will cast a damp upon the farther progress of this most useful and most rational branch of learning; and may defeat for a time the public-spirited design of our wise and munificent benefactor. And this he must more especially dread, when he feels by experience how unequal his abilities are (unassisted by preceding examples) to complete, in the manner he could wish, so extensive and arduous a task; since he freely confesses, that his former more private attempts have fallen very short of his own ideas of perfection. And yet the candour he has already experienced, and this last transcendent mark of regard, his present nomination by the free and unanimous suffrage of a great and learned university, (an honour to be ever remembered with the deepest and most affectionate gratitude,) these testimonies of your public judgment must entirely supersede his own, and forbid him to believe himself totally insufficient for the labour at least of this employment. One thing he will venture to hope for, and it certainly shall be his constant aim, by diligence and attention to atone for his other defects: esteeming, that the best return which he can possibly make for your favourable opinion of his capacity, will be his unwearied endeavours in some little degree to deserve it.

On a related note, Blackstone Trial Analytics is now open for utility maximization business.


Should Public Defender Caseloads Matter for Indigent Defense Outcomes?

Unsurprisingly, I believe the answer is yes; surprisingly, I expect heavy caseloads should improve defense outcomes. My reasoning:

  • Public defenders (PDs) behave as if they are not budget constrained (with respect to trial inputs they supply themselves).
  • Heavy caseloads should increase the probability of seeing any given case continued on a given day.
  • The average indigent defendant is more likely than the average wealthy defendant to have a prior criminal record, and the marginal disutility of additional criminal charges is probably strongly diminishing. In other words, prosecutors quickly lose (or often lack) the ability to tempt indigent defendants with plea bargains that offer features like amended charges with better labor market signaling. Indigent defendants have more taste for trials, which are costly to prosecutors.

(1) strongly suggests that PD clients are relatively more expensive to prosecute, especially at general district court levels where things like (costly) expert testimony are less common. (3) also suggests a level effect; when indigent defendants and relatively wealthy defendants have identical case details, the indigent defendants will probably do somewhat better in terms of trial outcomes as a group. (2) implies that as PD caseloads increase, the probability of continuances across the PD portfolio increase. Sentence maximizing prosecutors will know this, and will be induced to offer more favorable pleas or drop charges as needed.

Thoughts on Utah v. Strieff

From the Syllabus:

Narcotics detective Douglas Fackrell conducted surveillance on a South Salt Lake City residence based on an anonymous tip about drug activity. The number of people he observed making brief visits to the house over the course of a week made him suspicious that the occupants were dealing drugs. After observing respondent Edward Strieff leave the residence, Officer Fackrell detained Strieff at a nearby parking lot, identifying himself and asking Strieff what he was doing at the house. He then requested Strieff’s identification and relayed the information to a police dispatcher, who informed him that Strieff had an outstanding arrest warrant for a traffic violation. Officer Fackrell arrested Strieff, searched him, and found methamphetamine and drug paraphernalia. Strieff moved to suppress the evidence, arguing that it was derived from an unlawful investigatory stop. The trial court denied the motion, and the Utah Court of Appeals affirmed. The Utah Supreme Court reversed, however, and ordered the evidence suppressed.

Held: The evidence Officer Fackrell seized incident to Strieff’s arrest is admissible based on an application of the attenuation factors from Brown v. Illinois, 422 U. S. 590.

In previous posts, I’ve argued that judges are expert adjudicators. Just as we should trust the average view of plumbers over our own views on plumbing, or the average view of economists on economics, we should trust the average views of judges on adjudicating. So is the above decision optimal? Actually, I doubt that it is. The Justices on the Supreme Court voted 5-3 against suppression, but if you factor in the opinions of each judge who considered the case in lower courts, the vote appears to be 9-8 in favor of suppression.

Additionally, the majority flexes its consequentialism muscle, arguing (Thomas writing) that the exclusionary rule is socially costly. Paging Dr. Landsburg.

If I Love the Legal System So Much, Why Don’t I Marry It?

As a consultant, my job is simple: increase mean, decrease variance. In order to do this, I need to have some expertise in how the criminal justice system (CJS) works. I think the best way to understand the CJS is to break the system up into critical component parts. Each component part involves interactions between agents, and each agent has goals. Using court data, we can refine and parameterize our agent models until we have a predictive service to offer our clients. Many of the conclusions of this type of analysis are surprising – for example, the finding that judges are better adjudicators than most people believe. I frequently point out that the CJS today doesn’t seem wildly different from the CJS most reformers pine for.

This should not be construed to mean I think the CJS is optimal. Rather, I think judges and prosecutors behave basically like the types of agents you would want in an optimal CJS. The real problem isn’t with the adjudicators, it’s with the arresting authorities and the laws. The arresting authorities have an easy enough fix in principle; we could just tie their pay to their successful prosecution rate (or any derivative of this plan). Would this fix magically give us an optimal CJS? No, but it would likely help fight the pervasive over-arresting of blacks by police. The more fundamental problem is the law generally. The optimal CJS is that which maximizes aggregate utility; by backward induction the optimal set of laws is that which maximizes aggregate utility.

I submit that the current set of laws is nowhere close to the utility maximizing set. While I think many laws are undesirable and should be repealed, the worst feature of our legal regime seems to be the given penalties for breaking most laws. We all know long prison sentences are overrated; the wise Alex Tabarrok prefers this alternative (do read his entire post):

I favor more police on the street to make punishment more quick, clear, and consistent. I would be much happier with more police on the street, however, if that policy was combined with an end to the “war on drugs”, shorter sentences, and an end to brutal post-prison policies that exclude millions of citizens from voting, housing, and jobs.

I suspect such a policy regime would move us unambiguously closer to the optimal regime (i.e. more deterrence and more utility), but it neglects the problem of police behavior. Presumably, the social costs would still be born disproportionately by populations of color.

The Middle Way

Blackstone Trial Analytics hopes to reconcile two reasonable world views: Trial output is highly heterogeneous and trial output is intelligible. This conflict is ably summarized by two partisans in particular on one of my favorite Yahoo! Answers entries:


Thanks for your question. That is a more difficult question than you might expect. A win / loss record really refers to the lawyer’s success rate. However, what is defined as success in any given case will be defined by its particular circumstances. In a criminal defense case, for example, a successful outcome is not always defined as an acquittal. In some cases an acquittal is near impossible. However, that does not mean that the case cannot be dealt with with a “successful” result.

For example, a client with a long criminal record facing another charge where the case against him is overwhelming may define success as any outcome that does not result in jail time — a result that may be very difficult to achieve. Or what about the lawyer who tries a case, receives a terrible legal ruling, resulting in a conviction, and then has that loss later overturned on appeal?

Or what if the jury returns a verdict of not guilty for two counts, but guilty on another count, or even on a lesser included offense?

Although it may not be possible to get an acquittal in every case, the lawyer may be “successful” in helping the client attain his goal. Is such an outcome a win or a loss? In the examples above, both the lawyer and client may consider it a win. What if the lawyer considered it a loss even though he helped his client attain the desired outcome?


You guys are a bunch of f[uc]king A[ss]holes.. The question was simple but you got your emotion all tied up in knots. I’ve got [news for] you B[i]tches. “You are what you record says you are” I don’t care what you think. I want a winner!!! and if that attorney has never lost I want them on my team. Simple concept.

Are Police Better People? Probably Not.

After reviewing the FCGDC data, it seems more likely that disparate black/white CJS outcomes are driven by police rather than judges or prosecutors. It seems the fundamental problem is that blacks are arrested at rates much higher than you would expect based on Fairfax County demographic information. There are a couple of ways to think about this. Perhaps (1) police officers tend to arrest blacks at lower level of confidence than whites with respect to suspected guilt. Perhaps (2) police arrest whites and blacks at comparable confidence levels but they target relatively black neighborhoods. Alternatively, (3) black populations may be associated with more crimes per capita than white populations. Given the relationship between income and criminality, it seems likely that the third option may have some truth – but it seems intuitively unlikely that this can account for the vast disparity between white and black arrest rates.

On the other hand, imagine that police officers maximize arrests at a certain level of confidence. Imagine that they have no bias against any particular group. In a highly stylized setting where a representative police officer can choose between patrolling two neighborhoods, identical in every way, he will be indifferent. Now suppose one neighborhood is somewhat poorer and the average inhabitant of that neighborhood is somewhat more likely to be involved in criminal activity. If this police officer is the ideal social agent (i.e. only cares about enforcing society’s laws), he will focus disproportionately on the poorer neighborhood. Specifically, he will patrol the poorer neighborhood until the marginal benefit of search (in arrests) equals the marginal cost (in time); at that point he will be indifferent between patrolling either neighborhood. In this scenario, the majority of arrests will come from the poorer neighborhood. If we assume the relative population size of the two neighborhoods is large compared to the size of the police force, we should expect a vast majority of arrests to come from the poorer neighborhood. If we add enough assumptions, we can create a scenario where the police behavior in Fairfax County today is “socially optimal” under the dubious meta-assumption that the Code of Virginia is optimally designed.

Can we test to see if reality approximates the story above? Yes. We already know that blacks and whites face comparable trial outcomes when controlling for income, so it’s unlikely that the average black defendant is more guilty than the average white defendant (i.e. the fact bundle against black defendants seems as strong as the bundle against white defendants on average). So what? The important takeaway from this fact is that one group doesn’t seem to commit crimes more conspicuously or hide crimes more adeptly; framed differently, it isn’t easier to search for either black or white criminals, although the density of criminals in one area may be relatively higher than that of another area. This makes reality look somewhat similar to the theoretical world outlined above. We also know that blacks are relatively more likely than whites to have charges against them dropped. This suggests support for police behavior hypothesis (1), police arrest blacks at lower confidence levels than whites. Reviewing the FCGDC police data itself, we find additional support for hypothesis (1) and (2).

With respect to the data, the distribution of average defendant win-rates associated with individual police officers is distributed fairly normally (we only looked at officers with at least 50 trials); but a histogram alone doesn’t tell us too much in this case. Maybe police arrest everyone at 50% confidence levels, and variation in defendant win-rates is more about whether the individual police officer is in the short-run or long-run. This is analogous to flipping a fair coin some number of times; over 50 trials, you may see a relatively large number of heads turn up. Over 1,000,000 trials, the Heads/Tails ratio should settle down at 1:1. Actually, we find that defendant win-rate by police officer doesn’t change much over time. A much more plausible story is that individual police officers arrest defendants at individual confidence levels. We also find substantial differences in black/white defendant win-rates across a number of police officers.

To sum up, even assuming the unexpectedly high level of black arrests is purely a function of crime density in poorer areas:

  • Individual police officers make arrests at dramatically differing levels of confidence.
  • A number of police seem to arrest blacks at lower levels of confidence than whites.

This is about the most charitable view of the arresting authorities one could reasonably give and it doesn’t exactly paint the police in a very flattering light.

Are Judges Better People? Probably.

In my first post, I wrote about data from the Fairfax County General District Court (FCGDC) and how it seems to support the view that courts themselves should not be the primary target of the typical criminal justice reformer. This should not be misconstrued as evidence that courts are models of well-designed institutions that we should copy whenever possible. The FCGDC data provides support for the view that prosecutors are extremely effective at giving adjudicating authorities what they want, and by a stroke of fortune the adjudicating authorities want things like accuracy and procedural discipline instead of things like apartheid. In other words, the FCGDC doesn’t seem to be a particularly robust institution, but nevertheless it happens to work surprisingly well. In this post, I’ll give a few reasons why I think that’s true.

If the story I’ve outline is correct, judges are basically benevolent dictators. This is exceedingly rare in politics; why does it make sense in the courtroom?

From “’Ideology’ or ‘Situation Sense’? An Experimental Investigation of Motivated Reasoning and Professional Judgment” by Dan Kahan, David Hoffman, Danieli Evans, Neal Devins, Eugene Lucci, and Katherine Cheng:

The study involved a sample of sitting judges (n = 253), who, like members of a general public sample (n = 800), were culturally polarized on climate change, marijuana legalization and other contested issues. When the study subjects were assigned to analyze statutory interpretation problems, however, only the responses of the general-public subjects and not those of the judges varied in patterns that reflected the subjects’ cultural values. The responses of a sample of lawyers (n = 217) were also uninfluenced by their cultural values; the responses of a sample of law students (n = 284), in contrast, displayed a level of cultural bias only modestly less pronounced than that observed in the general-public sample.

The key takeaway from this study is that judges are less susceptible to cognitive biases than members of the public. The authors attribute this to legal training, and their results do seem to support this story; law students do worse than judges and members of the public do worse than law students. I wonder how much of the story can be explained by IQ alone (judges have higher IQs than law students, law students have higher IQs than the general population); it would be interesting to see the study redone testing additional high-IQ groups without formal legal training (sociologists, engineers, economists, etc.). At any rate, the results are interesting.

Obviously the study results are obtained by judges responding to an anonymous, low-stakes survey. But the results from the FCGDC dataset support that judges abstain from motivated reasoning even in relatively high-stakes trials. Given that few formal institutions constrain judge behavior (in practice they are basically free to adjudicate however they want), what can account for a judge’s style? Probably the same things that prevent people from producing low-quality work in general – it’s embarrassing, for one. Economists often note that the bureaucracy, for example, works better than you would expect assuming all of its agents are highly self-interested. The same seems to apply to judges.