April 23, 2008
Why empirical research is better at raising questions than answers --- some ruminations about ruminations about the Yale clerk study
Guest blogging at Balkinization, John Donohue has this very lengthy post, titled "Why I'd Stick With Yale Clerks -- Some Econometric Ruminations," which takes issue with this new provocative paper based on empirical research concerning Yale law clerks and judicial opinions. Here is the start and end of Professor Donohue's analysis in the post:
Another illustration of empiricism gone astray is provided by a new working paper by Royce de Rohan Barondes, which adopts the following provocative title: "Want Your Opinions Questioned or Reversed? Hire a Yale Clerk." The man bites dog nature of the claim is sure to raise interest in the paper, since Yale is obviously one of the most elite law schools in the U.S., and the hardest to get into. Unfortunately, counterintuitive empirical results almost always turn out to be wrong if they are not based on an appropriate empirical methodology for the inquiry at hand. In my opinion, the methodology of the Barondes is flawed, and the conclusions drawn from this research are either incorrect or unfounded. My review of the Barondes paper (as well as my own personal experience with Yale Law students) affords little reason to believe that the value of a Yale Law clerk is less than the law school’s preeminent ranking would suggest....
In sum, I am confident that a more suitable methodology than the one employed by Barondes would reveal that Yale Law clerks are extraordinarily capable and effective public servants. All judges will likely be pleased to hire them.
The dissection of the Yale clerk study between these two paragraphs is effective at raising a lot of great follow-up questions about the Barondes paper. But, I highlight the start and end of Professor Donohue's analysis because I am really stunned by the initial assertion that there is "little reason to believe that the value of a Yale Law clerk is less than the law school’s preeminent ranking would suggest" and by the ending assertion that he is "confident that a more suitable methodology than the one employed by Barondes would reveal that Yale Law clerks are extraordinarily capable and effective public servants." (Perhaps this ending assertion was written with tongue-in-check, but the post title suggests otherwise.)
It strikes me as very fitting and valuable for one empirically-oriented law professor to question and critique another law professor's empirical research. But, I am troubled that the critic (who is clearly biased by where he teaches) concludes his analysis by asserting with confidence that sounder research would prove the antithesis of what the critiqued study suggests.
Perhaps more important than my critique of the Donohue critique is my broader observation that empirical research and analysis is far more effective at raising important normative questions than at answering even descriptive ones. To focus again on start and end of Professor Donohue's comments, I wonder what judges, professors, practicing lawyers and lay people perceive or believe to be "the value of a law clerk." Similarly, I wonder what judges, professors, practicing lawyers and lay people perceive or believe to be the ways in which young lawyers can and should be "extraordinarily capable and effective public servants." Those are the big questions that neither the Barondes study or the Donohue critique really explores.
TrackBack URL for this entry:
Listed below are links to weblogs that reference Why empirical research is better at raising questions than answers --- some ruminations about ruminations about the Yale clerk study:
Tracked on Sep 14, 2009 7:24:33 AM
I would not say that "[t]he dissection of the Yale clerk study between these two paragraphs is effective," because what I understand to be the primary thrust of Prof. Donohue's criticism has now been retracted. A review of that criticism now shows large passages have been annotated by Prof. Donohue with overstrike, evidencing their retraction.
Posted by: Royce Barondes | Nov 3, 2008 10:01:32 PM