April 23, 2008
Why empirical research is better at raising questions than answers --- some ruminations about ruminations about the Yale clerk study
Guest blogging at Balkinization, John Donohue has this very lengthy post, titled "Why I'd Stick With Yale Clerks -- Some Econometric Ruminations," which takes issue with this new provocative paper based on empirical research concerning Yale law clerks and judicial opinions. Here is the start and end of Professor Donohue's analysis in the post:
Another illustration of empiricism gone astray is provided by a new working paper by Royce de Rohan Barondes, which adopts the following provocative title: "Want Your Opinions Questioned or Reversed? Hire a Yale Clerk." The man bites dog nature of the claim is sure to raise interest in the paper, since Yale is obviously one of the most elite law schools in the U.S., and the hardest to get into. Unfortunately, counterintuitive empirical results almost always turn out to be wrong if they are not based on an appropriate empirical methodology for the inquiry at hand. In my opinion, the methodology of the Barondes is flawed, and the conclusions drawn from this research are either incorrect or unfounded. My review of the Barondes paper (as well as my own personal experience with Yale Law students) affords little reason to believe that the value of a Yale Law clerk is less than the law school’s preeminent ranking would suggest....
In sum, I am confident that a more suitable methodology than the one employed by Barondes would reveal that Yale Law clerks are extraordinarily capable and effective public servants. All judges will likely be pleased to hire them.
The dissection of the Yale clerk study between these two paragraphs is effective at raising a lot of great follow-up questions about the Barondes paper. But, I highlight the start and end of Professor Donohue's analysis because I am really stunned by the initial assertion that there is "little reason to believe that the value of a Yale Law clerk is less than the law school’s preeminent ranking would suggest" and by the ending assertion that he is "confident that a more suitable methodology than the one employed by Barondes would reveal that Yale Law clerks are extraordinarily capable and effective public servants." (Perhaps this ending assertion was written with tongue-in-check, but the post title suggests otherwise.)
It strikes me as very fitting and valuable for one empirically-oriented law professor to question and critique another law professor's empirical research. But, I am troubled that the critic (who is clearly biased by where he teaches) concludes his analysis by asserting with confidence that sounder research would prove the antithesis of what the critiqued study suggests.
Perhaps more important than my critique of the Donohue critique is my broader observation that empirical research and analysis is far more effective at raising important normative questions than at answering even descriptive ones. To focus again on start and end of Professor Donohue's comments, I wonder what judges, professors, practicing lawyers and lay people perceive or believe to be "the value of a law clerk." Similarly, I wonder what judges, professors, practicing lawyers and lay people perceive or believe to be the ways in which young lawyers can and should be "extraordinarily capable and effective public servants." Those are the big questions that neither the Barondes study or the Donohue critique really explores.
April 22, 2008
Tenure, Scholarship, and the Slow, Slow Turn
My home institution is now in the midsts of a tenure controversy (not involving the law school, which had no candidates up for tenure this year). All the usual tenure-controversy events are playing out: overheated meetings with the administration, anguished cries from those denied tenure and their supporters, and a fair amount of befuddlement about how the process worked and is supposed to work.
What surprises me is that these controversies occur so often within the broader academy. You would think that by now the tenure process would be rationalized and normalized to the point where it was no longer seen as something akin to Goya's painting of Saturn devouring his children (depicted here).
In short, it seems to me that there are three groups involved in the process-- the teaching faculty, the deans/department chairs, and the upper administration (provost/president). As long as they all have the same expectations, tenure considerations seem to go relatively smoothly. It is when there is a disjuncture in the expectations of these groups that trouble arises.
One way those expectations get out of whack is when innovation comes from the top down. For example, consider the simplest (and most common) of innovations-- the expectation of more scholarship. If this innovation is decreed by the upper administration, there is a time lag before those expectations can fairly be made a part of the tenure process, or else the rules are being changed as people are approaching tenure decisions. Such patience is difficult, though, when pressure for better numbers are being placed on those administrators by their own bosses, the trustees. This is especially true where the senior faculty and even the deans resist the change, often because the innovation is seen as a rejection of the values or practices that have informed their own careers.
The battleship turns slowly, as too many institutions have learned the hard way.
-- Mark Osler