March 15, 2007
SSRN rankings and Leiter's (rank?) omission
Aided by this post at Feminist Law Professors by Ann Bartow, I see a possible controversy brewing over Brian Leiter's decision, in his most recent SSRN download calculations, to exclude my Ohio State colleague Chris Fairman's heavily downloaded paper, Fuck. Ann's post assails the decision, and my colleague Ruth Colker sent Leiter this e-mail (which she authorized my posting):
Brian, I was shocked that, after you acknowledged the bias in favor of individuals who write in the corporate law area with respect to download counts, you then excluded from your download list the only piece that is on a civil rights related topic, and included all the corporate pieces. I have read and learned significantly from Chris Fairman's piece entitled Fuck, which will soon be published in the Cardozo Law Review. A criteria-less exclusion of one article is empirical scholarship at its worst. Ironically, a reader of Chris' piece would have expected your actions, because law and society often overreact to the mere use of the term "fuck" without consideration of its meaning in a particular context. I encourage you to reverse your actions and republish the list without implicitly editorializing about whether one article "deserves" its downloads.
Joyfully, Chris Fairman has himself jumped into the fray with an intriguing essay discussing this situation available here (via SSRN, of course). Chris' new piece is entitled "Fuck and Law Faculty Rankings." Because sequels rarely do better than the original, I doubt that this new fuck-related meta-scholarship will perform as well as Chris' first piece. But perhaps it will still help folks figure out what the fuck to make of SSRN download counts.
Posted by DAB
UPDATE: In this post entitled Quantifying Scholarship, Neil Buchanan weighs in on the problems of relying on cite counts and/or SSRN downloads in law faculty rankings. Especially notable is this (unsurprising?) report Neil provides in the comment section to his post:
[A]n acquaintance from another law school called me recently and asked if I would please download all of his papers from SSRN. His dean is making a big push to pump up their numbers.
TrackBack URL for this entry:
Listed below are links to weblogs that reference SSRN rankings and Leiter's (rank?) omission:
I'm not sure Leiter's right here, but I'm not sure he's wrong, either. Do people really think that Leiter excluded the article because of the salacious title? Or was it excluded because it garnered such an absurdly high number of downloads that it skewed the rankings?
In the latter case, it's at least a debatable point, and is really just one of study methodology, about which empirical researchers have debates all the time. (Obviously, if the former is the case, then it's more troubling. But I'm willing to take a charitable position toward Leiter right now.)
Posted by: Jason | Mar 15, 2007 9:49:48 PM
A simple way to solve Brian's problem without the arbitrariness of Brian's solution is to exclude, say, the three most-downloaded members of each faculty. This gets rid of Chris, Brian, and me, among others. It also gets rid of most of the corporate finance stuff and most major bloggers (who seem inevitably to get higher downloads). I've reranked the Top 50 Law Schools by SSRN Downloads in a posting on Moneylaw at http://money-law.blogspot.com/2007/03/top-50-law-schools-by-ssrn-downloads.html. The resulting ranking, I suggest, is a measure of the scholarly depth of the relevant faculties. It avoids many of the common criticisms of SSRN-based indices.
Posted by: Theodore Seto | Mar 16, 2007 1:04:03 PM
Theodore, my chief citicisms of SSRN-based indices is that not everyone places all (or even most) of their scholarly work on SSRN or only on SSRN.
I think the SSRN indices measure who spends time on SSRN, not who is producing the most scholarship. This is even more turn now that many SSRN-oriented folks are posting lotsof old papers. I don't think your tweaks address this chief concern at all.
Posted by: Doug B. | Mar 16, 2007 3:17:14 PM
Maybe articles like this are the reason judges don't read or cite law review articles.
Posted by: former law student | Mar 19, 2007 11:37:53 AM
Download counts are a poor measure of the initial popularity of scholarship, which is a poor measure of scholarly quality, significance, and impact. Worse the various measures that SSRN uses to assure the "quality" of its download counts impair open access to scholarship. I've written about the issue, and several similar ones, in a brief essay entitled "SSRN Considered Harmful." Copies are available at BEPress and at SSRN. Interestingly, the version posted at SSRN is not considered part SSRN's eLibrary (it is instead "privately available), because SSRN does not consider it to be "scholarly." I wonder whether SSRN will consider Fairman's piece about download rankings to be "scholarly."
Posted by: James Grimmelmann | Mar 19, 2007 8:26:48 PM