Law School Innovation

"Professor's plea: Say no to 'law school porn'"

The title of this post is the headline of this new piece appearing in The National Law Journal. Here are excerpts:

It's that time of year when law school faculties are inundated with so-called "law school porn" — slick mailings extolling the virtues of individual law schools meant to sway voting in the U.S. News & World Report's reputation survey, now underway.

Some legal educators believe the annual barrage of mail has gotten out of control, and proves that rankings are driving administrative decisions.  They say it's time to stop paying for glossy brochures and invest that money in students.

"Some of the stuff I get is gorgeous," said University of New Hampshire law professor Sarah Redfield. "It's almost a book. Some people are spending a bunch of money on this."

A study released in 2009 that was partially funded by the Law School Admission Council, "Fear of Failing: The Effect of U.S. News & World Report Rankings on U.S. Law Schools," reached the same conclusion: that administrators are spending significant amounts of money on brochures and marketing materials in hopes of getting better results on the reputation survey.  The survey is based on voting by legal educators, lawyers and judges, and accounts for 40 percent of a school's ranking score.

In a recent blog post, University of California at Los Angeles School of Law professor Stephen Bainbridge estimated that this material — commonly referred to in legal academic circles as "law school porn" — comprises 67 percent of his work mail. He added that never reads it. He noted that he has started to receive law school promotional materials via e-mail, which he dismissed as spam.

This material does serve a few purposes, according to University of Alabama School of Law professor Paul Horwitz, who defended them on the PrawfsBlawg blog. They can provide useful information about as recent faculty hires, scholarly publications and other innovations, he wrote.

"On the whole, unlike many, I would rather receive these materials than not receive them," Horwitz wrote. "That's true even if, as is generally the case, they're ridiculously fulsome, as long as they're also informative. As long as a school wants to tell me more about who it's hired and what its folks are writing, I'll be happy to read its mailers."...

Redfield brought a thick stack of the material to a law school admissions conference at St. John's University School of Law on Nov. 11.  It represented about one quarter of what she had received this fall, she said. She theatrically dropped the stack into a recycling bin, producing a loud thud, and issued a challenge to the law deans in the audience and to U.S. News Director of Data Research Bob Morse, who sat on a panel with her.  Law schools should do away with law school porn and put the money toward diversity scholarships, Redfield said.

Morse did not sign on to the challenge, nor did his dismiss it out of hand.  Redfield's idea was met with skepticism by St. John's Dean Michael Simons.  He did not specify what the school spends on its mailings, but stipulated that it would not be enough to fund even a half-scholarship. The National Law Journal contacted a number of law schools to ask what they spend; none responded.

Willing yet again to consider putting my US News vote up for sale...

Long-time readers may recall this post from four years ago, in which I asked "Would it be unethical (or even illegal) to put my US News vote up for sale?"  Here is the back-story which prompted my (sincere? tongue-in-cheek?) inquiry in 2007:

For the second consecutive year, I have received US News' survey asking me to help identify "law schools having the top programs in intellectual property law."  My receipt of this survey highlights just how flawed some aspects of US News' rankings can be. 

I was an IP litigator a decade ago and I taught a few IP courses early in my career.  But, especially with my primary field so active, I cannot even hope to keep up with all the IP doings in law schools.  Nevertheless, US News seeks my opinion on which 15 schools have "the highest-quality intellectual property law courses or programs."  Candidly, I have absolutely no idea.

I suppose I could try to make educated guesses about the best IP programs based on who sent around the hottest "law porn"covering IP topics this year.  But I also could throw darts at the survey form and probably not do much worse. 

Consequently, I am now wondering if I could and should simply offer my US News survey to the highest bidder.  Helpfully, US News promises that survey responses are kept confidential, so nobody would know whether or to whom I sold my vote.

Of course, I do not want to do anything unethical or illegal, so I am genuinely seeking an answer to the question posed in this post's title.  I know vote selling in some contexts can be illegal, but I don't think a survey by a private magazine garners too much public protection.  As for ethics, well, what I am proposing seems no less savory than what some schools have reportedly done to game the US News rankings system.  Plus, some recent research suggests that open vote buying/selling may be efficient in this kind of setting.

Fast forward four years, and I am now in my office holding this year's version of the US News' survey asking me to again help identify "law schools having the top programs in intellectual property law."   Apparently the fact that I previously talked up the notion of selling my US News' vote to the highest bidder did not get me scratched from the list of potential voters.  (I assume that the people who run the US News' survey never got wind of my talk of vote selling.  But it is fun to imagine that they heard of my (joking?) plans and nevertheless still thought I was a good person to ask about the top 15 IP programs.)

If memory serves, nobody actually offered me any money for my US News vote back in 2007, so I never did have an opportunity to consider seriously whether I could be bought off.  I do recall a few folks responding to my post via e-mail with information about how great the IP program was at their school.  In other words, I most certainly did not get rich from, but I did get some useful information in response to, my prior post. 

Back in 2007, I ultimately concluded that too much personal bias and not enough valid information would end up informing my survey responses, and so I ripped up the form and tossed it away.  Disturbingly, this year's cover letter and US News form fails to suggest trashing the survey if one lacks the knowledge or information needed to fill it out appropriately.  That very fact has me wondering if US News actually would prefer me to complete the form after getting paid for my votes rather than fail to return the form at all.

"Measurement and Its Discontents" ... and US News rankings and law school grades

The title of this post starts with the headline of this interesting commentary published in yesterday's New York Times. Though not saying one word about legal education or law schools, I thought many parts of piece (and especially the passages quoted below) were especially interesting and deserved consideration as we head into the (never-ending) law school ranking season:

Why are we still stymied when trying to measure intelligence, schools, welfare and happiness?

The problem is not that we don’t yet have precise enough tools for measuring such things; it’s that there are two wholly different ways of measuring.

In one kind of measuring, we find how big or small a thing is using a scale, beginning point and unit. Something is x feet long, weighs y pounds or takes z seconds. We can call this “ontic” measuring, after the word philosophers apply to existing objects or properties.

But there’s another way of measuring that does not involve placing something alongside a stick or on a scale. This is the kind of measurement that Plato described as “fitting.” This involves less an act than an experience: we sense that things don’t “measure up” to what they could be. This is the kind of measuring that good examples invite. Aristotle, for instance, called the truly moral person a “measure,” because our encounters with such a person show us our shortcomings. We might call this “ontological” measuring, after the word philosophers use to describe how something exists.

The distinction between the two ways of measuring is often overlooked, sometimes with disastrous results. In his book “The Mismeasure of Man,” Stephen Jay Gould recounted the costs, both to society and to human knowledge, of the misguided attempt to measure human intelligence with a single quantity like I.Q. or brain size.  Intelligence is fundamentally misapprehended when seen as an isolatable entity rather than a complex ideal.  So too is teaching ability when measured solely by student test scores.

Confusing the two ways of measuring seems to be a characteristic of modern life.  As the modern world has perfected its ontic measures, our ability to measure ourselves ontologically seems to have diminished. We look away from what we are measuring, and why we are measuring, and fixate on the measuring itself.  We are tempted to seek all meaning in ontic measuring — and it’s no surprise that this ultimately leaves us disappointed and frustrated, drowned in carefully calibrated details....

But how are we supposed to measure how wise or prudent we are in choosing the instruments of measurement and interpreting the findings? Modern literature is full of references to the dehumanizing side of measurement, as exemplified by the character Thomas Gradgrind in Dickens’s “Hard Times,” a dry rational character who is “ready to weigh and measure any parcel of human nature, and tell you exactly what it comes to,” yet loses track of his own life.

How can we keep an eye on the difference between ontic and ontological measurement, and prevent the one from interfering with the other?

One way is to ask ourselves what is missing from our measurements.... In our increasingly quantified world, we have to determine precisely where and how our measurements fail to deliver.

I suspect many who read this blog would be quick to assert that law schools are "fundamentally misapprehended when seen as an isolatable entity [subject to ranking by US News] rather than a complex ideal."  In addition, lots has been said by many US News critics about what is missing from the US News measurements. 

And yet, I cannot help but wonder how much the entire traditional law school model — and especially traditional law school grading systems — also are subject to the problem of mixing "ontic and ontological measurement."  I have long thought and feared that law schools could hardly complain all that much about being peculiarly graded by US News when these institutions continue to graded the potential of future lawyers in a (more?) peculiar way.

Posted by DAB

Should you go far away to a higher ranked law school?

Above The Law makes the common claim:

“In most situations, going to the highest ranked U.S. News school that you can is going to be really important for your career.”

It's not clear to me that that is truly the case.  Does it make sense for someone in California to journey across the country to attend a law school that is a few ticks up on the US News ranking?  My suspicion is that, with the exception of the truly national schools--something akin to the top ten or so law schools in the country--most law schools are ultimately regional. That is, I suspect that their graduates generally end up working at firms in the same state--or in states adjacent to--the state of the law school they attend.

Ted Seto has done some important empirical work demonstrating this. In his paper, Where Do Partners Come From?, he argues that one should often choose a law school located in the geographic area in which one hopes to work.

As a professor, I often talk with applicants about how to realize their life goals. I recall in particular a student attempting to choose between Vanderbilt and the school at which I teach – Loyola Los Angeles. His ambition was to become a big-firm partner in Los Angeles. As students often do, he chose the higher U.S. News-ranked school. When he graduated from Vanderbilt, he was unable even to get an interview in LA. Had he attended Loyola, his paper credentials and performance at Vanderbilt suggest that he would have graduated near the top of his class. If he had, his chances of getting a Los Angeles big-firm offer would have been quite high. Again, based on the results of the study reported in this article, I can

Those deciding between law schools might do well to examine the list of firms that actually come calling to that law school's recruitment week.  For the most part, it is uneconomical for a firm to send partners to interview candidates in distant jurisdictions, because few students from those distant locations may be inclined to move to that firm's city. Staying within the state is likely to prove more efficient, in terms of partner time.

Anupam Chander

Rankings leakage and other buzz-worthy stuff around the blogosphere

The US News law school rankings are officially due out later this week, but there is already much buzzing in the blogosphere about the leakage of the (perhaps official) results.  Here is just a sample of all the blog talk on the rankings from various different law professor blogs:

• Early Release of 2010 U.S. Law School Rankings
• Extensive 2010 US News Law Rankings
• 2009 Reputational Scores from U.S. News Surveys of Academics and Practitioners
• U.S. News and "assessments"

I am, of course, eager to have comments about all the comments about rankings here.  But I would also like to recommend other law school blog reading that, in my opinion, deserves more attention from law professors than the latest round of the usual ranking wrangling:

• The Worthlessness of American Legal Education
• Pre-Law Courses Would Help

Amazing law school insights from an innovative source

I was at first amused and then amazed by the terrific law-school-selection advice to be found in this great column by Clay Travis at CBS Sports.  Though the first 7 points are cheeky, the final 10 are remarkably insightful about so many aspects of law school.  Here are extended snippets from a must-read piece for everyone around a law school or aspiring to be around a law school:

Judging by the e-mails to ClayNation the overlap between lawyers, aspiring lawyers, and sports fans is substantial.... So without further ado, here's ClayNation advice on how to pick a law school.

1. Visit the school when there's good weather, if at least half of the guys aren't wearing shorts, flip-flops, and t-shirts, then you don't want to go there.  Law school should be fun because being a lawyer isn't fun....

2. Don't be completely seduced by law school rankings....

4. Go sit in the law school library for a half-hour. Pretend to read a newspaper and check to see how often the students smile or laugh when interacting with each other. If no one ever smiles or laughs it's a horrible sign.

5. Think about the size of the law school....

8. Assuming the law school is above 80 percent, comparing bar passage rates tells you nothing about the quality of a law school. Don't be moved by the trumpeting of these stats. By the time you're studying for the bar exam you realize that most intelligent people could spend three months studying the Barbri course outlines, memorizing the absurd MBE fact patterns, and pass the bar exam. But by that time you've spent three years learning how to be a lawyer. Congratulations. Once you've practiced law for a couple of years you wouldn't be able to pass the bar exam. It's a great system.

9. The better the school you attend the more it costs but the less hard you have to work while you're there....

10. If you destroy the LSAT and have a good GPA you may have a decision to make regarding whether to take a scholarship to a lesser school or pay more to go to a more selective school.  It's hard to give advice in this situation because no matter what you think now, you have no real idea whether you'll actually like practicing law....

15. Where should I go to make the most money?  If money is your ultimate goal you shouldn't be a lawyer. There are thousands of ways you can make much more money. Plaintiff's lawyers notwithstanding, as a lawyer you're ultimately hamstrung by how many billable hours you can crank out. And every hour you bill is one less hour you get to have a life. Be careful chasing those big firm golden handcuffs, be careful.

16. Keep in mind that the law is completely unbalanced when it comes to career search.  The most competitive jobs in the legal profession are either the highest-paying or the lowest paying....

And if you're miserable [in law school] keep in mind that after six months of practicing there's a 100 percent chance you'd rather be a law student again. Law school is one of the few places on earth where students compete to one day be able to go back to law school. If you doubt this pick up the resumes of your favorite professors who are under 45. Look at how few years they actually spent practicing law. Yep, in the circle of law-school life, as soon as you leave, you start working on finding a way to get back.

Isn't bar passage a terrible law school ranking metric?

Responding to Mark's post on US News rankings, co-blogger Anupam comments that a useful ranking metric "might be Bar Passage, adjusted to reflect the jurisdiction's overall bar passage rate."  I could not disagree more, in part because I think bar passage is a very harmful aspect of the US News ranking system.  Let me explain:

Bar passage rates tell us what percentage of a law school class has passed the — silly? culturally biased? poorly graded? — timed high-pressured test that many jurisdictions use as one barrier to being a licensed lawyer.  I have long been troubled by bar exams for lots of reasons (too numerous to detail here), and I am especially troubled that US News gives these exams extra legitimacy through its ranking criteria.  Let me (too) quickly explain my anti-bar bias:

1.  I do not think the sole or chief goal of law schools is to help a student pass the timed high-pressured bar exam.  Notably, major law schools clearly don't think this should be their sole or chief goal: if they did, law school classes would look and sound and operate much more like Bar-Bri classes.

2.  Because law school is obviously about a lot more than bar passage, every rational law student (with sufficient resources) takes a bar prep course.  Consequently, it seems fair to assume that bar passage rates reflect the quality of a bar prep course more than the quality of law school instruction. 

3.  Bar passage rates also, obviously, reflect the quality of the student body that a law school admits.  But US News and other rankings already use a variety of other metrics to directly assess/reward the quality of the student body that a school admits.

4.  In my view, students and faculty at most schools — at least those outside the top 10 — already obsess way too much about bar passage rates (in part because US News has used this as a metric).  I do not want there to be even more energy focused on a timed high-pressured test that seems, in my view, to be pernicious in many ways.

Of course, I may be wailing on Anupam's comment principally because I have long wanted to wail on the craziness I see in bar exam realities.  So, because I realize I may be blinded by my anti-bar biases, I would like to hear Anupam and others explain why bar passage might be a useful and valuable law school ranking metric.

A new law school ranking system ... and no blog metric?!?!?!

According to this article online at Insider Higer Ed, The Green Bag has a new law school ranking system in the works.  Here is how the article describes the plan:

The Green Bag ... announces in an editorial in its forthcoming issue [that] this spring, it will begin work on the “Deadwood Report,” which it envisions being an annual assessment of “whether faculty members do the work that the law schools say they do.”  The journal acknowledges that the ranking will provide “rough and admittedly partial” measures of law school faculty quality, but posits that by being transparent (it will disclose the sources of its data and how it derives its numbers and rankings from those data), and by bringing more information into public view, “it will help law school applicants make better decisions about where to study or work.... We are trying to do some good here.”

The Green Bag editorial kicking this all off is a very interesting read, though I am not sure it (yet) provides the supposedly transparent reality it aspires to create.  Nevertheless, anyone with a web-based inlking will be interested to see this aspect of the proposed methodology:

An up-to-date web site is a wonderful thing.  That is where we will gather all of our information.  This seems reasonable to us because your web site is surely where most applicants and other inquisitive people go for information about your law school.  If a school cannot be bothered to provide accurate information about the teaching, scholarship, and service of its own faculty on its own web site, it deserves to be haunted by any inaccuracies.

This is no indication, however, that law faculty blogs or blogging activities will be a significant part of the endeavor, even though faculty blogging and/or other types of faculty web dissemination would seem to be a valid and significant way to assess and measure whether faculty do what law schools say they do.  Needless to say, blogging issues aside, this Deadwood Report is a project worth watching for all those interested in the law school universe.

Posted by DAB

Understanding US News Law School Rankings

Link: SSRN-Understanding the U.S. News Law School Rankings by Theodore Seto. Loyola (Los Angeles) prof Theodore Seto has written an important study of the US News methodology, which should raise many cautions among those who rely upon the study.

This Article explores in detail the U.S. News & World Report law school rankings. Its descriptions, analyses, and conclusions are based primarily on U.S. News' published descriptions of its 2006 computations, telephone conversations with U.S. News' staff clarifying those descriptions, and a spreadsheet I have written that approximately replicates those computations. The Article's goals are relatively modest: to help prospective students, employers, and other law school stakeholders read the U.S. News rankings more critically and to help law school administrators get a better handle on how to manage their schools' rankings. In addition, the Article suggests ways in which U.S. News methodology might be improved. It does not, however, purport to offer a systematic critique of either the U.S. News rankings or ranking in general.

Part I describes both U.S. News' 2006 methodology and problems involved in replicating it. Part II is intended to help prospective students, employers, and other law school stakeholders read U.S. News' results intelligently. Prospective students and others trying to understand how to use U.S. News' rankings in their decision-making may wish to focus on this part, although a reading of Part I may also be necessary to understand some of the technical details. Part III addresses the problem of managing rankings. Part IV, finally, suggests ways in which the rankings might be improved.

Anupam Chander

Stanford Law Students Rank Law Firm Diversity: Will Clients Notice?

A bunch of law students at Stanford have started assigning letter grades to their prospective employers...

The students are handing out “diversity report cards” to the big law firms, ranking them by how many female, minority and gay lawyers they have.

“Many of the firms have atrocious, appalling records on diversity,” said Michele Landis Dauber, a law professor at Stanford and the adviser for the project, called Building a Better Legal Profession. The rankings are at www.betterlegalprofession.org.

In New York, Cleary Gottlieb Steen & Hamilton got the top grade, an A-minus. At Cleary, the project says, 48.8 percent of the associates are women, 8.7 percent are black, 8.3 percent are Hispanic and 4.5 percent are openly gay.

Herrick, Feinstein, by contrast, got an F. Its numbers: 37.7 percent women, 4.9 percent black, 1.6 percent Hispanics, and no openly gay people.

... The numbers were provided to a central clearinghouse by the firms themselves. “Our process is simple,” the student group said in explaining its methodology. “Cut, paste and rank.”

Firms in the top fifth received A’s, in the second fifth B’s, and so on. Overall grades were arrived at by averaging grades for partners and associates in five categories: women, blacks, Hispanics, Asians and gay people.

The diversity breakdowns have been available to students for at least 15 years, through the NALP guide, which carefully catalogs the number of associates and partners by gender and ethnicity.

I suspect that minorities and women contemplating offers have long scrutinized the NALP reports, especially to see whether there is any viable partnership track for them, as indicated by history.

The Stanford Law Students leading this project have simply offered one kind of aggregation of those detailed numbers, simplifying the analysis for students. This strikes me as a valuable service. Firms that lag seriously in hiring women and minorities are likely to be unattractive for many.

One interesting question is whether the Better Legal Profession grades will become popular among clients. My own anecdotally-derived view is that businesses are ahead of law firms in diversity (is there a law firm equivalent of McKinsey's Rajat Kumar Gupta, Stan O'Neal, or Meg Whitman?), and thus they may be concerned when one of their major vendors seems indifferent to it. They are perhaps less likely to have used the NALP resource previously. But they may become concerned that their law firm receives low marks for diversity.

Cleary, Gottlieb, my old firm, has long sought to improve the diversity of its lawyers, driven in part by enlightened leadership and in part by the fact that a large part of its clientele is foreign.

Anupam Chander

Would it be unethical (or even illegal) to put my US News vote up for sale?

For the second consecutive year, I have received US News' survey asking me to help identify "law schools having the top programs in intellectual property law."  My receipt of this survey highlights just how flawed some aspects of US News' rankings can be. 

I was an IP litigator a decade ago and I taught a few IP courses early in my career.  But, especially with my primary field so active, I cannot even hope to keep up with all the IP doings in law schools.  Nevertheless, US News seeks my opinion on which 15 schools have "the highest-quality intellectual property law courses or programs."  Candidly, I have absolutely no idea.

I suppose I could try to make educated guesses about the best IP programs based on who sent around the hottest "law porn" covering IP topics this year.  But I also could throw darts at the survey form and probably not do much worse. 

Consequently, I am now wondering if I could and should simply offer my US News survey to the highest bidder.  Helpfully, US News promises that survey responses are kept confidential, so nobody would know whether or to whom I sold my vote.

Of course, I do not want to do anything unethical or illegal, so I am genuinely seeking an answer to the question posed in this post's title.  I know vote selling in some contexts can be illegal, but I don't think a survey by a private magazine garners too much public protection.  As for ethics, well, what I am proposing seems no less savory than what some schools have reportedly done to game the US News rankings system.  Plus, some recent research suggests that open vote buying/selling may be efficient in this kind of setting.

Posted by DAB

P.S.  Though I started this post in jest, the free-market libertarian lurking inside me is really getting jazzed about this US News vote-selling idea.  Perhaps some law-and-econ folks need to talk some sense into me.

Innovation and Law School Rankings

Doug asks in an earlier post whether it might be useful to rank schools based on innovation. I suspect that the principal uses of law school rankings are (1) to signal quality of the education to students; and (2) to signal quality of students to employers. An innovation-meter would not have a significant relationship to either, I think. Schools that innovate may not necessarily be great in teaching--some of the best teachers at UC Davis have been teaching using the same methods that they've used for decades. Yes, innovation can improve teaching, student experience, employment prospects, and scholarship--but it isn't the best indicator for the quality of the education offered, or for the quality of the student. Take two schools, Blackacre Law School and Whiteacre Law School, both exactly equal in T0. If Blackacre innovates--say by offering an array of teaching styles other than the traditional Socratic method, offering allowing its students to fulfill a substantial percentage of requirements by self-directed study or non-law related internship--does that make Blackacre the better school for either students or employers? This blog, of course, does not promote innovation-for-innovation's-sake, but innovation likely to improve legal education and scholarship, so perhaps my concern could be met by any innovation metric somehow accounting for the quality of the innovation. (Congrats, Doug, on the incisive, top of the fold, NY Times front page quote on the Libby commutation, by the way!) Anupam Chander

A lot of new buzz about rankings, though little talk of innovation

Perhaps in part because the summer is the time for admitted law students to decide where they are going, there lately has been lots of new buzz about law school rankings.  For example, the Wall Street Journal has this new article, entitled "Law Schools Also Ranked By Blogs Now," and the WSJ Law Blog has this follow-up post that has generated interesting comments.  Meanwhile, over at the National Law Journal there is this new article, entitled "Law Schools Unlikely to Boycott Magazine Rankings: Fallout from liberal arts boycott of 'U.S. News & World Report' survey minimal."  Disappointingly, I see little discussion of innovation in this coverage of rankings.

Perhaps this blog should start its own ranking of the most innovative law school.  I think my own school  has a number of innovative programs, though I am not sure how exactly I would develop a metric for ranking law school innovations.  Readers are highly encouraged in the comments to (1) suggest ways to assess/measure law school innovation, and (2) nominate particular school for top innovators.

Posted by DAB

How Do U.S. News Rankings Affect the Financial Resources of Public Colleges?

A recent study's findings suggest that "in addition to a consumer response, the publication of quality rankings may influence the provision of quality through a political channel." Details at Law Librarian Blog. -- Joe Hodnicki

Thoughts on US News rankings?

This year's US News rankings are out, and they are already being discussed broadly around the blogosphere as detailed here and here and here.  Commentary, rants and even praise(?) concerning the latest rankings are encouraged in the comments.

I'll start with a sports observation based on my own cynicism of the US News methodology.  I have long thought that success of a school's sports teams might raise US News rankings because of the positive buzz and name recognition that comes with football and basketball success.  But, providing anecdotal disproof of this hypothesis, the two schools that have dominated the college sports headlines this year, Florida and my own Ohio State, fared very differently in this year's rankings.   

Posted by DAB

Debating SSRN downloads and exclusions

Though Brian Leiter call himself a "skeptic about SSRN downloads," he created and posted here two lists of SSRN download rankings under the heading "Most Downloaded Law Faculties, 2006."  A fully accurate heading for Brian's rankings would be "Most Downlaoded Law Faculties, 2006 (excluding one outlier)" because Brian decided and explained that it "was necessary to exclude Ohio State and Emory, whose presence in the top 15 was due entirely to one provocatively titled article by Christopher Fairman."

Professors Ann Bartow at Feminist Law Professors and Neil Buchanan at Dorf on Law and Ruth Colker here at LSI criticized Brian's decision to exclude one paper from his rankings.  Ruth's expressed criticisms, which was sent via e-mail to Brian, begat an interesting e-mail discussion with Brian and others (on which I was cc:d).  I learned a great deal from the series of back-and-forth e-mails and I asked the group if I could post parts of the exchange on LSI, but not everyone was interested in a full public airing of the e-mail dialogue that Ruth started.

Interestingly, Brian has now discussed aspects of this exchange in this new post that appears to be his defense of excluding one paper from his rankings.  Brian's new post suggests that he believes that SSRN download rankings would be transformed from "a pretty weak measure of scholarly performance" to "just a joke" had he not excluded Chris's paper from his rankings.

As I explained on my home blog and as Neil lamented at Dorf on Law, there is a real risk that undue emphasis on SSRN downloads — which already seems to be a reality as evidenced by blog buzz here and here — could significantly distort how law professors produce, disseminate and market their scholarship.  I am thus especially interested how folks are responding to Brian's SSRN rankings and to his exclusion decision.

Any further thoughts on SSRN download rankings or on Brian's defense of his rankings?  (Comments need to be here because Brian does not enable comments on his blog.)

Posted by DAB

SSRN rankings and Leiter's (rank?) omission

Aided by this post at Feminist Law Professors by Ann Bartow, I see a possible controversy brewing over Brian Leiter's decision, in his most recent SSRN download calculations, to exclude my Ohio State colleague Chris Fairman's heavily downloaded paper, Fuck.  Ann's post assails the decision, and my colleague Ruth Colker sent Leiter this e-mail (which she authorized my posting):

Brian, I was shocked that, after you acknowledged the bias in favor of individuals who write in the corporate law area with respect to download counts, you then excluded from your download list the only piece that is on a civil rights related topic, and included all the corporate pieces. I have read and learned significantly from Chris Fairman's piece entitled Fuck, which will soon be published in the Cardozo Law Review.  A criteria-less exclusion of one article is empirical scholarship at its worst. Ironically, a reader of Chris' piece would have expected your actions, because law and society often overreact to the mere use of the term "fuck" without consideration of its meaning in a particular context.  I encourage you to reverse your actions and republish the list without implicitly editorializing about whether one article "deserves" its downloads.

Joyfully, Chris Fairman has himself jumped into the fray with an intriguing essay discussing this situation available here (via SSRN, of course).  Chris' new piece is entitled "Fuck and Law Faculty Rankings."  Because sequels rarely do better than the original, I doubt that this new fuck-related meta-scholarship will perform as well as Chris' first piece.  But perhaps it will still help folks figure out what the fuck to make of SSRN download counts.

Posted by DAB

UPDATE: In this post entitled Quantifying Scholarship, Neil Buchanan weighs in on the problems of relying on cite counts and/or SSRN downloads in law faculty rankings.  Especially notable is this (unsurprising?) report Neil provides in the comment section to his post: 

[A]n acquaintance from another law school called me recently and asked if I would please download all of his papers from SSRN.  His dean is making a big push to pump up their numbers.

Posner on the Economics of College and University Rankings

Asked:

The rankings raise several interesting economic questions: the effect of rankings on information costs, in general and with particular reference to higher education; the manipulability of rankings by the colleges themselves; the effect of the rankings on education; and why U.S. News & World Report's annual rankings, though fiercely criticized by prominent universities (such as Stanford), face little competition.

And answered by Judge Posner on The Becker-Posner Blog. -- Joe Hodnicki

How Do U.S. News Rankings Affect the Financial Resources of Public Colleges?

In The Power of Information: How Do U.S. News Rankings Affect the Financial Resources of Public Colleges? (NBER Working Paper No. 12941) Ginger Zhe Jin and Alex Whalley reach three main conclusions:

1. USNWR coverage causes colleges to increase educational and general expenditures per student.
2. These expenditure responses are funded by a 6.5% increase in state appropriations per student, but tuition revenue does not respond.
3. The state appropriation response to USNWR exposure is larger the larger the pre-college age population, voter turnout and USNWR newsstand sales are in a state.

The authors write "our results suggest that, in addition to a consumer response, the publication of quality rankings may influence the provision of quality through a political channel."  -- Joe Hodnicki

Are the U.S. News rankings a force for or against innovation?

Back in November, Doug asked essentially this question, and didn't get much response. Perhaps we should have offered "free coffie." However, as April approaches, law deans and profs around the country no doubt are feeling somewhat more anxious about rankings, even without a big cuppa coffie. There is probably nothing more important to the image of many schools than their ranking in the U.S. News survey each year, which will be released about five or six weeks from now.

Doug's observation was that the rankings probably spur innovation in lower-ranked schools (as they try to improve) and discourage them in higher-ranked ones (as they stay with a formula that got them there). Since he made that observation, however, at least two institutions (Harvard and Stanford) have announced curriculum changes that were described as major shifts in emphasis.

In the end, though, my own suspicion is that the rankings very generally suppress innovation at both high and low-ranked schools. Over time, the two proven methods of boosting (or maintaining) ranking seem to be (1) using scholarships to attract more-qualified students and (2) increasing production of scholarship by the faculty. Notably, for the most part the type of scholarship that is rewarded is the most traditional kind (long law review articles), and there isn't much innovative about enticing students with cash. So long as resources are focused on these two things, innovative teaching and scholarship (which is inherently riskier) probably are not going to be valued as highly.

This is not to say that innovation doesn't have some positive effect on rankings. For example, if a well-qualified prospective student sees innovative teaching, she may be drawn to that school. Similarly, some of the most successful scholarship is also innovative in its approach, and at times is successful in large part because it is innovative. However, this is just a fraction of the innovation people are bringing to their teaching and scholarship. Shouldn't it count for more?

-- Mark Osler