The Case Against Transparency in Law School Rankings Methodology

Another Spring, and another season of hand-wringing over the U.S. News law school rankings is upon us.  As usual, legal academics almost uniformly criticize the rankings, while their schools restructure themselves to achieve better results in those same rankings.

I don't think it is possible to overstate the damage this cycle causes.  Few people seem to respect the methodology used by U.S. News, yet most schools seem to make very substantive decisions primarily with an eye to increasing their standing in one factor or another that is measured by the rankings.  For example, financial aid is often used to increase the LSAT numbers of incoming students, rather than to increase diversity or to help students with true financial need.  Some schools employ their own graduates at make-work jobs to bump up their employment numbers, while others seem to spend more heavily to promote faculty scholarship to U.S. News voters than they do to support the production of that scholarship.  Some schools, it seems, are just lying about things like employment numbers.  All of this is morally wrong and systemically degrading the integrity of our work.

In short, there are two wrongs associated with the U.S. News rankings-- the rankings themselves, and the way we have reacted to them.  We, as legal academics, control only the latter, and we have shown very little discipline in that area.  Because we do not seem able to resist the urge to manipulate our programs to fit the rankings criteria, perhaps we would be better off if the rankings criteria were unknown.  While the rankings would still exist with all their problems, this would curtail the unfortunate reactions at too many law schools.

The cost of non-transparency is obvious-- we would lose the ability to effectively criticize the rankings methodology.  This is a legitimate concern.  However, how much good is our criticism doing?  U.S. News seems unwilling either to cease production of the cash-cow rankings or substantively change the formula.  It could be that the cost of losing our ability to issue ineffective criticism is worth the gain of retrieving our integrity.

Sadly, the moment for non-transparency has probably passed.  If U.S. News stopped telling us, today, how they calculate the rankings, the games would continue under the assumption that roughly the same methodology was being used.  However, if U.S. News fails as a business enterprise and the rankings project is scooped up by another entity, it may be an opportunity to re-start the process.  (I know some would hope that the rankings would just go away at that point, but that ignores economic realities).  A committee including leading critics could be brought together to come up with the formula, which then would be kept secret.    

In nearly all things, I am for transparency.  However, I am wondering if this is a unique situation where transparency does more harm than good.

-- Mark Osler