October 26, 2011
Willing yet again to consider putting my US News vote up for sale...
Long-time readers may recall this post from four years ago, in which I asked "Would it be unethical (or even illegal) to put my US News vote up for sale?" Here is the back-story which prompted my (sincere? tongue-in-cheek?) inquiry in 2007:
For the second consecutive year, I have received US News' survey asking me to help identify "law schools having the top programs in intellectual property law." My receipt of this survey highlights just how flawed some aspects of US News' rankings can be.
I was an IP litigator a decade ago and I taught a few IP courses early in my career. But, especially with my primary field so active, I cannot even hope to keep up with all the IP doings in law schools. Nevertheless, US News seeks my opinion on which 15 schools have "the highest-quality intellectual property law courses or programs." Candidly, I have absolutely no idea.
I suppose I could try to make educated guesses about the best IP programs based on who sent around the hottest "law porn"covering IP topics this year. But I also could throw darts at the survey form and probably not do much worse.
Consequently, I am now wondering if I could and should simply offer my US News survey to the highest bidder. Helpfully, US News promises that survey responses are kept confidential, so nobody would know whether or to whom I sold my vote.
Of course, I do not want to do anything unethical or illegal, so I am genuinely seeking an answer to the question posed in this post's title. I know vote selling in some contexts can be illegal, but I don't think a survey by a private magazine garners too much public protection. As for ethics, well, what I am proposing seems no less savory than what some schools have reportedly done to game the US News rankings system. Plus, some recent research suggests that open vote buying/selling may be efficient in this kind of setting.
Fast forward four years, and I am now in my office holding this year's version of the US News' survey asking me to again help identify "law schools having the top programs in intellectual property law." Apparently the fact that I previously talked up the notion of selling my US News' vote to the highest bidder did not get me scratched from the list of potential voters. (I assume that the people who run the US News' survey never got wind of my talk of vote selling. But it is fun to imagine that they heard of my (joking?) plans and nevertheless still thought I was a good person to ask about the top 15 IP programs.)
If memory serves, nobody actually offered me any money for my US News vote back in 2007, so I never did have an opportunity to consider seriously whether I could be bought off. I do recall a few folks responding to my post via e-mail with information about how great the IP program was at their school. In other words, I most certainly did not get rich from, but I did get some useful information in response to, my prior post.
Back in 2007, I ultimately concluded that too much personal bias and not enough valid information would end up informing my survey responses, and so I ripped up the form and tossed it away. Disturbingly, this year's cover letter and US News form fails to suggest trashing the survey if one lacks the knowledge or information needed to fill it out appropriately. That very fact has me wondering if US News actually would prefer me to complete the form after getting paid for my votes rather than fail to return the form at all.
October 24, 2011
"Measurement and Its Discontents" ... and US News rankings and law school grades
The title of this post starts with the headline of this interesting commentary published in yesterday's New York Times. Though not saying one word about legal education or law schools, I thought many parts of piece (and especially the passages quoted below) were especially interesting and deserved consideration as we head into the (never-ending) law school ranking season:
Why are we still stymied when trying to measure intelligence, schools, welfare and happiness?
The problem is not that we don’t yet have precise enough tools for measuring such things; it’s that there are two wholly different ways of measuring.
In one kind of measuring, we find how big or small a thing is using a scale, beginning point and unit. Something is x feet long, weighs y pounds or takes z seconds. We can call this “ontic” measuring, after the word philosophers apply to existing objects or properties.
But there’s another way of measuring that does not involve placing something alongside a stick or on a scale. This is the kind of measurement that Plato described as “fitting.” This involves less an act than an experience: we sense that things don’t “measure up” to what they could be. This is the kind of measuring that good examples invite. Aristotle, for instance, called the truly moral person a “measure,” because our encounters with such a person show us our shortcomings. We might call this “ontological” measuring, after the word philosophers use to describe how something exists.
The distinction between the two ways of measuring is often overlooked, sometimes with disastrous results. In his book “The Mismeasure of Man,” Stephen Jay Gould recounted the costs, both to society and to human knowledge, of the misguided attempt to measure human intelligence with a single quantity like I.Q. or brain size. Intelligence is fundamentally misapprehended when seen as an isolatable entity rather than a complex ideal. So too is teaching ability when measured solely by student test scores.
Confusing the two ways of measuring seems to be a characteristic of modern life. As the modern world has perfected its ontic measures, our ability to measure ourselves ontologically seems to have diminished. We look away from what we are measuring, and why we are measuring, and fixate on the measuring itself. We are tempted to seek all meaning in ontic measuring — and it’s no surprise that this ultimately leaves us disappointed and frustrated, drowned in carefully calibrated details....
But how are we supposed to measure how wise or prudent we are in choosing the instruments of measurement and interpreting the findings? Modern literature is full of references to the dehumanizing side of measurement, as exemplified by the character Thomas Gradgrind in Dickens’s “Hard Times,” a dry rational character who is “ready to weigh and measure any parcel of human nature, and tell you exactly what it comes to,” yet loses track of his own life.
How can we keep an eye on the difference between ontic and ontological measurement, and prevent the one from interfering with the other?
One way is to ask ourselves what is missing from our measurements.... In our increasingly quantified world, we have to determine precisely where and how our measurements fail to deliver.
I suspect many who read this blog would be quick to assert that law schools are "fundamentally misapprehended when seen as an isolatable entity [subject to ranking by US News] rather than a complex ideal." In addition, lots has been said by many US News critics about what is missing from the US News measurements.
And yet, I cannot help but wonder how much the entire traditional law school model — and especially traditional law school grading systems — also are subject to the problem of mixing "ontic and ontological measurement." I have long thought and feared that law schools could hardly complain all that much about being peculiarly graded by US News when these institutions continue to graded the potential of future lawyers in a (more?) peculiar way.
Posted by DAB