Law School Innovation

"Measurement and Its Discontents" ... and US News rankings and law school grades

The title of this post starts with the headline of this interesting commentary published in yesterday's New York Times. Though not saying one word about legal education or law schools, I thought many parts of piece (and especially the passages quoted below) were especially interesting and deserved consideration as we head into the (never-ending) law school ranking season:

Why are we still stymied when trying to measure intelligence, schools, welfare and happiness?

The problem is not that we don’t yet have precise enough tools for measuring such things; it’s that there are two wholly different ways of measuring.

In one kind of measuring, we find how big or small a thing is using a scale, beginning point and unit. Something is x feet long, weighs y pounds or takes z seconds. We can call this “ontic” measuring, after the word philosophers apply to existing objects or properties.

But there’s another way of measuring that does not involve placing something alongside a stick or on a scale. This is the kind of measurement that Plato described as “fitting.” This involves less an act than an experience: we sense that things don’t “measure up” to what they could be. This is the kind of measuring that good examples invite. Aristotle, for instance, called the truly moral person a “measure,” because our encounters with such a person show us our shortcomings. We might call this “ontological” measuring, after the word philosophers use to describe how something exists.

The distinction between the two ways of measuring is often overlooked, sometimes with disastrous results. In his book “The Mismeasure of Man,” Stephen Jay Gould recounted the costs, both to society and to human knowledge, of the misguided attempt to measure human intelligence with a single quantity like I.Q. or brain size.  Intelligence is fundamentally misapprehended when seen as an isolatable entity rather than a complex ideal.  So too is teaching ability when measured solely by student test scores.

Confusing the two ways of measuring seems to be a characteristic of modern life.  As the modern world has perfected its ontic measures, our ability to measure ourselves ontologically seems to have diminished. We look away from what we are measuring, and why we are measuring, and fixate on the measuring itself.  We are tempted to seek all meaning in ontic measuring — and it’s no surprise that this ultimately leaves us disappointed and frustrated, drowned in carefully calibrated details....

But how are we supposed to measure how wise or prudent we are in choosing the instruments of measurement and interpreting the findings? Modern literature is full of references to the dehumanizing side of measurement, as exemplified by the character Thomas Gradgrind in Dickens’s “Hard Times,” a dry rational character who is “ready to weigh and measure any parcel of human nature, and tell you exactly what it comes to,” yet loses track of his own life.

How can we keep an eye on the difference between ontic and ontological measurement, and prevent the one from interfering with the other?

One way is to ask ourselves what is missing from our measurements.... In our increasingly quantified world, we have to determine precisely where and how our measurements fail to deliver.

I suspect many who read this blog would be quick to assert that law schools are "fundamentally misapprehended when seen as an isolatable entity [subject to ranking by US News] rather than a complex ideal."  In addition, lots has been said by many US News critics about what is missing from the US News measurements. 

And yet, I cannot help but wonder how much the entire traditional law school model — and especially traditional law school grading systems — also are subject to the problem of mixing "ontic and ontological measurement."  I have long thought and feared that law schools could hardly complain all that much about being peculiarly graded by US News when these institutions continue to graded the potential of future lawyers in a (more?) peculiar way.

Posted by DAB

What is the latest state of multiple-choice testing in law school?

I am trying to catch up on some law school readings this summer, and I noticed this interesting-looking piece on SSRN.  The piece by Janet Fisher is titled "Multiple-Choice: Choosing the Best Options for More Effective and Less Frustrating Law School Testing," and here is the abstract:

Multiple-choice testing presents challenges and frustrations not only for the students who take the tests, but also for the doctrinal faculty who prepare and score the tests and for the academic support faculty who work with students having difficulty with multiple-choice tests. This article discusses means by which the multiple-choice testing experience in law school could be improved for both students and faculty.  After a brief overview of the history of multiple-choice testing, the article describes problems that arise in connection with multiple-choice testing and the possible effects of flawed multiple-choice questions.  The article then reviews basic multiple-choice item-writing guidelines and some general principles of test validity.  For this, the article draws upon the work of law professor Michael Josephson and testing authority Thomas Haladyna. Finally, the article evaluates appeal and answer-justification procedures that could be used to enhance multiple-choice testing.

In addition to being eager to engender a debate over the use of multiple-choice testing in law school, I am curious to know if anyone has tried to quantify how many settings in which law schools are using multiple-choice testing is utilized.

Will new grading systems really lead law schools away from traditional exams?

The National Law Journal has this piece reviewing the new grading systems that are becoming all the rage at elite law schools.  Here is how the piece begins:

Several leading law schools are retooling their grading policies, with some institutions making major revisions and others merely tweaking their systems. Harvard Law School and Stanford Law School, for example, are switching from the traditional grade and letter policies to pass/fail systems. At the same time, New York University School of Law now allows professors to give more A's. And some institutions, such as Columbia Law School, are reviewing their grading systems to see whether they need updating.

And here is a idea within the piece that really caught my attention:

Stanford Law Dean Larry Kramer said that it's too early to draw conclusions about the new system but that it seems to be working well.... "One, [the new system] conveys more accurate information to employers without diminishing student incentive to work; two, it reduces needless grading anxiety; and three, it encourages faculty to experiment more with evaluative things they do in their classes," Kramer said.  For example, the system allows faculty to use a combination of short-term papers with exams for evaluation, Kramer said.

I find the notion that new grading systems could foster a move away from the traditional law school exam both plausible and appealing.  Indeed, though I have been agnostic in my views about all these grading developments, I will get on the honors/pass/fail grading bandwagon if such a system proves to be a catalyst for moving law schools away from traditional in-class exams.

Posted by DAB

Some related LSI posts:

• What will be (and should be) the future of traditional law school grading?
• Is there an ideal grading system for law schools?
• What should we make of all the (not-always-so?) innovative grading changes?

What should we make of all the (not-always-so?) innovative grading changes?

Though law school grading and modern grade inflation are hardly an innovative topic, the news of more big schools altering or raising their grading system seems worthy of bloggy reflection.  This week the news of revised (and inflated) grading curvescome from NYU (as reported here and here) and USC (as reported here and here).

Lots of folks surely have lots of thoughts about lots of different factors that can influences and justify changes to lots of different kinds of law school grading schemes.  And I would like to hear some of those thoughts in the comments. 

I would also love to hear reactions to what I think makes the most sense as a law school grading system.  Specifically, were I creating a law school grading system from scratch, I would use a completely opaque numbering scheme to define/grade performance in individual classes (say 12 through 27) combined with a very student-friendly class rank system that reports on which student make the top 5%, top 10%, top 25%, top 33%, top 50% and top 67% of the class. 

Part of my suggestion for a student-friendly class rank system class rank system would be to include a "best in ____" or "top 10% in ____" profile for every student.  That is, I think a school could create a computer system that would figure out a name/category for whatever sets of courses each particular student has done especially well.  So, even if a student is only ranked in the top half of her class, or even ends up in the bottom third of his class, the student's official transcript might still say "best in business courses" or "top 10% in public law course" or "best in legal writing courses."

Some related LSI posts:

• What will be (and should be) the future of traditional law school grading?
• Is there an ideal grading system for law schools?

Posted by DAB

What will be (and should be) the future of traditional law school grading?

At his law school blog, Brian Leiter has this notable post, titled "Will Other Schools Follow the Yale/Harvard/Stanford Lead of Effectively Eliminating Grades?".  Here is how it starts:

There are rumors aplenty that Columbia and NYU may move to something like the Yale system of essentially two grades -- Honors/Pass -- now that Harvard and Stanford are going that route (though perhaps these two will actually utilize Low Pass and Fail, unlike Yale).  NYU, given its size, can probably least afford to eliminate sorting mechanisms, especially since it appears Columbia grads are still slightly preferred by the very top NYC firms.  For those who have asked, I think there is essentially no chance Chicago will go this route, or anything like this route.

Among interesting aspects of this post is the notion that the virtues and vices of a two-grade system may depend significantly on the size of a law school (even though Harvard and Stanford represent opposite extremes as to law school size).  Of course, it seems obvious that the virtues and vices of a two-grade system may also depend significantly on the national reputation of a law school, though I could readily imagine good (and bad) arguments for schools not consistently ranked in the Top 10 considering a move to an Honors/Pass grading system.

Some related LSI posts:

• Does (law school) size matter?
• Is there an ideal grading system for law schools?