March 15, 2009
Could a new test of lawyer skills really challenge the LSAT?
Earlier this week the New York Times ran this interesting article, headlined "Study Offers a New Test of Potential Lawyers." Here are snippets:
Just what makes a good lawyer? In trying to answer that question, professors at the University of California, Berkeley, have come up with a test that they say is better at predicting success in the field than the widely used Law School Admission Test.
The LSAT, as the half-day exam is known, does not claim to predict much beyond a student’s performance in law school. But critics contend that it does not evaluate how good a lawyer someone will be and tests for the wrong things. They also say it keeps many black and Hispanic students — who tend to have lower scores — out of the legal profession....
To find out what applicant traits should figure in admissions decisions at law schools, [Professors Marjorie Shultz and Sheldon Zedeck] coordinated individual interviews, focus groups and ultimately a survey of judges, law school professors, law firm clients and hundreds of graduates of Berkeley’s law school. They asked, among other things, “If you were looking for a lawyer for an important matter for yourself, what qualities would you most look for? What kind of lawyer do you want to teach or be?”
The survey produced a list of 26 characteristics, or “effectiveness factors,” like the ability to write, manage stress, listen, research the law and solve problems. The professors then collected examples from the Berkeley alumni of specific behavior by lawyers that were considered more or less effective.
Using the examples, Professor Shultz and Professor Zedeck developed a test that could be administered to law school applicants to measure their raw lawyerly talent. Instead of focusing on analytic ability, the new test includes questions about how to respond to hypothetical situations. For example, it might describe a company with a policy requiring immediate firing of any employee who lied on an application, then ask what a test taker would do upon discovering that a top-performing employee had omitted something on an application.
More than 1,100 lawyers took the test and agreed to let the researchers see their original LSAT scores, as well as grades from college and law school. The study concluded that while LSAT scores, for example, “were not particularly useful” in predicting lawyer effectiveness, the new, alternative test results were — although the new test was no better at predicting how well participants would do in law school. Unlike the LSAT, the new test did not produce a gap in scores among different racial or ethnic groups.
But participants might have performed differently on it, had they taken the test when they were applying to law school. Professor Shultz said this was one reason the next step in the research should include tracking test takers over time, from when they apply to law school through their careers.
David E. Van Zandt, dean of the law school at Northwestern, said he would welcome a supplement to the LSAT to evaluate applicants, a sentiment echoed by other law school deans. John H. Garvey, dean of Boston College Law School and past president of the Association of American Law Schools, said, “It would be good for us and for other schools to have other measures that complement the LSAT and that would help us identify promising candidates.”
I have nothing but praise for the efforts here to develop an LSAT alternative, and I hope this project keeps moving ahead. I also hope that the development of new visions and new assessments of lawyer “effectiveness factors” might also impact law-school testing methods, too.
Posted by DAB
TrackBack URL for this entry:
Listed below are links to weblogs that reference Could a new test of lawyer skills really challenge the LSAT?:
I would like to see an out of state lawyer take on the racial disparity of the bar exam pass rates. Because the exam is scientifically a lawyer hierarchy piece of anti-scientific garbage, the exam is discriminatory and unlawful. Its garbage format comes from the 13th Century. It does not come close to meeting standards of modern test making. It has no reliability statistics. It has no validation. They do not even allow independent examination of their statistics. It has to get enjoined. All people who lives were disrupted by this biased, garbage exam should be compensated.
Posted by: Supremacy Claus | Mar 15, 2009 5:47:39 PM