Moneyball for law firms?

 

As kind of an aside in a recent post, ELS’s Bill Henderson notes a study by Kerma Partners on lawyer productivity/success at large law firms. The idea is that law firms want to develop better indicators or predictors of lawyer success at their firms. Of course, they want factors that can be gleaned from students’ (and perhaps laterals’) resumes and interviews. This also requires developing a measure of “success” or “productivity”. To give this away early, at least two of these success measures are billable hours and tenure at the firm (# of months) (apparently attrition has been a big problem – it is not clear whether such attrition has been voluntary or involuntary). Here are some excerpts from the study:

Together with Redwood (a part of LexisNexis), we just completed a “success factors” engagement with a top 25 US law firm – which was designed to assist our client in identifying and attracting graduate recruits who not only have the credentials to enter the firm, but also possess the “stuff” to thrive at the firm. Although our client has always competed for the “best” candidates (by traditional thinking, those having high GPAs at the top law schools), the firm, like most (or all) of its peers, has nonetheless experienced mixed results in its recruiting efforts, suffering in recent years from high associate attrition rates. 

So, what to do…

We set out to determine whether — in addition to a candidate’s law school pedigree, and some of the subjective things one can learn about a candidate in an interview setting — there was something more the firm could look for in its candidates that would be predictive of the candidate’s mid- to long-term success at the firm. We found that there is — and determined that incorporating these findings into a firm’s recruiting program can improve a firm’s bottom line.

And, how to do it …

 

On the basis of several “soft” and “hard” performance measures that were of particular relevance to our client, we investigated over 1300 timekeepers (past and present) to identify those who were “outperformers” and those who were “underperformers”. Against this statistically ample population of lawyers, we tested the relationship to performance of about 200 “attributes” — each of which can be ascertained objectively prior to the point of hire, either from the CV, the transcripts (graduate and undergraduate), or interviews.

We identified several attributes that were disproportionately present among the population of outperformers. Each such “success factor,” when observed individually, was significantly more “predictive” of success at the firm than law school rank or GPA.² Also when observed collectively, the success factors showed a strong relationship to outperformance.

 

And then, something really interesting…

By several different measures, a high law school rank/GPA was only moderately indicative of success (as shown at the “school/grade baseline”). In contrast, lawyers having “at least one” of the success factors, “at least two” of the success factors, and so on, were increasingly more likely to be outperformers. The strong odds were that a lawyer having none of the success factors was destined to fail. (emphasis added)

Did you see that? – only “moderately indicative of success”. The two primary factors in hiring by my memory of law school recruiting are not that helpful. Or, are they? Perhaps there is a selection effect at work. If the variance in these factors is constrained, then this might be the case. For instance, if all of the firm’s attorneys’ class ranks are in the top 5 to 15%, then this variance might understandably not be that telling on performance, however, if the firm has attorneys with ranks varying from bottom half to top 1%, then the fact that rank isn’t that helpful in explaining performance seems very telling. 

Now Kerma isn’t giving away their “factors of success”, and this is certainly understandable. If I was able to put together a model that predicted the stock market extremely well, then I can assure you that I would not be publishing it – at least not for free. 

However, I would like to take this opportunity to offer my services to law firms to conduct this type of analysis for their recruiting efforts. So, for the price of one junior partner’s weekend in Vegas, you can get Moneyball analysis of your firm’s “determinants of lawyer success”. What a bargain  

What are my “factors” – well, I have some ideas, but of course, I’m not telling. I imagine that some firms might also have different methods of determining success beyond billable hours and months at the firm. Firms also might want to test their own hunches as to what predicts success.  

Academic departments and law schools could likely undertake similar analysis to play Moneyball, but that would assume that measures of faculty “productivity” or “success” could be agreed upon.