By: Michael Feldman

Legal tech startup LawGeex has published a study demonstrating their artificial intelligence software is able to identify issues in Non-Disclosure Agreements (NDAs) faster and more accurately than experienced lawyers.

In the study, five legal contracts were analyzed by 20 experienced lawyers, as well as the LawGeex software, to find issues referenced in the text that could suggest risk for clients – things like liability, indemnification, non-compete restrictions, and so on. The contracts used in the study were actual NDAs used by US companies, in this case Enron, Pacific Gas and Electric, Cargill, and InterGen. Prior to the test, the LawGeex AI had been trained on thousands of NDA contracts (although not these five) using machine learning and deep learning technologies.

The result: the LawGeex solution correctly identified the specificed legal issues with 94 percent accuracy compared to an average of 85 percent for the lawyers. The best lawyer actually matched LawGeex with a 94 percent score, while the lowest performing lawyer managed just 67 percent.

 

 

Perhaps more significantly, the LawGeex software took just 26 seconds to complete the entire task, against an average of 92 minutes for the lawyers. The time savings points to the real utility of this solution, namely getting software to do much of the grunt work involved in looking through reams of legalese and flagging potential problem areas. Given the volume of legal contracts that have to be processed, this can offer a real advantage to corporate law firms looking to improve their own price-performance. LawGeex estimates their solution is able to save 90 percent of the costs compared to manual review. At the same time, the better accuracy means clients are less exposed to risk.

The press release quotes Gillian K. Hadfield, Professor of Law and Economics at the University of Southern , one of the three legal experts that collaborated on the study. According to him, the experiment may actually understate the advantage of the LawGeex solution. “The lawyers who reviewed these documents were fully focused on the task: it didn't sink to the bottom of a to-do list, it didn't get rushed through while waiting for a plane or with one eye on the clock to get out the door to pick up the kids,” said Hatfield. “The margin of efficiency is likely to be even greater than the results shown here.”

The study points out that this is not the first time AI was used in the place of has human legal expertise. In one case, another AI startup, called CaseCrunch, pitted lawyers against its own software to determine if customer claims for improper selling of Payment Protection Insurance (PPI) would hold up in court. Another case involved AI software that was used to predict legal decisions made by the European Court of Human Rights. In both of those examples, the AI was used to help guide the lawyers, rather than perform specific legal tasks, which is what the LawGeex software does.

The broader premise of employing AI for legal services is outlined in the study’s introduction, which references a McKinsey report that estimates 22 percent of a lawyer’s job and 35 percent of a paralegal’s job can be automated.  No only does that mean the human lawyers can be freed to devote more of their time for strategic work and other high-level tasks, but considering that the people spend about $600 billion annually on legal services, that suggests AI solutions like this could change the economic fundamentals of the market.

The full study can be downloaded here.