AI Makes Lawyers Faster. But Does It Make Them Better?

Five randomized controlled trials, 622 law students, three model generations. That is the entire controlled evidence base for whether AI assistance improves legal work, as of this writing. It is a small literature, but it is a real one — and it converges on an answer that is more specific, and less flattering to the current wave of legal AI products, than either the boosters or the skeptics are saying.

The short version: AI makes lawyers faster. The evidence that it makes them better is much weaker, and concentrated almost entirely in dimensions other than correctness. Every trial we found shows a time reduction. Almost none show a reliable gain in accuracy. The one dimension clients actually pay for — is the analysis right — is the one dimension the evidence has barely touched.

This piece is a companion to our earlier review, "Human vs. AI vs. Human + AI", which found a "jagged frontier" across consulting, clinical work, writing, and software engineering but no legal-specific RCT rigorous enough to include. Five now exist. This is what they show.

Methodology

We searched Google Scholar, SSRN, arXiv, Crossref, SAGE, Wiley, the AEA RCT Registry, Open Science Framework, and the institutional repositories of the University of Michigan, University of Minnesota, and ETH Zurich, plus forward and backward citation chaining from every study we found. Search conducted August 2026.

Inclusion required: random assignment to AI-access vs. control; a legal-task outcome; objective grading, not self-report; and a primary source we could independently verify. That excluded vendor and industry surveys with self-reported outcomes; benchmark studies of AI performing legal tasks alone, with no human-AI comparison¹; and one genuine field experiment whose outcome measures were entirely self-reported² despite otherwise meeting every other bar. Five RCTs, covering 622 law students, met every criterion.

We use the same four evidence tiers as our other research: primary source, peer-reviewed research, working paper or preprint (not yet peer-reviewed), and program or advocacy report. Claims that go beyond what a cited study itself states are marked as interpretation; everything else is a direct report of a source's own finding.

Every study below was independently verified against its primary document — the published article where available, the author's working-paper version otherwise — in August 2026. That pass caught real problems in an earlier draft of this piece: a sample size we had wrong, a set of quality figures that did not match any table in the actual paper, and a blanket claim that no study found a significant accuracy gain, when one study did, in one narrow condition. Those are corrected throughout what follows, and the Limitations section says more about what that process found.

*153 students enrolled; 127 completed. Figures throughout use the completer sample.

What the evidence supports

Speed: the one finding that replicates cleanly

Every task-level speed comparison across the four studies that measured it was negative — AI-assisted participants finished faster, with no exceptions. Contract drafting saw the single largest gain, a 32.1% reduction in time alongside the only statistically significant quality improvement in that study³. In-document highlighting of complaint filings cut review time by roughly 30%, with quality unchanged⁴. A tool grounding its recommendations in quoted case-filing passages improved both accuracy (6.0%) and speed (25.9%) reviewing default judgments⁶.

What is notable is what did not change across three model generations: speed gains from GPT-4-era tools (2023–2024) and from newer RAG and reasoning-model tools (2024–2026) are statistically indistinguishable. The researchers behind the most recent of these trials put it plainly: they found little evidence that the magnitude of time savings differed between GPT-4 and the two newer tools they tested⁵. Two years of model progress does not appear to have bought additional speed on these tasks.

Quality: modest, uneven, and mostly not about accuracy

The largest and most recent study in this literature tested six realistic legal tasks against two tools — a retrieval-augmented research assistant and a reasoning model — and graded outputs on five dimensions: accuracy, analysis, organization, clarity, and professionalism⁵. The pattern across those six tasks is consistent and, we think, the most important finding in the piece: gains concentrated in clarity, organization, and professionalism. Accuracy, the dimension a client is actually paying for, moved in only one case — a reasoning model, on a task that required focusing analysis on a single source document, produced a significant accuracy improvement. Every other accuracy comparison across both tools and the remaining five tasks was not statistically significant. One tool arm even produced a significant negative accuracy effect, on a persuasive-letter task.

One task in that same study — drafting an NDA — showed no significant effect on any dimension, from either tool. It also had the highest no-AI baseline score of the six tasks, leaving comparatively little room to improve⁵. A separate, earlier study found the opposite pattern on a different contract-drafting task: the largest quality gain in that entire study³. The two tasks are not comparable — different contracts, different graders, different baselines — and the honest reading is that the popular claim "AI is good at contracts" is not supported by controlled evidence in either direction. It depends enormously on the specific task.

The skill-leveling question is now genuinely disputed

An earlier, widely cited study found that AI assistance compresses the performance distribution: the weakest students in that trial gained roughly 45 percentile points on average, while the strongest lost roughly 20⁸. A follow-up RCT confirmed the same compression pattern on two drafting tasks, with the relationship between without-AI and with-AI performance flattening substantially for both³.

A more recent study complicates this considerably. Under conditions where AI use was optional rather than mandatory — which is how firms actually deploy these tools — the pattern reversed: strong students tended to decline the tool altogether, while weak students adopted it and used it unproductively. Training changed who adopted it, not how well the tool worked once adopted, and reversed the disparity (adoption 41% vs. 26% untrained, p=0.044; trained users scored meaningfully higher)⁷.

These findings are not necessarily contradictory. Mandatory-use designs measure the effect of AI on someone already using it; optional-use designs measure the effect of making AI available — which is the actual condition under which every firm rollout happens. The profession routinely conflates the two.

Verifiability, not model capability, predicts whether assistance helps

The clearest positive results in this literature share one property: they let the user cheaply check the machine's work against a specific source. Highlighting relevant passages in the original document cut review time by 30% with no quality cost⁴. Grounding recommendations in quoted, citable passages from the underlying filings improved both accuracy and speed with very high statistical confidence⁶. The clearest null, by contrast, came from an opaque format: AI-generated summaries alone produced no significant change in time or quality, and adding summaries on top of highlighting performed worse than highlighting by itself⁴.

If there is one actionable design principle in this evidence base, it is this: interventions that ask the human to trust an opaque output underperform interventions that let the human cheaply verify it. That is a claim about interaction design, not about how capable the underlying model is.

The practicing-attorney gap

This is the central limitation of the whole literature, not a footnote to it. Every study used law students. None used a licensed, practicing attorney. The only study that came close — a field deployment with legal aid professionals — was excluded here because its outcomes were entirely self-reported², despite genuine field randomization.

That gap matters in at least four ways. A law student drafting a complaint under a five-hour cap is not a fourth-year litigation associate who has drafted two hundred; if AI compresses performance by lifting weak performers, a population sampled from the bottom will overstate the gain for someone already experienced. The tasks themselves are simplified — elements supplied rather than researched, cases provided rather than found, no opposing counsel, no client changing facts mid-week, no malpractice exposure. None of these designs capture the verification burden a real firm bears before AI output reaches a client or a court — the speed gains measured here are, at best, upper bounds once that cost is added back in. And flat-fee study payment is a reasonable proxy for effort but not for the realization-rate pressure an associate faces in an annual review.

The honest summary: this literature tells us, with reasonable confidence, what happens when trained law students use AI on realistic-but-simplified tasks under a clock. It does not yet tell us what happens when an experienced attorney uses it on a live matter with real stakes.

Limitations

This was a structured search, not a formally registered systematic review — no prospective protocol, single-reviewer rather than independent dual screening, and a narrower database sweep than a Cochrane-standard review would use. The evidence base is small: five studies, 622 people, all law students. Two of the five studies are preprints, not yet peer-reviewed. Four of the six papers cited here share overlapping author teams centered at the University of Minnesota, which reduces how independent the "replication" of the speed finding really is.

One correction is worth stating plainly, since it changed how this piece reads. An earlier draft, built from a document that carried no clear provenance, attributed a specific sample size and a table of per-dimension quality statistics to the largest study in this review — and stated flatly that no study in the literature found a significant accuracy improvement. Neither survived a check against the actual paper. The real completer sample is 127, not the figure first used; no such quality table exists in the source document, which instead reports results task-by-task; and the paper does report one significant accuracy gain, in one narrow condition. Both are corrected above. We're noting this here because a review whose entire value proposition is "we checked the primary sources" owes readers a record of what checking actually found, including our own error.

A second citation problem, unrelated to that document, is worth a separate note: a benchmark study of an LLM acting as a judge¹, excluded from this review because it has no human-AI comparison arm, was previously misattributed to a journal placement we could not verify. The source is, as far as we can confirm, a university working paper. It doesn't affect any finding above, since the study was excluded either way, but it's corrected in the source list below.

References

1. Posner, E. A., & Saran, S. (2025). "Judge AI: Assessing Large Language Models in Judicial Decision-Making." Coase-Sandor Institute for Law & Economics Research Paper Series, No. 25-03, University of Chicago. Replicates a human-judge/student experiment using an LLM alone; no human-AI randomized comparison arm. Working paper.
2. Chien, C. V., & Kim, M. (2025). "Generative AI and Legal Aid: Results from a Field Study and 100 Use Cases to Bridge the Access to Justice Gap." Loyola of Los Angeles Law Review, 57(4), 903. Field study with genuine randomization across 91 legal aid professionals; outcomes are self-reported, not independently graded.
3. Choi, J. H., Monahan, A. B., & Schwarcz, D. (2024). "Lawyering in the Age of Artificial Intelligence." Minnesota Law Review, 109(1), 147–218. RCT, N=59, GPT-4, University of Minnesota.
4. Nielsen, A., Skylaki, S., Norkute, M., & Stremitzer, A. (2024). "Building a Better Lawyer: Experimental Evidence That Artificial Intelligence Can Increase Legal Work Efficiency." Journal of Empirical Legal Studies, 21(4), 979–1022. RCT, N=206, 9 U.S. law schools. Verified against the author-posted working-paper text.
5. Schwarcz, D., Manning, S., Prescott, J. J., Barry, P., Cleveland, D. R., & Rich, B. (2026). "AI-Powered Lawyering: AI Reasoning Models, Retrieval Augmented Generation, and the Future of Legal Practice." Working paper. RCT, 153 enrolled / 127 completed, University of Minnesota & University of Michigan, October 2024 fieldwork. We could not independently confirm a final journal placement for this manuscript and cite it as a working paper.
6. Worledge, T., Bensouda Koraichi, O., Bernal, D., Caspi, A., Hashimoto, T., Guestrin, C., & Engstrom, D. F. (2026). "AI Assistance for Human Review of Default Judgments." arXiv preprint arXiv:2607.01256. RCT, N=66, simulated court review.
7. Chen, B. M., & Bao, H. (2026). "Training for Technology: Adoption and Productive Use of Generative AI in Legal Analysis." arXiv preprint arXiv:2603.04982v3. RCT, N=164, Hong Kong law students. Verified against the full paper. The authors caveat two of their own findings more heavily than a summary conveys: the "adoption dominates effectiveness" conclusion rests on wide, overlapping confidence intervals, which the authors say they cannot use to statistically rule out effectiveness as the real mechanism; and the finding that higher-ability students opt out without training is explicitly labeled by the authors as descriptive rather than confirmatory, since their ability measure is a post-treatment outcome from the same exam, not a clean pre-treatment proxy.
8. Choi, J. H., & Schwarcz, D. (2025). "AI Assistance in Legal Analysis: An Empirical Study." Journal of Legal Education, 73(2), 384–420. Source of the "equalization thesis" — AI narrowing the performance gap between weak and strong performers on multiple-choice questions.