You are local counsel in a case. It’s 3 p.m. You just got the brief from national counsel that’s due to be filed in two hours. It’s 60 pages long and has 150 case citations. You have no way of knowing whether they are all to real cases or accurate. What do you do?
That’s the problem that GenAI presents. But at least one vendor has a solution and it’s not all GenAI.
The Problem. And A Solution.
There’s an old saying about someone who has more hat than cattle. It refers to someone who is talk with little substance. This is why the GenAI volcano is poised to erupt: too much talk, not enough substance.
Melissa Rogozinski and I have talked a lot recently in our Pompeii series of articles (see links at the conclusion of this article) about the often-unspoken problems and challenges AI presents to legal in an age of hype. As one of our professors used to say, the problem is the problem. And the problem here is that too few are actually trying to solve problems, and too many are hyping AI products as the only game in town.
That’s why Clearbrief and its new product, Cite Check Report, which automatically verifies GenAI outputs, is important. The headline isn’t just what Cite Check Report does, it’s that Clearbrief adopted a process and attacked a problem in a practical way that actually helps lawyers, paralegals, and legal professionals. Clearbrief’s approach is not more GenAI but to use non-GenAI tools to solve one of GenAI’s more vexing problems.
In order to understand why we are excited about this solution and why it may work, you need to know something about Clearbrief.
Clearbrief
Clearbrief is legal tech provider that has a track record of listening to its customers and providing solutions to their problems and pain points, much as Ki, who we recently discussed, has done. Clearbrief offers AI and natural language tools to help its customers draft legal pleadings and briefs that are accurate and well done.
Clearbrief was founded in 2020 by Jacqueline Schafer, a former litigator and leading thinker in the legal ecosystem. In 2022, it was a finalist at the annual ABA TechShow start up competition. In a testament to what it does and how it treats customers, it’s now a leading provider to Am Law 200 firms as well as smaller firms and corporations.
Cite Check Report
It’s no secret that GenAI tools routinely make cases up or cite cases for the wrong propositions. As a result and as we have discussed, one of the leading drawbacks to GenAI tools is that it takes lawyers and legal professionals more time to verify the outputs than the time it saves. As we also discussed, this disrupts workflows and destroy the underlying trust that is the foundation of how lawyers get work done.
Cite Check Report checks both cites and factual assertions against existing major data bases and a customer’s internal documents. It identifies missing sources and cases and corrects formatting errors. It scores the cited material based on accuracy and identifies mismatches between quotes and what is actually contained in the case or source. It will also show how the author of document addressed the red flags.
Importantly, Cite Check Report does not use generative AI, but what Clearbrief calls classic AI to do this. Because it’s not relying only on GenAI, the checks are accurate.
And Clearbrief’s tools do more: they eliminate pain points by catching typos, catching slight misses in quotes and factual assertions, and automatically doing tedious things lawyers and paralegals hate doing.
Schafer says, “These are like 10 different tools that litigators normally have to do and open and go into to get a single brief filed. And this is the unsexy stuff that AI doesn’t care about as much because it’s hard.”
She’s right about all this. Litigators and paralegals spend an inordinate amount of time doing stuff they don’t want to do but which has to be done. Stuff that they didn’t spend time in school and learning to do. This is what AI and its providers should be about: solving problems and making customers’ lives easier, not harder.
An Elegant Solution
Schafer and her crew recognized these problems and developed an elegant solution like Cite Check Report. For example, Clearbrief recognized, as did we, that lawyers cite hallucinated cases for a variety of reasons: tight deadlines, over work, overreliance on the tools provided.
To ignore this practical reality is akin to what a lot of AI vendors do: ignore practical realities and limitations.
But Schafer and her team got there by listening to customers and then critically looking at what GenAI tools can and can’t do. Schafer told us: “I go back to our customer calls practically every day. It’s what grounds me and tells me exactly what to build. I know what problems they’re having. We are in the trenches with them. What they need to use on the real work.”
And Schafer has some thoughts about where we are with GenAI as well: “I feel like we’re just in this sort of AI slop era…One of the biggest things that I’ve learned from being in the trenches and developing these tools are that there are just certain workflows that generative AI is not best suited for because of the uncertainty element. We needed a way where there’s no possibility of hallucination and GenAI creativity coming into play for certain types of tasks.”
The Verification Paradox
Cite Check Report effectively attacked what we describe in Part II of our series as the verification paradox.
The fact is, if AI is used, lawyers and legal professionals have to take the time to verify each and every AI cite. As we mentioned in Part II of our series, this verification necessity often results in more time being sent than is saved by using AI in the first place. GenAI may cut your research time down from 6 hours to 30 minutes. But that’s cold comfort if you have to spend 10 hours checking and reading all the cites and making corrections. The GenAI tool isn’t worth it.
Cite Check Report eliminates the paradox because it can very quickly and automatically do the cite check of cases and factual assertions and provide an analysis of the magnitude of any errors. (The latter is important because we all have stretched the meaning of cases before. Cite Check tells us if we may have gone too far).
Cite Check Report turns the paradox on its head. Instead of costing more to have GenAI do a task, Cite Check can do the things that a human would otherwise have to do in checking cites, correcting typos and finding and noting slight errors and even misaligned arguments in a fraction of the time.
Looking at the Why
Every day we hear of lawyers using AI and citing cases provided by AI that either don’t exist or don’t stand for the proposition they are offered. The results can be catastrophic: fines, loss of clients, embarrassment, even malpractice claims. Yet for whatever reasons, lawyers aren’t saving themselves by meticulously checking cites.
Clearbrief recognizes why this is so. In its refreshing non-hyperbole press release announcing Cite Check Report, it articulated the reasons in a way I have addressed before:
While it’s tempting to chastise any lawyer who fails to meaningfully review pleadings before signing, the reality of modern litigation is that a single filing often contains hundreds of citations to both facts and law, and many hands touch a complex pleading before it gets filed.
Moreover, generative AI mistakes can be so subtle that the human eye can easily miss them, such as replacing a single number in a citation such that the cite now appears to be from the court’s own jurisdiction rather than another circuit court where it is not binding precedent.
Says Schafer, “Partners are being sanctioned and suffering reputational damage for citation errors they didn’t personally make. We built the Cite Check Report to give them what courts are demanding: documented proof that they satisfied their ethical obligations before signing that pleading.”
The Trust Problem
Indeed, it was precisely this problem we were getting at in Part III of our series: the work process in legal depends heavily on work being done by someone for someone else . This could be an associate for a senior partner, a national counsel sending brief to be filed to local counsel or vice versa, even a law clerk for a judge. Before generative AI, that work was by and large trusted by the recipient. If it can’t be and everyone has to check every cite tendered to them, it bogs the work process down to the point that AI becomes useless. As Schaefer puts it, “It’s breaking the trust that we have.”
And practically speaking, Gen AI would be useless for certain tasks like legal research.
It’s Not Just for Lawyers
It’s often forgotten that much of what goes into legal work is not performed by lawyers but by paralegals and other legal professionals. It’s the paralegals who have to check cites. They are called on to do the timelines and run other checks on documents and pleadings. But like the lawyers, it’s not work they went to school to do. It’s boring and tedious. And if something goes wrong, it’s the paralegal who ultimately catches hell.
So, in addition to helping lawyers, Cite Check Report and other Clearbrief tools help everyone in the workflow do better work. To use their skill where they have the most impact. As Schaefer put it, “There’s multiple layers in legal workflows.” And the better those in the work flows are at what they do, the better the product.
More Hat Than Cattle
We need more thinkers like Schafer who offer more cattle than hat, who offer more substance than talk. People who look at what we can do to prevent the problem, to help lawyers save themselves seamlessly and easily, who make using AI easier and more practical. That would be a refreshing change from the constant “hat” many vendors offer.
The solution to GenAI challenges is not just more GenAI. Sometimes it’s the use of other tools. And it will always be the result of independent thinking instead of mass hype.
Looking for a way to solve a problem or get rid of a pain point? Stop talking to vendors who lead with GenAI capabilities. Start with those who lead with solving your problems.
The Pompeii Series:
Like Lawyers In Pompeii: Is Legal Ignoring The Coming AI Infrastructure Crisis? (Part I)
Like Lawyers In Pompeii: Is Legal Ignoring The Coming AI Cost Crisis? (Part II)
Like Lawyers In Pompeii: Is Legal Ignoring The Coming AI Trust Crisis? (Part III)
Like Lawyers In Pompeii: Is Legal Ignoring The Coming AI Financial Crisis? (Part IV)
Like Lawyers In Pompeii: Is Legal Ignoring The Coming AI Definition Crisis? (Part V)
Stephen Embry is a lawyer, speaker, blogger, and writer. He publishes TechLaw Crossroads, a blog devoted to the examination of the tension between technology, the law, and the practice of law.
Melissa “Rogo” Rogozinski is an operations-driven executive with more than three decades of experience scaling high-growth legal-tech startups and B2B organizations. A trusted partner to CEOs and founders, Rogo aligns systems, product, marketing, sales, and client success into a unified, performance-focused engine that accelerates organizational maturity. Connect with Rogo on LinkedIn.
The post Like Lawyers In Pompeii: Is Legal Ignoring Solutions? More Cattle And Less Hat (Part VI) appeared first on Above the Law.