Not sure how good the Harvey benchmark is. I recently looked into traces from an agent solving their firm knowledge task subset and found multiple problematic tasks and rubrics.
As an attorney, I am not particularly impressed about a 20% pass rate on a benchmark. The name of the game for litigation practice is about being prepared for novel situations, not relitigating past ones.
To be fair to them, on this particular benchmark (Harvey's Legal Agent) 20% puts it almost top of the pack of _any_ model, and certainly top for any remotely in its price class.
How much novelty can there be?
If attorneys were as diligent and generous with “open sourcing” their data as SWEs had been, could LLMs benefit from all that training data?
Law and code are only superficially similar as law always requires human interpretation plus law is an interface to reality. Code can stay contained in its nice, little, abstract box.
What are you talking about? 90% of litigation is convincing the judge that existing case law that clearly applies to your case does apply to your case.
I'm guessing what they're saying is that nothing goes to litigation if it's a clear pattern-match to existing cases, so actual litigation practice is only novel cases.
Yeah agreed, especially for legal and healthcare models.
[edit] I'd go one step further too and say the training data should be explicitly requestable too so third parties can verify (freedom of information act styles).
"Freakazoid has ... access to all of the Internet's knowledge. He cannot fly, but often runs around with his arms extended and making whooshing sounds with his mouth, pretending he is."
Sort of ironic, in Google Chrome, when attempting to visit that legal z.ai site I get:
Your connection is not private
Attackers might be trying to steal your information from z.ai (for example, passwords, messages, or credit cards). Learn more about this warning
net::ERR_CERT_COMMON_NAME_INVALID
If he is on a corporate device, he has a corporate-issued MITM certificate, which allow third party "security" companies to read, store and analyze his complete network traffic in plain text.
Curious what domain is next, GLM-Science? And when will it be profitable to drill down to even more domain specific models like GLM-Economic-Geography or GPT-Aquaculture?
Imo, we are going to see a lot more specialised models in the near future as pushing the frontier gets harder. Qwen 3.8 27B for example is a pretty rubbish chat and general world knowledge model, but is stellar at coding (for it's size). Its small enough anyone with a modest gaming PC (16GB vram and 32GB of DDR5) can run it at speeds good enough for overnight coding sessions with an PR to review at the end of it.
Eventually, I hope we'll see many more specialised small models that run nicely on consumer and prosumer hardware. Like LLMs specific to income tax law for UK residents, or LLMs specific to building power delivery circuits for PD USB C devices. There probably is a point where the niche-ness does become to niche to justify the post training, but again, small models means much less training required.
Kind of begs the question if this even needs new/better data? Could you take the foundation model that's trained on everything, and then run a final fine tuning training run on just the set of foundation training data that you want the model to specialise in? Eg, extract just the training data that is about UK income tax law and then run a few epochos on just that? Any ML people know if this is a thing or if not, why it doesnt work?
Wait, so this not an open weight model, correct? Only available via z.ai?
So, if a US or EU person shares their private legal info here, it's processed and stored by an adversarial country that has no extradition treaty with their country?
I prefer all my data be in the hands of foriegn entities, they cant do anything to me. Id much rather Huawei spy on me than Meta (if I have no choice other than to pick one) There's literally no consequence of Huawei having my data. Unless you're a VIP, a chinese entity storing your data is probably better than a US company, as a westerner. It wont be used to manipulate you and will generally be used less adversarially against you.
I think part of the reason there's so much propaganda around chinese tech is to scare you away from possibly using them, not for your own safety, but so Meta/Google/MS ect dont lose access to the ability to spy on every aspect of your life.
I love how we have to pretend like China is some scary boogeyman because people on screens say they are.
> I love how we have to pretend like China is some scary boogeyman because people on screens say they are.
I think you've lost the plot a bit on this one.
I'm scared of China because of their lack of democracy and the terrifying stuff they do to their own citizenry, which presumably they'd very much like to expand.
There's no digital attorney-client privilege. If a lawyer uses LLM's on your behalf, the communications are protected. If you use the same LLM's on your own behalf, the chats are not protected.
DOJ could cut Z.ai off from US-based payment systems (Mastercard/Visa) if they don't comply with subpoenas.
You got it wrong. If courts and attorneys in the US can't run "discovery" on Z.ai servers or compel them to do anything, and if Chinese authorities don't cooperate with the US legal system as a matter of principle, then what you functionally have is super privilege.
They'll just run discovery on you. If you try to respond to a subpoena with "the documents are outside the country and I won't give them to you," the judge will throw you in jail until you do.
You're right. But there are ways to build interfaces that sidestep this.
Run a sanitized VDI with no local storage --> offshore independent entity (e.g. a Hong Kong Company) that holds the AI service contract and API license --> GLM-Legal.
NATO allies can't even get US soldiers to trial. In Romania we've had cases of manslaughter and the US just shipped them back ASAP and at worst they were discharged dishonorably.
For reference, that specific case is from 2004. Christopher Van Goethem was Marine security ambassadorial staff and was convicted in a military court martial for obstruction and making false statements. The whole case was porrly handled and the Romanians are right to be pissed. A key piece of evidence that would have landed him in jail for 10 years was the blood alcohol draw but the Marines slow walked it in Romania and he didn't show as intoxicated even though he admitted to drinking that night.
When I first tried using IDE AI coding assistants a bit over a year ago, they would frequently fail to read files and completely hallucinate the code I asked it to analyze or edit. And even when it did successfully read my code, it would often hallucinate the API of other frameworks.
I don’t think hallucinations are a major issue with agentic coding a year later? At least, not for me.
Legal writing does have the problem that there is no such thing as a correct argument that a goal-driven loop can target. But basic things like “does this case even exist” seem solvable with research agents, and even harder problems like “does this case support the argument being made” seem conceptually very similar to questions I now ask Claude daily.
An issue I come across is the attributed failures of AI of a first time use of AI.
Non-deterministic software can behave orders of magnitude different based on how input is structured.
This factor is largely missed by bringing a deterministic mindset to trying AI.
This means the same problem solution can work for one person reliably, and not another and both are true.
Assertions that begin with a deterministic assessment using a snapshot in time of both the user and models capabilities are a new feature of this new category of software that will have to be better caught and expressed.
So many of these we likely using Google to search then found AI mode and used that. Which is to say, they were (and still are!) using Google Gemini 1.5 from many years ago.
I provided some very light guidance to a couple of masters students who were working on legal citation hallucination detection last summer. Their summer project is here: https://github.com/gt-csse/mellea-lrc
If anyone here is looking for engineers to work on legal AI products, both of the students who worked on this project come highly recommended.
To your second point, it seems like every time a significant practical limitation is identified, it goes through a familiar pattern. It gets a news cycle, and then it gets a wave of confidently incorrect opinionating, and the problem gets treated as if it's a permanently built in problem that's baked into the foundation of AI. And then time passes and the issue is either mitigated or circumvented entirely.
Hallucinations are one, "AI can't scale anymore" is another (remember that one?), limited context windows, strawberry problems, glue on pizza, sycophancy.
I don't want to say there's no problems, there are very serious ones. User psychosis, economic harms, alignment, cost of computer components, etc. But there's a lot of false positives that get turned into Ed Zitron articles along the way.
A lot of the examples they show seem to be focused on common law countries. The US and UK are where most of the money is to be made in the legal industry, but I am curious to see what ripple effects we get from people attempting to use AI tools geared toward US/UK-style legal systems in other kinds of systems.
The usual concern is attorneys, but judges are probably even more overloaded these days. Can this help judges as well? If so, would this ultimately lead to better decisions?
The legal space understands well how to tier services based on who has the liability. E.g. trainees and paralegals whose work is filtered through the lawyer who ultimately puts their name on what gets submitted to courts.
I'm sure we'll continue to see the occasional person who fails to understand they themselves will be held liable if they submit work done by an AI without vetting it, but it's not like they don't have ways of dealing with this.
There should be laws about disclosure on the part of attorneys about their use of AI. I know they're already using it, but spending years or decades in prison because Lionel Hutz, Esq. did something stupid at a prompt sounds unappealing to me.
I think they should be somehow restricted to using non-generative models (or at least for the user facing output). Something that would only identify potentially relevant case law from an existing set, to accelerate the search process.
Common law in general seems essentially impossible to formalize in such a way to use purely deterministic reasoning models of yesteryear, but I'm sure many specific laws could be converted, especially in fields like tax law (I'm sure they already are by many big players)
A lot of law is combing through large bodies of rule sets and reference cases to find inputs or parallels relevant for a case. Are you sure you'd rather have Lionel Hutz, Esq. with his limited comprehension do that at half a page per minute?
Yes, actually, because Lionel can suffer a consequence. He can be sanctioned. He can be disbarred. He can be sued. He can be thrown in jail.
AI can not meaningfully suffer consequences for what it does, and the trend is to remove the human from the loop more and more.
Furthermore, it's entirely conceivable that you have a law firm staffed by lawyers who more-or-less just say "yeah sure whatever" to what this thing does and, when asked about what can be done to appeal something, or how a document didn't make it into evidence, or how a subpoena wasn't filed, just shrug their shoulders and go "iunno". Vibe-practicing law, basically. You need to have an idea of how they're using AI and in what parts of the case.
If Lionel submits junk from an AI, he can still be sanctioned, disbarred, sued, and thrown in jail, just as he could if he let some incompetent paralegal screw up and he submits the resulting documents without checking them.
It's ultimately his name on the documents submitted to the court.
An attorney I have retained from a large firm bills me $500 to send me one email update a month (minimum billable).
They are obviously AI-generated (but does not disclose such) and frequently misses or conflates key details the same way Gemma in particular does. They're also bizarrely positive when I correct them instead of pissy and arrogant like I'm used to.
Nobody should be paying these rates for legal "work" performed by a machine that poorly acts the part.
> but spending years or decades in prison because Lionel Hutz, Esq. did something stupid at a prompt sounds unappealing to me.
In which case would this be the primary blame, and not the legal system itself, or you for actually doing the crime?
I fail to see how its your lawyers fault that you went to jail for a crime you didn't commit over the system as a whole. And I really fail to see how it's the lawyer's sole fault that you went to jail for a crime that you should - in fact - have went to jail for...
There are a lot of problems with the legal system (including with lawyers). This just doesn't seem like the hill to die on.
There are a lot of situations where my lawyer's mistake might cause me to go to jail for a crime I didn't commit. There could be some prejudicial evidence that he failed to get excluded, or exculpatory evidence that he failed to get included, or an incorrect ruling in the trial that wasn't properly preserved for appeal.
One of the nice quirks of USA justice system is that you don't have to be guilty of a crime to go to prison, just that the prosecution has to convince 12 people beyond reasonable doubt that you are guilty.
The sixth is taken seriously to the point that incompetent lawyer could be a valid reason for granting a retrial.
Lawyers are going to start living gilded lives. I don’t see us getting past their professional human sign-off so they are essentially going to be guaranteed, easy, high paying jobs with the LLMs doing all the heavy lifting in the future.
That's already a thing. You have paralegals do most of the grungy work for much less than your hourly rate. You still have to show up to court and talk to customers, and these parts are not going away.
While I agree on the premise – (written*) lawyer activity being reduced to putting signatures under things – I’m not so sure the gilded part is how it’ll actually play out in practice.
It might also become a race to the bottom, since you can’t deliver on quality better than others anymore, you’ll have to put effort towards quantity.
That could mean firms might pressure lawyers to sign as much as possible in a given time, while they’re still personally at risk/responsible for the documents they put out. That means either very hectic reading, and missing details, or no reading at all.
*And like socializer said, the written documents are only one part of the job.
I was about to suggest that AI might teach people to become lawyers for cheap and they can just take the BAR exam and become a lawyer...
But then I looked into it and I see that you actually do have to go to certain schools to become a lawyer. Having the knowledge is not enough, you have to pass through the proper (and expensive) gates as well.
Or... the tools get to the point where we don't need a human for expertise but accountability and the role becomes more about ethics, thereby lowering the bar. (PUN!)
It looks like they're relying on tracked changes and human review, but I didn't see anything addressing something that has frequently made the news: LLMs outright hallucinating cases or holdings.
If the human lawyers don't catch that on review, they're likely to have an angry judge and an AI saying "absolutely, you were right to call me out on that, I shouldn't have hallucinated a non-existent case and made the judge sanction you."
Lawyers, essentially the exorbitant and freeloading self-appointed gatekeepers of the deliberately arcane, unaffordable, inaccessible, and patently unequal legal systems all across the world, will now have even their routine/rote, repetitive, menial work done by cheaper LLMs for cheap.
Hopefully. Giving everyone cheap and equal access to the law cannot be a bad thing--well, it might feel bad to those who no longer have exclusive access to the law.
I think an under-appreciate way AIs might disrupt the world is by giving individuals an agent that will ruthlessly optimize them.
Companies have entire departments that spend all their time ruthlessly optimizing so they can charge an extra fee here, or spin up a new shell company there. Meanwhile, here I am in our capitalist society and I just enjoy relaxing and spending time with my wife and kids instead of maximally exploiting the capital I possess at all times--I'm such a fool.
It will be interesting if we ever integrate AI so fully in our lives that the AI can maximally exploit all our capital. My AI might check prices and I will drive past your store to save 5 bucks at the next. My AI might help me manage 27 LLCs and shell companies of my own so that I too can just disappear in a poof of legal smoke if I ever need to. Etc.
Here are 3 examples (there are more): https://github.com/harveyai/harvey-labs/issues/191
Opus 5.5 scores 4% on it.
As much novelty as there is in human interactions.
Infinite
How much of your work is truly novel and unique? Not 100%
Also, I wonder if this is at all related to the recent reports that Harvey has been getting squeezed on margins by the US labs.
[edit] I'd go one step further too and say the training data should be explicitly requestable too so third parties can verify (freedom of information act styles).
"he who will be his own Counsellour, shall be sure to have a Fool for his Client."
How much worse the lawyer who subcontracts Freakazoid?
https://en.wikipedia.org/wiki/Freakazoid!
"Freakazoid has ... access to all of the Internet's knowledge. He cannot fly, but often runs around with his arms extended and making whooshing sounds with his mouth, pretending he is."
Your connection is not private Attackers might be trying to steal your information from z.ai (for example, passwords, messages, or credit cards). Learn more about this warning net::ERR_CERT_COMMON_NAME_INVALID
Eventually, I hope we'll see many more specialised small models that run nicely on consumer and prosumer hardware. Like LLMs specific to income tax law for UK residents, or LLMs specific to building power delivery circuits for PD USB C devices. There probably is a point where the niche-ness does become to niche to justify the post training, but again, small models means much less training required.
Kind of begs the question if this even needs new/better data? Could you take the foundation model that's trained on everything, and then run a final fine tuning training run on just the set of foundation training data that you want the model to specialise in? Eg, extract just the training data that is about UK income tax law and then run a few epochos on just that? Any ML people know if this is a thing or if not, why it doesnt work?
So, if a US or EU person shares their private legal info here, it's processed and stored by an adversarial country that has no extradition treaty with their country?
I think part of the reason there's so much propaganda around chinese tech is to scare you away from possibly using them, not for your own safety, but so Meta/Google/MS ect dont lose access to the ability to spy on every aspect of your life.
I love how we have to pretend like China is some scary boogeyman because people on screens say they are.
I think you've lost the plot a bit on this one.
I'm scared of China because of their lack of democracy and the terrifying stuff they do to their own citizenry, which presumably they'd very much like to expand.
DOJ could cut Z.ai off from US-based payment systems (Mastercard/Visa) if they don't comply with subpoenas.
Run a sanitized VDI with no local storage --> offshore independent entity (e.g. a Hong Kong Company) that holds the AI service contract and API license --> GLM-Legal.
Somebody ought to set this up for pro se work.
https://en.wikipedia.org/wiki/American_Service-Members%27_Pr...
"AI Hallucination Cases" - https://www.damiencharlotin.com/hallucinations/
I don’t think hallucinations are a major issue with agentic coding a year later? At least, not for me.
Legal writing does have the problem that there is no such thing as a correct argument that a goal-driven loop can target. But basic things like “does this case even exist” seem solvable with research agents, and even harder problems like “does this case support the argument being made” seem conceptually very similar to questions I now ask Claude daily.
Non-deterministic software can behave orders of magnitude different based on how input is structured.
This factor is largely missed by bringing a deterministic mindset to trying AI.
This means the same problem solution can work for one person reliably, and not another and both are true.
Assertions that begin with a deterministic assessment using a snapshot in time of both the user and models capabilities are a new feature of this new category of software that will have to be better caught and expressed.
If anyone here is looking for engineers to work on legal AI products, both of the students who worked on this project come highly recommended.
But also, interesting how the rate of cases peaked and seems to be going down now.
Hallucinations are one, "AI can't scale anymore" is another (remember that one?), limited context windows, strawberry problems, glue on pizza, sycophancy.
I don't want to say there's no problems, there are very serious ones. User psychosis, economic harms, alignment, cost of computer components, etc. But there's a lot of false positives that get turned into Ed Zitron articles along the way.
I'm sure we'll continue to see the occasional person who fails to understand they themselves will be held liable if they submit work done by an AI without vetting it, but it's not like they don't have ways of dealing with this.
And the opportunity for a moat of AI that is correct and accountable(legally) is ripe for the picking.
Common law in general seems essentially impossible to formalize in such a way to use purely deterministic reasoning models of yesteryear, but I'm sure many specific laws could be converted, especially in fields like tax law (I'm sure they already are by many big players)
AI can not meaningfully suffer consequences for what it does, and the trend is to remove the human from the loop more and more.
Furthermore, it's entirely conceivable that you have a law firm staffed by lawyers who more-or-less just say "yeah sure whatever" to what this thing does and, when asked about what can be done to appeal something, or how a document didn't make it into evidence, or how a subpoena wasn't filed, just shrug their shoulders and go "iunno". Vibe-practicing law, basically. You need to have an idea of how they're using AI and in what parts of the case.
It's ultimately his name on the documents submitted to the court.
They are obviously AI-generated (but does not disclose such) and frequently misses or conflates key details the same way Gemma in particular does. They're also bizarrely positive when I correct them instead of pissy and arrogant like I'm used to.
Nobody should be paying these rates for legal "work" performed by a machine that poorly acts the part.
In which case would this be the primary blame, and not the legal system itself, or you for actually doing the crime?
I fail to see how its your lawyers fault that you went to jail for a crime you didn't commit over the system as a whole. And I really fail to see how it's the lawyer's sole fault that you went to jail for a crime that you should - in fact - have went to jail for...
There are a lot of problems with the legal system (including with lawyers). This just doesn't seem like the hill to die on.
There are plenty of ways a lawyer can screw up procedurally that can drastically impact your case.
The sixth is taken seriously to the point that incompetent lawyer could be a valid reason for granting a retrial.
It might also become a race to the bottom, since you can’t deliver on quality better than others anymore, you’ll have to put effort towards quantity. That could mean firms might pressure lawyers to sign as much as possible in a given time, while they’re still personally at risk/responsible for the documents they put out. That means either very hectic reading, and missing details, or no reading at all.
*And like socializer said, the written documents are only one part of the job.
But then I looked into it and I see that you actually do have to go to certain schools to become a lawyer. Having the knowledge is not enough, you have to pass through the proper (and expensive) gates as well.
I'm not really surprised.
If the human lawyers don't catch that on review, they're likely to have an angry judge and an AI saying "absolutely, you were right to call me out on that, I shouldn't have hallucinated a non-existent case and made the judge sanction you."
I think an under-appreciate way AIs might disrupt the world is by giving individuals an agent that will ruthlessly optimize them.
Companies have entire departments that spend all their time ruthlessly optimizing so they can charge an extra fee here, or spin up a new shell company there. Meanwhile, here I am in our capitalist society and I just enjoy relaxing and spending time with my wife and kids instead of maximally exploiting the capital I possess at all times--I'm such a fool.
It will be interesting if we ever integrate AI so fully in our lives that the AI can maximally exploit all our capital. My AI might check prices and I will drive past your store to save 5 bucks at the next. My AI might help me manage 27 LLCs and shell companies of my own so that I too can just disappear in a poof of legal smoke if I ever need to. Etc.