I am a lawyer myself. I recently had an order that practically looks and smells ai generated.
I sent the output to Turnitin and it returned almost every line turned out ai generated.
Heck, 4 paragraphs were the same text reworked as a filler.
I filed a second appeal and the case has not been picked up.
My question is, if the judge is using an ai to decide a case, and the judge has outsourced their "judgement" to ai, I, as a lawyer should be able to influence that.
One could argue " well that's the judge's discretion, who decides" but I don't.
When I appear for a client before a judge, I "expect" that judge to apply their own mind, and only their own mind without any bias. Judge offloading their rights imo renders them biased and I should be able to influence the other person (or the ai)
Not that I have done it before. Just saying
State of art ai shouldn't beat me in court. I want a better lawyer to beat me in court. I want a human judge who has experience to "judge" between the two sides, based on evidence and arguments and law. Thats all I am asking. Is that too much?
If there is an opposing state of art ai, I would deploy my own so who wins?
I am using Claude, my opponent is using Claude, judge is using Claude so what are we even doing?
Not having read the article, but I feel this is a risky stand for the court to take. It makes sense that you should stop others from impeding your review system - however you implement it, yet "prompt injection" is hard to define. What is stopping the court from fining people for a "logical argument" in future filings?
But Brasil court doesn't work under precedents. So unless this becomes law, it doesn't change anything for future cases.
German courts official doesn't work under precedent either. But in practice it often behaves like it does. Eg very prominently discussed in the media for judgements around renting of residential property.
In the other direction, in a common law system it's up to the judge to decide what precedent, if any, applies. You can always find some peculiarity of the case that justifies deviation, or you can ignore some peculiarity to justify following a precedent you do like.
The higher court that handles appeals.
That’s the generic answer to “what is stopping the court from…” in a functional judicial system.
Seems pretty easy to me. Any injection attack is an attempt to exploit technical weakness by injecting commands into data streams.
Is the problem not they're using AI for review and court decisions that can easily get hacked? If I open ports that can get hacked or allow sql injections I'll lose my job straight away tomorrow, but apparently the justice system is immune from that?
For any reasonable definition of "prompt injection", that pretty clearly is one.
I’m not a lawyer, but I’m curious if it would also be a fineable offense to ask for this directly. Obviously nobody would ever do that… it would come off as someone thinking they had Jedi powers or something. But it doesn’t seem to me (again, not a lawyer) that this would be illegal.
So why is it when you ask it of AI?
Asking an LLM to do this through a prompt injection attack is less an unreasonable request and more an active effort to undermine the court's infrastructure and processes.
I'm not a lawyer; that's my intuition on the matter.
There are probably certain humans who would follow commands as given, perhaps a child or child-like?