German ruling declares Google liable for false answers in AI Overviews
151 points by ahlCVA 3 hours ago | 71 comments

Swizec 2 hours ago
Good. The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS. Same as it works with employees.

Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.

But watch as Germany soon loses AI Google results.

reply
sva_ 43 seconds ago
It doesn't "bury [it] deep in their TOS", it says right under the box:

> AI can make mistakes, so double-check responses

reply
JumpCrisscross 42 minutes ago
> Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger

Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)

reply
heathrow83829 19 minutes ago
at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth. what if there's a 1/1000 chance of some error, then the company could be sued millions of times per day.

down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.

reply
em-bee 10 minutes ago
the user needs to use some judgement on the information they receive and have some critical thinking skills

how?

errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.

google would be right to remove all AI results from germany

i'd consider that a win.

reply
gmerc 13 minutes ago
So scale of harm creates immunity, is that the argument ?
reply
necovek 6 minutes ago
Did we not already see that with the financial sector in 2008?
reply
drfloyd51 6 minutes ago
Too big to fail. Lol.
reply
onetokeoverthe 6 minutes ago
[dead]
reply
Swizec 12 minutes ago
> at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth

If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”

reply
Dylan16807 48 minutes ago
Why would that mean AGI? You can get into liability-accepting territory by restricting scope, a lot easier than by making your AI smarter.

Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.

reply
wombatpm 19 minutes ago
Wasn’t Tesla found to have FSD disengaging just before a crash so that the driver would be at fault?
reply
wisty 35 minutes ago
Banning all technology because someone might misuse it is an illogical extreme.

As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.

A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).

reply
MichaelZuo 53 minutes ago
Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.

Otherwise most of it would not even exist.

Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).

And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.

reply
eqvinox 14 minutes ago
> Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.

A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.

Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.

reply
TalkingCodeMonk 4 minutes ago
That is a false dichotomy. The solution to failed laws and regulations is not crime and corruption. The solution is to hold the policial and business leadership accountable; to fix the laws and regulations.

The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.

Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them for profit, ad infinitum.

Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because you wrote that in the T&C's, its a digital product, or your doing everything behind the liability shield of an LLC, doesn't change that.

What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated by corporate sociopaths they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one).

reply
throwaway27448 36 minutes ago
> The true mark of AGI

Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results

reply
kevinxsun 34 minutes ago
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. I wonder how many victims are there now.
reply
cmiles8 2 hours ago
Companies generally are liable if their product doesn’t perform. No reason AI should be any different.
reply
clear-octopus 35 minutes ago
[dead]
reply
jjcm 8 minutes ago
What constitutes a correct answer though?

Is something like,

"People online say that x y and z because a b c"

a credible, correct answer, even if it isn't because of a/b/c?

reply
why_at 27 minutes ago
I agree with the ruling, but this makes me wonder if it will be possible to have any AI agent at all if it's consistently applied.

After all, if I can get ChatGPT or Claude to say something false that should count too, right?

reply
eqvinox 4 minutes ago
In normal flow yes, but likely not if you intentionally entrap it to say something wrong.

A disclaimer and couched language will probably fly through. And it's going to matter what expectations an user could reasonably have, too.

reply
kevinxsun 34 minutes ago
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. If you are a victim too, reply below.
reply
benoau 3 hours ago
> In this case, Google's AI had wrongly linked two publishers to scams and shady business practices.

Guess that's the end of their AI overviews in the EU!

reply
incompatible 2 hours ago
You'd think so, along with other countries that have defamation laws. But there's no indication of any penalty, and Google wasn't even made to pay all the legal fees. Perhaps their business model (if there is any) can cope.
reply
dawnerd 45 minutes ago
And lawyers will use this case to build a better defense next time.
reply
CamperBob2 31 minutes ago
In sane countries, it's enough for them to post a disclaimer ("This is AI. AI can make mistakes. Check all results.") Which is what they do.

Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.

reply
oowa 2 hours ago
that would be hela curryketchup nice
reply
donaldjbiden 3 hours ago
It depends if Google feels the profit is worth the risks.

What profit? I don't know either but they enabled this for a reason right?

reply
missedthecue 18 minutes ago
If companies can be held liable (in spite of very visible disclaimers, ToS, and usage policies) for the output of non-deterministic software, isn't this just a soft ban on the deployment of non-deterministic software?
reply
Frieren 10 minutes ago
> (in spite of very visible disclaimers, ToS, and usage policies)

If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.

Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.

> isn't this just a soft ban on the deployment of non-deterministic software?

Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.

reply
em-bee 7 minutes ago
no, just a ban on using non-deterministic software for situations where deterministic responses are expected.
reply
MrBuddyCasino 5 minutes ago
What to do if the software automatically and wrongly libels you on a public search engine?

Honestly I can understand the ruling, but the side effects might be severe.

reply
eesmith 9 minutes ago
Where did it say the liability only applied to non-deterministic software?
reply
keyle 2 hours ago
I'm surprised this is even a thing. After all, you go to Google not for the truth, but to search Google. Since when is truthiness the "guarantee of service"?

You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.

I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.

But I'm surprised that they lost this argument, and the line they took in the first place.

The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?

reply
cortesoft 2 hours ago
That is exactly the point of the ruling, though... they are saying that AI summaries are NOT the same as search. If Google was just returning search results, and then users clicked on a website and read the content there, Google is not responsible for the content.

If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.

That makes sense to me?

reply
foolfoolz 2 hours ago
why? tons of websites push misinformation intentionally. is there a truth requirement anywhere? i don’t get why this is a thing at all
reply
burpingtree 2 hours ago
What don’t you understand? Those websites that defame a company are liable for that defamation. In this case Google defamed a company in its AI summary and is this liable for that defamation.
reply
riskd 53 minutes ago
So if I edit a Wikipedia article to share that consuming poison is safe and someone consumes poison after reading it… is Wikipedia legally liable?
reply
eqvinox 24 minutes ago
> is Wikipedia legally liable?

Probably not, because it's a similar situation where Wikipedia is accumulating user provided content. And people know Wikipedia can be freely edited.

You, however, might be liable. It's your content.

reply
anematode 47 minutes ago
No, because Wikimedia isn't responsible for the behavior of its editors.
reply
JumpCrisscross 41 minutes ago
> is Wikipedia legally liable?

Directly? Quite possibly. They'd then have to transfer that liability to you.

reply
vor_ 55 minutes ago
> is there a truth requirement anywhere?

Yes, and it's called defamation when you don't follow it.

reply
etchalon 40 minutes ago
There is absolutely a truth requirement.

This is why you have to say "I think this person is a murderer" and not "This person is a murderer."

One is opinion. One is fact.

This isn't super hard.

reply
msy 36 minutes ago
That's the difference between returning search results and interpreting the information and summarising them. If a newspaper says 'so-and-so has been arrested for theft' it's not the same as them summarising to 'so-and-so is a thief', the second is potentially libel. Why should Google be held to a different standard?
reply
SXX 52 minutes ago
Google itself is more trustworthy from a normal person perspective as they use it a lot.

None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".

So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.

reply
trollbridge 28 minutes ago
I go to Google to search, but get spammed as if I wanted to talk to a chatbot (and a very poor quality chatbot at that).

This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.

reply
weird-eye-issue 2 hours ago
I think it's very clear that Google's AI overviews go far beyond just searching Google because they often incorrectly compile sources to come up with an incorrect answer. For example of this look at the comment I made in this thread
reply
why_at 40 minutes ago
The title is misleading IMO. It should say "German ruling declares Google liable for libel in AI Overviews"

I was prepared to say the same thing as you but after reading it seems totally fair.

The key difference is that this would be illegal if a human wrote it too.

reply
BikDk 2 hours ago
Playing the perception game wins you the perspective price.
reply
sourcegrift 2 hours ago
Nothing is free. Google benefits off you when they show you search page. Either today (ads) or later
reply
tristanj 2 hours ago
Anyone know if this ruling applies to answers generated by AI chatbots, such as ChatGPT/Gemini/Claude?

All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.

reply
Kina 2 hours ago
> Pretty much the exact thing that Google AI Overview does.

No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.

> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."

> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."

Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”

reply
sandeepkd 2 hours ago
> You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway

There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.

This is probably the right step in some sense to make one liable for their statements/assertions.

reply
asdfaoeu 22 minutes ago
This ruling was about search clearly, however, there's definitely ways implications for chatbots too.
reply
incompatible 2 hours ago
Apparently, if they were search results, they wouldn't have been liable, since there's an exception to defamation laws. Without any exception, defamation is defamation, it doesn't matter how it's presented.
reply
Kina 2 hours ago
Yes, if it was that a search result returned a defamatory article that Google had nothing to do with outside of indexing, it is likely they would not be found liable. The court is clearly trying to make a distinction that the AI search results are produced by Google and thus they can make an editorial decision on whether to publish it despite knowing that it is potentially defamatory.
reply
asdfaoeu 30 minutes ago
Google does remove defamatory results I believe at least partially in response to being sued. However there is a distinction if they have been informed it is defamatory.
reply
incompatible 2 hours ago
I don't see any reason why it wouldn't.
reply
weird-eye-issue 2 hours ago
I have a business where our support email is recommended when people are searching for how to cancel a completely unrelated scam subscription that is showing up on their bank or credit card charges. We get emails almost daily from confused people.
reply
cm2187 49 minutes ago
Doesn't libel require to be deliberate? Ie you can't sue for libel if the author admits a mistake and corrects it?
reply
bmandale 26 minutes ago
It requires the claim to be made with "willful disregard for the truth". Notifying someone, especially with a cease-and-desist on fancy letterhead, makes it legally clear that they know better, and thereafter would be definitely libelling you (assuming the claims are in fact untrue and harmful). But you can still sue them for the claim prior to the notice, you just have to prove they should have known better prior to making the claim.
reply
asdfaoeu 37 minutes ago
In this case it looks like they were notified and didn't do anything.
reply
razorbeamz 43 minutes ago
That's just America.
reply
Heirlomb 2 hours ago
Some digital matters concern the state and others are private and there should be no sovereignty of the state over private matters.
reply
kg 59 minutes ago
Who will adjudicate private disputes between citizens if not the state?
reply
tjpnz 56 minutes ago
Does this extend to ads displayed in search results? Because they absolutely should be liable for the scams they advertise also.
reply
JumpCrisscross 39 minutes ago
> Does this extend to ads displayed in search results?

Probably not, for the same reason search results aren't an issue.

reply
dyauspitr 17 minutes ago
So stupid. What is this with making perfect the enemy of good. You can never guarantee the output of an LLM does that mean Germany does get to use them?
reply
russellbeattie 43 minutes ago
I've found a fun and pretty reliable way to get Gemini to output incorrect information: Ask for a chapter by chapter summary of a book.

I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.

Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.

Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.

If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.

All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.

reply
wyager 32 minutes ago
EU countries continuing to ensure the conditions for their future economic competitivity
reply
unliftedq 2 hours ago
[flagged]
reply
maxdo 3 hours ago
Their digital sovereignty
reply