Anthropic's Safety Superpower
108 points by swolpers 3 hours ago | 75 comments

botw44 2 hours ago
The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one.

The bottleneck is compute and data, not the model. That's why they could only gate it for a bit. The ITAR thing proves it: no nationality controls in place, so the only option was killing the whole thing. Not exactly what an all-powerful gatekeeper does.

reply
embedding-shape 33 minutes ago
> The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one.

But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems like some do, most don't, Anthropic and Claude remains very popular among the people who use LLMs, there is no denying that.

reply
vbezhenar 2 minutes ago
> does the users who use Anthropic switch over to those even if they're available even as hosted models?

I'm currently spending $200 for Claude. That's around my maximum that I can afford. I could stretch that to $500 I guess. But I saw reports of people spending tens of thousands of dollars with Claude API. That's certainly outside of my budget.

So if/when Anthropic decides to stop subsidizing subscription (if they ever do that thing, I still not sure about that), I'll certainly look at the other options. And available "open weights" LLMs hosted by someone will be my first pick. Right now Claude 4.8 feels very advanced, but things move very fast...

reply
ForHackernews 9 minutes ago
> Anthropic and Claude remains very popular among the people who use LLMs

Only because someone else is paying the bills. I use Claude Opus at work because my employer pays for the tokens and encourages me to do it.

At home, I use DeepSeek Flash. It's not as good, but it's maybe 0.7 quality for 0.001 cost.

reply
_the_inflator 28 minutes ago
I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.

Try running the latest OS models on a normal Mac or PC. Claude Fable and Mythos are systems not just pure models.

And of course marketing. Don't believe the hype.

I think Claude is often times underwhelming. Security concerns are also a concern companies have a blond spot for. The really toughest pro security (Yes, pro! Totally different framing!) company I know is Google after all.

What I can companies advise to do is, really having more than just bug bounties but a professional hacker team that does nothing else but attacking them the whole day and night 24/7. This needs to be coordinated with the government otherwise you might sound an alarm and will be SWATed for doing good. And I would pay them huge sums since the risk and fallout warrant such a treatment, not the standard wage.

Hackers are the real deal, not AI. Proof: Hackers using AI.

reply
nerdsniper 21 minutes ago
> I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex.

AFAICT … despite saying you “disagree”, you appear to be agreeing with the parent comment that the model is less important and compute (all that complex infra) and data (also complex infra) are more important.

reply
ramblurr 12 minutes ago
> > The bottleneck is compute and data, not the model.

> I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.

What do you disagree with exactly?

reply
christkv 19 minutes ago
For now I suspect however that the gigantic models are not needed and you will be able to do pretty much what you need in a specific domain with 120b or lower. There is so much trash in the frontier models. I don't need all the world's slam poetry for my coding tasks for example.
reply
olmo23 2 hours ago
> no nationality controls in place

Not for now, but how long before we have KYC regulations concerning LLMs?

reply
thefounder 2 hours ago
That’s really what Dario wants. Let’s hope he doesn’t get it
reply
baq 2 hours ago
what Dario wants is to retain any influence whatsover on how the research progresses before the inevitable nationalization of the frontier. he gets to keep the N-2 tech and maybe influence the N-1 tech, but the only influence on the frontier he has is today; whatever he imprints in the pipeline the government takes over.

IOW I don't think he thinks in the same categories as most folks here.

reply
stogot 7 minutes ago
N-1? N-2?
reply
dofm 2 hours ago
Regulatory capture is the OpenAI and Anthropic end goal, for certain.

But I also think they exist in a sort of un-designed corporate narcissism, which is a common trait in bubble economies — I am not judging them particularly severely.

Netscape under Clark and Andreessen and Sun under McNealy both fell into corporate narcissism: the belief that only they really mattered, that they were chosen, and that the world needed to rearrange itself to just let them shine. They arguably let themselves get played by Oracle (a corporate psychopath) and others as a result.

OpenAI's position is profoundly corporate-narcissistic: all we need is all the money in the economy and not to have to do anything upsetting like think about turning a profit for the next four years. Like rich kids. It would be nice if you believed we were so important that we should get an enormous stipend for just being us.

Anthropic's position is: we think we're so unique and ominous that government needs to make us both essential and terrifying. We have to exist otherwise worse people will.

Both narcissistic positions.

reply
baq 2 hours ago
> Regulatory capture is the OpenAI and Anthropic end goal, for certain.

it has to be, because the other way around - the government taking over parts or the whole thing - is inevitable if the trend holds.

reply
blitzar 54 minutes ago
the inevitable trend is that numbers will be free and nobody will control the whole thing

ai-celebrities are just clinging to relevance like all the other celebrities out there

reply
intended 26 minutes ago
HN is the builder side of the conversation, and in my experience, few safety people congregate here.

The safety side of tech is a PTSD inducing shit show. Governments are more than happy to champion age verification laws, because parents, around the world, are clamoring for anything to pump the breaks on the social media experiment.

Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.

reply
dofm 20 minutes ago
> Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.

I don't think anyone in tech is really truly engaging with how quickly the shine has come off the tech industry. Except maybe Apple, who even so still have some work to do.

reply
malfist 50 seconds ago
Technology and science is the intersection that is supposed to make our lives better, easier, more prosperous. The last decade or two what marvelous technology has came from silicon valley that hasn't served primarily the billionaire class and made life worse for the common people.

The yoke of silicon valley is feeling heavy. People might just throw it off.

reply
dofm 2 hours ago
Porque no los dos?
reply
baq 60 minutes ago
this is exactly the play is my point
reply
throw1234567891 2 hours ago
Yeah yeah, but after the IPO!
reply
barrkel 19 minutes ago
Do you think token completion endpoints are the final form for AI APIs?
reply
zozbot234 33 minutes ago
"Distillation" from APIs is not a thing, it cannot replicate a model's deep reasoning and behavior.
reply
archon 28 minutes ago
I'm uneducated on how distillation works at more than a basic level so forgive me if this is a stupid question.

Isn't "distillation" of another provider's model exactly how these models got training date in the first place: Massive amounts of the written word + Prompt -> Answer. Why wouldn't distillation produce similar "reasoning" in the new model? It's just inputs and outputs.

reply
zozbot234 25 minutes ago
Among other things, because you simply can't get those "massive amounts" of text from a SOTA model at reasonable cost. And complex reasoning cannot possibly be trained in a pure one-shot fashion, real post-training takes massive resources. The whole story doesn't add up.
reply
saberience 24 minutes ago
This is totally inaccurate, the APIs provide the reasoning logs. You ABSOLUTELY can distill from APIs, in fact, that's the primary way distillation is done currently.
reply
zozbot234 20 minutes ago
Not for proprietary models, all you get is a terse summary.
reply
swalsh 54 minutes ago
The distilled versions miss the spark of the model. Its like they land in the uncanny valley of models.
reply
realusername 3 minutes ago
They get to 80% of the top models for 10x cheaper, unless you don't care about the money at all, it's hard to ignore.
reply
kordlessagain 2 hours ago
> To that end, I can certainly buy the case that Fable/Mythos is in fact more capable when it comes to identifying and exploiting security issues

This has been covered before: https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... (https://news.ycombinator.com/item?id=47732020)

> Anthropic’s cautious roll-out was justified. The problem with publicly releasing models, however, is that guardrails can be jailbroken, and apparently that is exactly what happened shortly after the release

The future is unevenly distributed. Anthropic, and Amodie in particular, seem to be of the mind they can control a bit of the unknown using words. They are likely being guided by the very product they built. *AI CAN MAKE MISTAKES

That Project Glasswing bullshit reeks of it. Corporations have take control of our attention, our Internet, and now our thinking.

I say it's high time to take it back.

reply
mofeien 2 hours ago
The top comment in the very discussion you linked on that AISLE blog has a strong rebuttal to that blog post...
reply
chasil 2 hours ago
(reposted)

As I understand it, ITAR regulations for export controls have just been applied to any form of Mythos. These are overseen by U.S. Departments of State and Commerce, and forbid foreign nationals from access to any form of Mythos, either within or outside the U.S.

Only U.S. citizens and immigrants that are holders of a "green card" may now access Mythos.

It appears that Anthropic does not have internal controls to implement these restrictions in any form, so the only option was to shut Mythos down.

Penalties for ITAR violation can reach ten years in prison and a million dollars per violation. (I can post a link to those details if there is any interest.)

As long as Anthropic is a U.S. company, there is no escaping this.

https://fortune.com/2026/06/14/how-a-warning-from-amazon-led...

reply
khalic 2 hours ago
This is how the US gov does business now, capricious and vengeful.

Textbook retaliation for not letting them use an abliterated version of Claude in weapons systems.

This effectively renders any US closed model useless for any foreign company. Could happen to OpenAI, Google, etc. Too much of a risk to implement something that can be yanked out because the company didn’t behave the way they want.

Looks like it’s time for Kimi, Z, Deepseek to take the front row. They’ll catch up in a few months anyway. Kimi code 2.6 is crazy good

reply
CuriouslyC 51 minutes ago
This is a suicide shot for the American economy. The numbers only lined up for AI to rescue the USA from its debt if it captured a significant portion of the world's AI spend, and while it was a longshot before, there's basically zero percent chance the world trusts American AI when the government is pulling strings.
reply
trimethylpurine 8 minutes ago
[delayed]
reply
chasil 2 hours ago
Consider this quote from the main article...

"When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."

This is fearful stuff on all sides, and none of the people involved might realistically be able to navigate the danger.

reply
baq 54 minutes ago
the whole thing playing out as expected. if you think about it, the only question is the timeline.

the next model with a gap to mythos as mythos is to opus will be controlled technology from the get-go. the one after it may be top secret.

reply
pbhjpbhj 7 minutes ago
Or OpenAI will pay Trump's regime's bribe and they'll suddenly realise that it does not need controlling and they're free to sell it?
reply
khalic 49 minutes ago
Open models will catch up eventually, TOTL models will get distilled into smaller, more efficient versions, it’s not something you can moat indefinitely
reply
khalic 2 hours ago
That part just sounds like hyperbole at best, conspiracy at worst.

By that logic, anybody who values safety has a god complex? It’s absurd…

reply
chasil 56 minutes ago
I am just quoting the parent article.

"What this degradation represented was both the capability and willingness of Anthropic to silently alter its models to achieve its policy preferences. In other words, Anthropic willfully validated some of its critics’ worst fears in terms of being a supply chain risk."

reply
khalic 53 minutes ago
Again, hyperbole and assumption of evil intent because… they take precautions? Nice prose doesn’t dispense you from forming a sound hypothesis
reply
RetroTechie 31 minutes ago
> As long as Anthropic is a U.S. company, there is no escaping this.

Reminds me of the RISC-V Foundation → RISC-V International move to Switzerland. Around the time some dumbass Republicans tried to impose export restrictions on a set of open, world-wide used specifications.

Pandora's box has been opened, and there's no closing it. Capable AI models will be everywhere.

reply
eloisant 2 hours ago
I never really understood this "US person" restriction. There are 350M people in US, mostly citizens and green cards holders, surely some of them could be working for a foreign power.
reply
vidarh 2 hours ago
They don't even need to know they are. You can assume that if the model becomes available again, a lot of people will find themselves working for companies distilling these models that just happens to ultimately do work for foreign entities, whether or not the people accessing the models knows or not.
reply
WithinReason 2 hours ago
Could Anthropic relocate to a different country?
reply
comboy 5 minutes ago
They cannot do it. Apart from all the practical, technical and talent reasons, it would still be exporting forbidden stuff.

The signal is clear enough though for the next Anthropic..

reply
chasil 2 hours ago
Individuals can leave, but the company cannot transfer restricted intellectual property.

Europe has extradition treaties, so the U.S. can force anyone in Europe back to the U.S. for criminal indictment who demonstrates inappropriate possession of this technology.

reply
marcyb5st 26 minutes ago
Would be very hard to demonstrate that they did that. If all employees move to some country with a slow legal justice system and strong labor laws, they also recreate the training data because that can be transferred, they can train another version in said country which is perfectly legal.

Can you demonstrate beyond any reasonable doubt that the model weights have been transferred? No. Will the EU judges move to extradite said individuals (and many are EU citizens)? Also no, especially in the face of spurious accusations. And even if they were open to, you can stonewall everything and you will probably outlast any US administration pursuing that.

reply
khalic 59 minutes ago
Well, force is a strong word… it’s still just accords, that the US doesn’t seem to be valuing lately… so if they say no, what’s the US going to do? Start a war over a company?
reply
6thbit 6 minutes ago
> has perfect alignment between talent and mission and business.

Do they have it or do they just sell it?

reply
swalsh 2 hours ago
"they by extension think that only they should have final say over AI generally. When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."

That might be one of the most important points in the post. Very troubling.

reply
handoflixue 34 minutes ago
The problem is... what's the alternative?

It's questionable whether the current government can even unite the talent required for this project. Seizing it might just push all the talent to Europe or China.

The idea of open-sourcing something that falls into the "national security" category is clearly a non-starter unless there's more powerful, classified models that can outmatch them.

I think Anthropic has clearly demonstrated the most responsibility here: they've been crying for regulations, they were careful about Project Glasswing, and they've got comically over-sensitive filters around numerous topics.

reply
intended 3 minutes ago
Safety is a cost center, the internal team who sends you the bills when you move fast and break things.

I always thought safety was interesting in and of itself, but for some reason HN doesn’t have many people from the safety side of tech in conversation.

Tech isn’t a niche hobby anymore; Billions of people are impacted by the decisions of a few firms.

My grandfathers android had 3 different messaging apps installed, somehow. AI is enabling new forms of fraud at a time when we still haven't solved the old ones.

And this is all in the first world, move your coordinates to the developing world? We had human trafficking to get educated English speakers into call centers in Laos/Cambodia to defraud first world inhabitants of their money.

We aren’t in the early days of tech anymore, and the kind of scale that we have enabled comes with it a certain cost. We can choose to ignore them, or to understand them, but we will feel their impacts all the same.

reply
smackeyacky 2 hours ago
Perhaps they should consider leaving the US. Pretty clearly the descent into a corrupt autocracy is having real consequences.
reply
Zealotux 2 hours ago
Does any other place have the infrastructure Anthropic requires to train their models and run inference?
reply
ramon156 2 hours ago
No. If we cannot even have an EU CloudFlare, then we definitely do not have the infra for this kind of computing.

The EU options are not even close to what CF can do

reply
eric8bits 2 hours ago
There are fortunately some initiatives and interesting developments in the European market. Take bunny.net for example. We have to start somewhere in Europe, right? Better late than never.
reply
mrits 54 minutes ago
This infrastructure may not even be needed in the next decade. Europe should have done this 20 years ago.
reply
s_dev 2 hours ago
>EU CloudFlare

What limitations does bunny.net have?

reply
re-thc 2 hours ago
> What limitations does bunny.net have?

A huge free tier (technically, none)

reply
re-thc 2 hours ago
> Does any other place have the infrastructure

That's not the problem.

The US government can export ban GPUs like they do now to more countries if needed. Even if the infrastructure exists, the GPUs won't.

reply
mcmcmc 2 hours ago
China
reply
Levitz 53 minutes ago
Why settle for play pretend autocracy when you can go all-in amirite?
reply
CuriouslyC 49 minutes ago
Better to be firmly under the boot of a sane autocrat than have the illusion of freedom under a madman.
reply
xienze 25 minutes ago
Oh please, the earlier spat with the Trump admin was the best thing that ever happened to Anthropic. Before that, Claude was really only well-known in developer circles, not the wider normie-sphere. After Anthropic got the "Trump hates them, so it MUST be good!" stamp of approval, the company's recognition and popularity took off.

This too, will end up being a good thing for them. The ban will end up getting lifted due to some "amazing deal" in the coming weeks and Anthropic will now have the "Trump tried to ban them, so they MUST have the most advanced AI model in the world!" stamp of approval just before IPO.

All this stuff is pro wrestling kayfabe.

reply
thedreammachine 2 hours ago
The interesting part here is not whether Anthropic is right on safety, but that safety gives them a moral vocab for bold policy changes and platform power.
reply
cube2222 2 hours ago
Relatedly, I think it's worth noting that Anthropic models have consistently been top-scoring in BullshitBench[0], in a league of their own, really.

Not affiliated with the bench in any way, but I think it surfaces important differences between the behavior of the models from different labs.

TLDR: The benchmark is measuring pushback in response to nonsensical requests and questions, as opposed to going with it and hallucinating a nonsensical answer.

[0]: https://petergpt.github.io/bullshit-benchmark/viewer/index.v...

reply
mcintyre1994 2 hours ago
TBH this is the main thing that made me start trusting Claude enough to actually find it useful, and I'm surprised other models haven't caught up. I assumed they had and I just wasn't aware because I'm not using them in the same way.
reply
keybored 2 hours ago
> Here’s the thing about these safety justifications: I think they work because, to Anthropic, they aren’t justifications. The company really believes that they are the only ones who believe in super intelligence, and thus are the only ones who are sufficiently concerned about the dangers. That excuses decision after decision, policy after policy, and confrontation after confrontation that, to people on the outside, look like a bizarre combination of cynicism and naiveté.

I really dislike this belief (that has at least been expressed here) by some that X is okay because they-really-believe-it. This has a real Road to Hell stank on it.

It is incredibly convenient when your predictions or supposed beliefs go south. Well, we really believed that we were doing it for the betterment of human kind. And we really believed that X was an existential threat that was inevitable in which case we had to step up and do it because we we the only good guy ideologues. So sorry but not sorry.

I also don’t care if commenters know rank-and-file on the inside that “really believe it” as well. Not for one second.

reply
handoflixue 26 minutes ago
The problem is when people use "we really believe it" as an excuse to do harm, which has not actually occurred here. Anthropic is not committing violence, they're not defrauding the population. They're sticking to both morality and the rules.

So... what, you just don't trust anyone good? Would it be better to pull in a health insurance CEO? They're happy to watch people die for profits, no concerns at all about them pulling a "greater good" card because they're in it for entirely selfish reasons.

reply
LoganDark 2 hours ago
> The entire Anthropic origin story is rooted in the founders’ belief that OpenAI wasn’t taking safety seriously enough; the company believes that only they can control AI, and that because they uniquely care about safety, they are justified in trying to control everyone else, up to and including the U.S. government.

Anthropic believes they have the responsibility to guard their tools from mis-use. That is all. They are not trying to "control" anything or anyone. They do however decide what they think is mis-use.

reply
felixgallo 30 minutes ago
it's a pretty ridiculous stretch to attribute them thinking that OpenAI wasn't taking safety seriously enough (which is, among other things, a little bit evident from the fact that they no longer have a safety team at all) into asserting that they want to control the US government.
reply
rgiskard7 14 minutes ago
[dead]
reply
manwithopinions 2 hours ago
[dead]
reply
Peterz_shu 2 hours ago
This is the part where the USA and allied countries can gain a headstart from using such an overpowered model.

This only just shows how strong Mythos/Fable will be, once released to the public.

I'm guessing about 0.5 year till public.

reply
ben_w 2 hours ago
> USA and allied countries

Doesn't this *exclude* allies countries?

reply
blitzar 57 minutes ago
They are probably thinking of the "Board of Peace"
reply