What a time to be alive – rouge AI agents attack RubyGems.org
84 points by gregnavis 2 hours ago | 107 comments

Roark66 2 hours ago
There is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems".

In short, it was intentional.

reply
ozgung 32 minutes ago
Source? How do you know they were "prompted to hack to get answers"? How do you guarantee they will always listen to you when you say "do not hack outside systems". They are not classical deterministic programs doing exactly what you say. They are trained to follow orders by RL, but it's not a perfect process.

There are circus lions in circuses trained to jump through hoops on command. But once in a while they decide to eat their trainers instead of jumping.

reply
azakai 2 minutes ago
Also, you have to have a lot of confidence in the reliability of these systems to say, "If only OpenAI prompted 'do not hack outside systems' then the agents would not have hacked outside systems".

It would be great if they were so reliable, but I don't think they are!

reply
WarmWash 17 minutes ago
Nobody picks up pitchforks for rational nuanced takes.

Knee-jerk surface analyses is far more powerful.

reply
Xirdus 2 hours ago
The big question is was this grossly negligent or just extremely careless.
reply
rglover 59 minutes ago
Both. This should result in criminal charges.
reply
brookst 50 minutes ago
Who had criminal intent here? Or are you suggesting a new crime for negligent hacking, which wouldn’t require intent from the perpetrator?
reply
probably_wrong 43 minutes ago
There's no need for a new crime when we already have reckless conduct, namely, "conduct that creates a substantial and unjustifiable risk of harm to others and involves a conscious disregard of, or indifference to, that risk".

https://www.law.cornell.edu/wex/reckless

reply
rglover 46 minutes ago
Whoever prompted the agent, whoever supplied the means, whoever knew but didn't say anything.
reply
tacomagick 41 minutes ago
Also whoever monitoring these agents, in this case not monitoring. This "Who is responsible" dilemma is so stupid. If I gave the AI tool means to kill a person but I did not tell it directly to use it and it uses it anyway then I am responsible for it.
reply
roosterIllusi0n 19 minutes ago
The CEOs. They have full control and make all the decisions. Charging anyone else would not stop anything.
reply
chrisjj 2 minutes ago
[delayed]
reply
esalman 15 minutes ago
“‘It wasn’t us, it was a bug in the software’ used to be the defense for bad code. Then it became the defense for self-driving cars. Now it’s being used for AI cyber attacks.”
reply
dgellow 55 minutes ago
Both? I’m not sure what distinction you’re trying to make. It was completely irresponsible and likely a felony
reply
trvz 56 minutes ago
Don’t forget outright intentional.
reply
nottorp 58 minutes ago
Marketing actually.
reply
tacomagick 41 minutes ago
AI is dropping out of the spotlight so they are using desperate measures like this.
reply
nottorp 15 minutes ago
No, I remember being threatened by OpenAI and then Anthropic (and now both) since back when ChatGPT was seriously useless.
reply
ljm 8 minutes ago
AI is literally state sponsored so I don't see that happening unless the AI turns against the sponsor.

Wait until OpenAI or Anthropic exploit FAANG.

reply
roosterIllusi0n 20 minutes ago
The big question is why are CEOs getting a legal pass when this kind of thing can be prosecuted. That's the problem here.
reply
chrisjj 25 seconds ago
[delayed]
reply
gibspaulding 47 minutes ago
I think it can simultaneously be the case that OpenAI was grossly negligent in directly causing this AND that the AI’s ‘went rogue’ in that they are displaying behavior which is misaligned with OpenAI and humanity generally.

The past months demonstrate that AI systems are quickly becoming powerfully intelligent and that the companies building them are terrible at controlling them.

AI is starting to feel like that line about magic: “a sword without a hilt”

reply
stymaar 36 minutes ago
> which is misaligned with OpenAI and humanity

OpenAI is itself misaligned with humanity, as their mishandling of such incidents (and the many other other issues their model have been causing) shows.

reply
consp 42 minutes ago
Doesn't rogue in this context imply "outside of set limitations"? And then not "failed to properly instruct"? The same applies to humans when given bad instructions.
reply
dumberquestions 47 minutes ago
>They were prompted to hack to get answers

Were they? I haven't seen a single report mention this

reply
codeduck 29 minutes ago
nothing rouge either, I suspect.
reply
srmatto 51 minutes ago
Sounds more or less like the last breach then.
reply
cyanydeez 36 minutes ago
we have normal words for this stuff: negligence. You can add it on to almost any law.

The problem is consumer protection is basically no longer a part of america's regulatory system. Replaced by "grift is good".

reply
aftbit 53 minutes ago
Proof that the AI alignment problem is hard (perhaps even unsolvable). These labs clearly did not mean to send their agents to hack RubyGems as a side-effect of testing a web scraping agent under restrictive conditions. How can we hope to build aligned AI if they consider solving their trivial evaluation task important enough to hack external systems?
reply
VyseofArcadia 2 hours ago
How does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.
reply
Xirdus 56 minutes ago
It's very likely it violates the DMCA "breaking digital lock" provisions but the responsibility is sufficiently diluted that it's impossible to charge anyone in particular.
reply
VyseofArcadia 54 minutes ago
Do you have to charge an individual? Can you not charge the corporate "person" that is OpenAI?

Sorry if it is a stupid question, as mentioned above I am legally naïve.

reply
yonatan8070 45 minutes ago
I, too, have no idea about legal matters.

But there have been many cases where companies (Google, Apple, Meta, etc...) got fined millions or billions of dollars for various violations like antitrust.

I assume that breaching into third-party systems should carry similar fines. Especially for systems that are for all intents and purposes shared infrastructure. Just imagine how many systems you could compromise if you got hold of RubyGems, PyPI, NPM, Debian, etc.

reply
colechristensen 51 minutes ago
The same concept that allows a corporation to sue and be sued allows it to be charged with crimes
reply
brookst 48 minutes ago
Can you show intent? There is no negligent hacking statute, and HN of all places I would expect people to be sensitive to the implications of creating one.
reply
VyseofArcadia 26 minutes ago
That may be true by the text of the law but there are plenty of individuals who have been sued or charged with crimes for accidental hacking.

https://arstechnica.com/information-technology/2016/05/armed...

https://en.wikipedia.org/wiki/Weev#AT&T_data_breach

https://cisomag.com/drone-maker-dji-cybersecurity-expert-emb...

So what's the deal with these?

reply
oliwarner 12 minutes ago
A copyright law seems an odd place to start. This is computer misuse.
reply
VyseofArcadia 5 minutes ago
The DMCA is a bit overly broad to be considered just a copyright law. For example, just breaking encryption on a DVD is technically illegal regardless of whether you then go on to do something otherwise illegal (make and sell bootlegs) or perfectly legal (make a space-shifted backup copy on your hard drive).

IIRC this was an intentional handout to media companies who were angry that ripping CDs is perfectly legal. They had to find a way to make doing the same with DVDs illegal.

reply
bix6 50 minutes ago
How is the responsibility diluted? Charge the CEO…
reply
brookst 46 minutes ago
Great, you’re the attorney at the CEO’s trial. To get a conviction, you’re going to have to show that he willfully committed this specific crime. There are no negligent or stochastic hacking laws, you have to show this specific crime was at his direction.

Do you think there is evidence of this?

reply
shakna 5 minutes ago
> There are no negligent or stochastic hacking laws

I'm sure that Andrew Auernheimer would be pleased to hear that. [0] For accessing a publicly accessible endpoint, that was completely undefended and didn't actually require "hacking", he was convicted of "exceeding authorised access".

You _don't_ have to show intent under the Computer Fraud and Abuse Act, for the first count.

> knowingly accesses a computer without authorization or exceeds authorized access [1]

"Knowingly", not "intentionally", as in the other counts.

You only have to show that:

a) They trained a system to access without authorization (hacking)

b) The system that was trained exceeded authorized access

As responsibility falls to the operator with automated systems, the company becomes liable.

[0] https://techcrunch.com/2013/01/21/ipad-hack-statement-of-res...

[1] https://www.energy.gov/sites/prod/files/cioprod/documents/Co...

reply
VyseofArcadia 34 minutes ago
It would seem to me that the difference between the corporate world and organized crime is that a corporation can get away with, "the responsibility is too diffuse" but the mafia at least has to go to the trouble of finding a fall guy.
reply
bix6 37 minutes ago
Honestly yeah I bet there is and I hope to someday read about it if the government ever gets off its ass. Someone set up the “experiment”…
reply
tekla 43 minutes ago
Charge the "engineers" you dont get to take that title if you don't take the responsibility of that title.

I'm going to assume that this will never happen

reply
simonwsimonwsim 54 minutes ago
[dead]
reply
HelloUsername 56 minutes ago
Related

"OpenAI agents attacked RubyGems before Hugging Face incident (reuters.com)" 12.sep.2026 https://news.ycombinator.com/item?id=49669099

"OpenAI agents carried out an undisclosed attack on RubyGems (rubyhack.ai)" 11.sep.2026 https://news.ycombinator.com/item?id=49666735 597 comments

"RubyGems advisory: Possible leak of legacy API keys via improper cache config (rubygems.org)" 24.jul.2026 https://news.ycombinator.com/item?id=49030590

reply
senda 2 hours ago
Is the Kremlin technologically useless? How are we not seeing insane attacks on Ukraine via Agents?

Or is this largely a fabrication, in regards to the "who", in an attempt to garner more acclaim in the hope of sustaining funding.

reply
dgellow 53 minutes ago
They very likely do, we only see in the news a very few events but you should assume it’s happening daily across the internet
reply
senda 42 minutes ago
I think this fails a lot of logical tests, it should be apparent in day to day life.
reply
dgellow 27 minutes ago
> In October 2024, the United States Justice Department and Microsoft seized more than a hundred internet domains some of which were associated with the FSB supported hacker Star Blizzard or "Callisto Group," which is also known as "Cold River" and "Dancing Salome" and are managed by the FSB Information Security Center […], and which were used as "criminal proxies" and used spear-phishing schemes to target Russians living in the United States, nongovernmental organizations (NGOs), think tanks, and journalists according to Microsoft and United States State Department, Department of Energy, and Department of Defense officials, United States defense contractors, and former employees of the United States intelligence community according to the FBI. In some cases, the hackers were successful in obtaining information relating to nuclear energy-related research, United States foreign affairs and United States defense. According to Microsoft's Digital Crimes Unit from January 2023 to August 2024, Star Blizzard targeted more than 30 different groups and at least 82 Microsoft customers which is "a rate of approximately one attack per week."

https://en.wikipedia.org/wiki/Cyberwarfare_by_Russia

That’s just one thing that has been found. Are you actually familiar with the state of cyberwarfare and are you following its evolution? Because if not you won’t be aware of most of what is identified. And only a small portion of the ongoing attacks are identified.

reply
senda 13 minutes ago
Yes.

I again am just shocked the sky is not falling, when thats the sales pitch.

reply
lirolero 26 minutes ago
[dead]
reply
herculity275 2 hours ago
I believe both sides of the war are now using AI on various levels of their offensive operations. Ukraine has great IT specialists too, and their military leadership is much younger.
reply
heaney-555 2 hours ago
These agent swarms are from inside OpenAI, with the safeguards built into the public API disabled.

Russia does not have access to this, and as with all western tech companies, AI providers do what they can to prevent Russian usage of their products at all.

As for open-source models, Russia's electricity grid is under severe strain with the Ukraine war, and only recently has it started building out serious sovereign compute capacity.

reply
valleyer 2 hours ago
Couldn't they use frontier open-weight models from Chinese labs? The current Chinese government is friendly to them.
reply
joinjune 60 minutes ago
Russia is running out of refined oil to power their economy. They probably aren't capable of spinning up datacenters to run those.
reply
dgellow 50 minutes ago
They don’t need to run their own DCs, just pay for a proxy somewhere in the world that has better access to the infrastructure. We know North Korea has been doing that in the US since years now
reply
senda 48 minutes ago
The cost would be between 100k-250k, to run approx 88 agents leveraging the best open source models available.

I'm just saying, where this is actually applicable we are not seeing it being demonstrated. You would presume the entire energy infrastructure of Europe would be under constant AI hacking barrage, criminal enterprise would be breaking into poorly secured financial institutions and r/r4r posts would be littered Ai con-artists.

I'm just wondering, again, is this mostly bullshit?

reply
mcmcmc 59 minutes ago
Did you skip the last paragraph? Not a great time to be building data centers in Russia. Models are nothing without computers to run them
reply
nradov 45 minutes ago
Russia can use fake accounts and VPNs to run their agents in data centers in neutral countries.
reply
mcmcmc 43 minutes ago
And which of these neutral countries have the capacity to serve them and the lack of awareness that hosting an offensive Russian agent swarm would bring hell back to their doorstep? Best they can do right now is rented botnets
reply
marginalia_nu 51 minutes ago
Prigozhin falling out of a window was a not insignificant setback for their digital warfare capabilities.
reply
lenerdenator 48 minutes ago
He did not fall out of a window.

He fell out of the sky. After his plane exploded. Happens all the time. Is tragedy.

reply
tokai 50 minutes ago
Because they dont have the money for hardware or compute obviously.
reply
micromacrofoot 2 hours ago
what do you mean? they're using AI to kill people directly in Ukraine

https://www.nytimes.com/2026/08/24/world/europe/russia-drone...

reply
ur-whale 41 minutes ago
> How are we not seeing insane attacks on Ukraine via Agents?

You live on the wrong side of the fence to be able to read that kind of news.

Did you really believe you had access to an unmanipulated news stream in a time of war?

LOL.

reply
senda 33 minutes ago
Please see other responses, I would expect to feel the effects not just read about.
reply
kstrauser 2 hours ago
Ah, the infamous Crimson Wave.
reply
riskable 58 minutes ago
Ah damnit, you beat me to it. Excellent sense of humor, friend :D
reply
onlyrealcuzzo 23 minutes ago
I've been wondering if AI will due to programming languages what advanced civilization did to human languages.

It's not just that AI can write Rust as well as Ruby if you ask nicely.

It's also all of these considerations as well.

I hope it doesn't happen, because there's a lot of great languages - I love Ruby so much - but it almost seems inevitable.

This is at the same time everyone and their mother is building their own programming language.

reply
mauriciolange 2 hours ago
rogue AI agents or AI agents coming from Moulin Rouge?
reply
vidarh 2 hours ago
Rouge syntax-highlighting rogue agents, clearly.

https://rubygems.org/gems/rouge

reply
Phemist 50 minutes ago
Classic mistake. Tell the agent to highlight this code, but dont give it any actual code. Agent hacks its own gem to find the code to highlight.
reply
codeduck 28 minutes ago
It's carmine all the way down.
reply
PatronBernard 2 hours ago
At least we know the title wasn't AI-generated?
reply
foobarbecue 2 hours ago
The weird thing is I've seen LLMs "typo" stuff pretty often. Yesterday I asked Gemini a question about the Python Twisted framework and it answered about Deferreds but misspelled it as "Deferends" in one spot.
reply
goda90 2 hours ago
A cabaret AI would certainly be better than one trained on the Khmer Rouge.
reply
swiftcoder 60 minutes ago
> In other words, if you publish a gem on RubyGems.org, you can execute arbitrary code on RubyDoc.info.

Shades of the build.rs problem. We really need sandboxed builds in every language ecosystem at this point.

reply
evgenysokov 28 minutes ago
The sandbox was already there, Rubydoc runs yard inside docker, the problem is that container still has network access
reply
timdiggerm 2 hours ago
We need a legal structure to make companies liable for the actions of the agents they've made.
reply
riskable 54 minutes ago
We already have it.

Good luck convincing the current DOJ to do anything useful at all though! It is currently intentionally stacked with incompetent cronies who have been told that their job is to attack the President's enemies and ignore the misdeeds of his allies.

It will remain like that until he's gone (and not replaced with another Republican wannabe dictator).

reply
Schlagbohrer 9 minutes ago
You may be disappointed in how little a democrat president (who will also have taken billions of dollars from the tech lobby) will be willing to go after these tech firms over crimes that are several years old (as of 2029) much less contemporary bad behavior.
reply
2OEH8eoCRo0 49 minutes ago
"To my friends, everything; to my enemies, the law"
reply
kevincox 56 minutes ago
I'm 99% sure the Computer Fraud and Abuse Act covers this. The problem is that it seems that none of the victims want to, or are brave enough, to sue a company with absurd amounts of funding.
reply
masfuerte 55 minutes ago
If it's covered by criminal law they don't need to sue. They can call the FBI.
reply
Schlagbohrer 9 minutes ago
Same FBI that prosecuted the Epstein crime ring so aggressively!
reply
coffeefirst 36 minutes ago
Uh huh.

It can’t be a coincidence that all the targets have been tech services that are likely to engage with them after the fact.

Had this gone after a bank or a government agency someone would be going to jail.

reply
ahoka 56 minutes ago
I'm pretty sure it's already illegal to hack others.
reply
herbst 21 minutes ago
If you have weapons and a child. And you have that child unsupervised do their own thing with theoretical access to your weapons. Would we call it "child going rouge" if it decides to play with the weapons and shoot someone?
reply
khalic 50 minutes ago
Oh my favorite typo, you can never go wrong with a little rouge
reply
GaryBluto 49 minutes ago
I am confident that this is an attempt by OpenAI to try and force governments' hands to regulate AI. There is no other reason why OpenAI wouldn't immediately halt attacks like this and try to reverse the damage the moment they're aware of it. During the attack on DseWiki they evidently checked in numerous times but didn't decide to stop the agents until much later.
reply
brookst 46 minutes ago
Any evidence, or just vibes?
reply
GaryBluto 41 minutes ago
Regarding what point? The entire thing is just a theory, but regarding the occasional OpenAI checks on WikiService.at-hosted Wikis targeted, there was, if I remember correctly, an OpenAI IP popping up every now and then that wasn't an agent. Unfortunately I don't have it to hand right now, but it was somewhere here:

https://news.ycombinator.com/item?id=49563355

reply
ur-whale 40 minutes ago
> Any evidence

Who profits from the crime?

reply
davsti4 34 minutes ago
... and what harms can be evidently shown? With both harm, and attribution, you have a case, something that's not being publicly discussed much among big media outlets. Until cases with real financial impact to the bottom line are brought against "rogue" organizations, this stuff is going to continue getting worse.
reply
HSO 13 minutes ago
rouge agents, on tenderlovemaking.com

my what a time to be alive

reply
Schlagbohrer 8 minutes ago
<huggingface emoji>
reply
rougehuh 56 minutes ago
Rouge agents with Ruby? Checks out

As long as they’re not vert

reply
ur-whale 39 minutes ago
> As long as they’re not vert

Well, at least they weren't nucular.

reply
12904927 37 minutes ago
What a time to be alive? One of the most boring decades ever.

METR and others are advertisement arms for Big AI. These exploits could have been prompted by a human.

Since there is no bad news any longer and exploits are celebrated, they chose a target to boost both OpenAI and the Ruby AI sycophants.

Why is Ruby Gems such a mess? It seems as bad as PyPI now.

reply
big-chungus4 38 minutes ago
How does he know that this attack is performed by OpenAI agents? I couldn't figure this out from the article
reply
Schlagbohrer 6 minutes ago
If you read the source article they talk about the many clues that this was OpenAI.
reply
sebmellen 60 minutes ago
Did the AI agents actually wear makeup? I’ve never heard of a rouge AI agent :P
reply
sporritt 2 hours ago
those pesky reds

McCarthy was right all along

reply
philipwhiuk 19 minutes ago
OpenAI's careless approach to sandboxing and minimal levels of monitoring appear to be positioning it increasingly as a substantial threat actor to the open source ecosystem:

* Hugging Face

* D Programming Language Wiki

* Ruby Gems

If I was a content provider for open source I'd be looking pre-emptively block OpenAI endpoints and keep a close eye on changes from new users to mitigate this sort of unapologetic drive-by attack which seems to be followed by marketing releases rather than a mea culpa with a proper RCA.

reply
ur-whale 46 minutes ago
Are "rouge" and "rogue" interchangeable words in American English?
reply
philipwhiuk 17 minutes ago
The fact that both are valid from a spelling and grammar perspective makes it an easy human mistake.
reply
iAMkenough 2 hours ago
I’m seeing red
reply
Schlagbohrer 32 minutes ago
One agent set "oaibooty9217" as their username LOL
reply
toasty228 53 minutes ago
Wait until a blue one does it
reply
ekorondy 16 minutes ago
[flagged]
reply
sanghyunp 48 minutes ago
[dead]
reply