when the "give a shit quotient" of the employee moves towards "highly engaged with the organization and its mission" to "this is just a job, I'm gonna rest and vest"
We can put in systems in place to catch bugs. And that's the system that's working here.
Now to the second part of the question -
How did they not catch it before? If it were "easy", it would not have been worth 90k, and hindsight is 20/20 - once you know where to look for, it is all very very obvious.
We're probably at the end of the era of these kinds of big payouts. AI tools will drive the going rates to approximately the costs of the tokens. Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.
In reality, the modal benefit accruing to a company running a bounty program on receipt of a new serious vulnerability is simply engineering prioritization. You get bug reports from users when you have reliability and UX bugs, but you don't get them for security flaws, because attackers generally have to force vulnerabilities into revealing themselves --- normal users almost never run into them. Bug bounty programs fill that gap.
If you’re saying that bug bounties exist for a reason according to both the program runners and the contributors then how is that not the reason they exist?
The people paying and the people working are all in agreement that it exists for that reason.
Is this really true? After all, reselling the exploit would be illegal, and most people don't want to break the law to earn income.
Cyber companies are in the fortune 100 now, and it's not like it used to be back when you actually had to be interested in it and dedicate your life to learning the skillz from shady forums. Now it's just a job to most practitioners.
On the other hand, if the value of that drops you can expect to see some folks start taking the wrong path.
Oh, maybe that's why "Claude" was substituted with "I", because Claude wrote this blog post from a first person perspective.
I agree effort and prior knowledge are required, but a few magic words is like 80% of the effort.
This is the typical "ugh that's just rsync" when Dropbox was announced, but for a new LLM age.
_If it's so easy, do it_. Don't post about it. Do it. Fix these bugs at scale. Charge people money for it once you've built up a reputation. Get your own bug bounties.
Research taste, among other things are still abilities that differentiate among humans.
In fact, I expect the elimination of other limiting factors by AI to lead to those becoming more important. By the same mechanism that make ugly hedge fund managers more performant.
The problem then is that composition of the top performers will change. Which is problematic since being a top performer ( ‘artist’, ‘researcher’, ‘writer’,..) is often an identity which ‘we often create to protect ourselves: the strong one who never breaks, the intelligent one who is always certain... The mask that once protected you can eventually become the cage that prevents you from changing’
No thank you says pretty much everyone.
I do have to send input to the thing to get a response, no?
Same old tired complaining about people not using technology correctly, or the way a _real_ coder would do it.
I also am wondering if Claude will proceed on a prompt which is about finding a bug like that? I am not sure, has anyone tried?
The cookie contained a pickle-serialized payload, "which was bad enough on its own." maybe not best practice but not bad on it's own
Flask session cookies are signed", not encrypted," so the... LLMs love to tell you what things aren't, as an emphasis on what they are
;)
"Claude did this, not you" - does that mean that humans have no agency at all when they use an LLM? Is that the argument?
Maybe the norm should be to credit the LLM and harness used, like "I used ___ to help me find an RCE", but I can see arguments not to do that in some cases.
edit: The giant thread with this now got flagged, so some users may not see it.
I suspect a lot of things fall into the latter. It is a common neg to defer everything to the former though.
Yet, they are being used with endless surprise as non-deterministic layers of abstraction on top of deterministic compilers.
My point is to use a technology for what it is capable for (where capabilities are discovered weekly and hard to keep up with), not trying to get it to do everything.
Bringing a deterministic lens to non-deterministic software has its dangers too. Non-deterministic software seems to be its own category of software, increasingly capable of non-deterministic approaches that deterministic compilers software couldn’t attempt.
Maybe that’s less compilers and more different type of scaffolding of vectors.
I’m all for skepticism, the issue with cynicism is capabilities often pass those perspectives by sometimes too quickly.
But it needs a disclosure if Claude was used: "The cookie is what caught my eye" - if that was actually Claude's eye then yeah, Simon is claiming to have a keen eye that he in fact does not.
"Claude found a $90K RCE" would have been just as interesting if not more.
It is fair an interesting to disclose tools used, but I am with GP in leaving agency with humans, at least for now.
If I post an image and I say, "I drew this", and someone says, "no you didn't. That is clearly output from Qwen Image 2.1", can I reply, "Qwen Image 2.1 is just a tool, like Photoshop"? I mean, I can, but if I do this, I am missing the point. The point is that I didn't draw anything. I didn't draw. I didn't draw digitally. I didn't draw analog. I didn't draw with my mind. I typed a prompt. This is as close to drawing as photography is to drawing.
So did this person actually find a vulnerability on their own? Hell if I know, but I take issue with LLM-authored blog posts; I don't trust people who post LLM-authored blog posts as if they were written by themselves to be honest about the nature of their work.
If you really are still doing work and understanding and just using LLMs as a tool, show your understanding in your blog post that you are communicating to the world, instead of having Claude or GPT show its understanding. I don't really care whether you used Claude and didn't list it, if it's clear as day you understand what you were doing with it.
I'm dying on this hill. Until HN bans this sort of blog post or bans all of us complaining, these comments are going to be very, very boring. And I am stubborn for longer than most people can stand to read the same comments. Believe me.
We can discuss the quality of the offering. But that’s a different discussion
Drawing with Photoshop absolutely resembles drawing. Photobashing with Photoshop does not. Photography also isn't drawing. These things are all distinct. That's the actual point I was trying to get at here by invoking Photoshop. Image models differ from text in that text models do resemble something closer to what people are actually doing, but you just have to accept that analogies have limitations here. Just like you can draw or not draw with Photoshop, you can use Claude in ways where you're really driving it like a tool to do work, and you can also use Claude in a "fix the bug" sense where you're doing absolutely nothing other than paying $200/mo and typing occasionally.
But whereas Photoshop introduces nuance to what it means to draw or make art by introducing different levels of involvement versus remixing and etc., Claude and Stable Diffusion utterly decimate nuance in a thermonuclear blast that plunges us into a nuance Nuclear Winter, by offering for perhaps the first time a way for the computer to do nearly all of the generalized "thinking" while you sit on your ass and argue on HN. (Reminder for everyone to check on their agents.)
Trust me, I use LLMs, I get the idea. I can "vibe code". On my workbench is a laptop running an entire desktop system including an application suite that I really didn't write. It would be cool to go online and say "Hey guys check out this cool desktop environment I wrote!" but I won't because I really didn't. It was just an experiment. Did I put in effort? Yes. Was it coding? Nope, it was closer to management, though I did write some code. The experiment was some mix of failure and success, but what I would never do is say that I wrote it. I may some day blog about the experience and the pros and cons and how I feel about the resulting code, but for right now I feel the Internet is full of enough slop and most people who I respect are thoroughly tired of hearing about LLMs all day (I am too at this point, though I am not saying I find none of it interesting.)
Do I think all LLM use absolutely must be disclosed? Actually no, and especially right now I understand why some people don't. I know I just said LLMs and diffusion models destroy all nuance, but that's not strictly true. My line in the sand is this: if nobody can tell overtly that you're using LLMs, and you can demonstrate your own understanding of what you are doing, then I don't care. In that case it's more of a concern for the person using the LLM, because the real concern in that case is becoming dependent on LLMs and being unable to write software/etc. without them anymore. But that isn't really my concern.
If I can tell your blog post was written by Claude within the first few sentences you have completely failed to demonstrate anything but your willingness to deceive people. How can you not feel some shame putting yourself on the By-line of a blog post you didn't write? I mean it's written in first person! And I know damn well that not everyone can immediately pick out LLM-generated text, but that isn't entirely the point either. It's one thing to use LLMs to do work, but the point of a blog post isn't just the text, blogging is a form of (formerly...) human expression. It just doesn't make any sense to put Claude-slop blog posts right next to human-authored blog posts, any more than it makes sense to put AI-generated images next to drawings. The point isn't the pixels and isn't the words. That contrasts with code, because with code, the point really is the code, and other stuff is secondary.
I can't tell you whether this person found this vulnerability and triaged it or Claude did. I don't even care. However, I don't trust a blog post they didn't write anyway.
This is a long comment, but it's hard to make it shorter. LLMs and diffusion models are not comparable to tools like Photoshop, in a way that is more extreme than the difference between Photoshop and a pencil. Analogies are nice (limited) shortcuts that can help us reason about things, but in this case analogies alone are insufficient because we are dealing with something that has a brand new shape.
When you write your blog post with claude, I start hitting these phrases and then start only skimming and looking for them, and stop reading what you have to write.
I've spotted a few memes about this and can admit to doing the same..
ai;dr
(tl;dr, evolved)