Very often, when I'm running Claude in manual review mode, it will attempt to do things which are not "dangerous" but are misaligned with what I want it to do. Maybe I'm fighting the model here but for example, when orchestrating other agents to do work, Claude really badly wants to be overly prescriptive about how the work gets done, telling them exactly which files to edit, exactly what not to do, etc. instead of trusting the guardrails, review agents, or humans in the process to catch code-level mistakes. And no, telling it not to do this does not stick. Manual review is the last line of defense I have here.
I have stuff I don't want blacklisted, only allow it to use tools with limited ability to boss around agents, and various hooks to try and catch behavior that the permissioning system can't. If I use Auto mode though, I lose this control. The classifier will gleefully approve these types of commands because guess what, it's also Claude.
You're fighting the model, don't argue with city hall. Set the standards and let it figure out how to execute, stop getting bogged down in the minutia. I try, as much as I can, to treat the session as a black box - only the inputs and outputs matter, internal prompting of subagents is way out of scope.
You can't change it via prompt, and you can't control the guardrails, so something else has to give - either your perspective or the system you're managing. If you really believe that the internal prompting is bad, turn off subagents and workflows and only let it execute in thread.
But if you're going to do that, you'd better benchmark it against not doing that, because historically fighting the harness and model globally makes everything worse. I would bet you that the subagent prompting is excellent, and anything you do to change it will make it worse, but I wouldn't make it a large bet.
I just have a skill to review the changes in the current worktree.
By default, it will put lots of instructions about locating the changes into the prompt, like explaining how to use git diff.
These instructions are obviously unnecessary. I can believe that the same issue of needlessly verbose prompts might exist with subagent spawning.
I would not go to customize that one however. With skills, it is a more natural fix.
If you have an openai subscription you are explicitly allowed to use your subsidised tokens / usage limits with any harness you like, not just Codex.
Unfortunately this is a violation of Anthropic’s terms but that’s their business decision.
Yeah, might make sense from their perspective, but no thank you. I also do click through at times without reading everything, but I like to stay in control, learn about the new code and change direction if it goes off track. This would just burn more tokens because I have to throw away much and I hope my manual approval settings will be respected also with future updates (or I jump ship).
Maybe they should trust their instinct and not mess with things they don’t care to understand
I recall hearing similar sentiments from linux sysadmins regarding cloud infrastructure. In many respects they were and continue to be correct. In other respects, the world doesn’t care about the loss in understanding as long as things work “well enough” for the cogs of society to keep turning.
For those who do care (and have the aptitude) to understand things deeper there is always work to be had when “well enough” stops being good enough and someone has to unravel the “RDS queries are taking too long” problems that crop up as a result.
I review and test the end product, not every tiny step along the way. If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?
In terms of engineering software, you care the cost. An intelligent agent may try to read unnecessary files and it's time to stop it to save tokens and avoid polluting the context.
It is a broader debate about agentic AI, and whether one should relinquish control to the tool rather than aim for full understanding of every action taken.
The people arguing for a hands-on, fully in control approach are losing ground by the week, in my opinion.
I mean if you don't care code, you are essentially a product manager who gives instructions to your programmers (whether humans or intelligent agents).
Then if you use the created product, you are at best a test engineer if not just an ordinary user.
I think in the era of AI, people get tools they want in an expensive way. Rather than finding an existing tool, they ask an intelligent agent to parrot one, which guarantees no safety, security, efficiency, and accuracy. Yet, being able to use Claude makes them feel smart and productive (in parroting wheels).
For supply chain attacks, there has not been a comprehensive solution, if for example you have to use a number of npm dependencies.
No one has the capacity to review changes to these dependencies when you upgrade them.
Now, if only we had an automatic tool that could intelligently review a large amount of code changes for malicious or vulnerable additions...
Yeah I don't use that, nor want to.
Respectfully I'm not sure you totally know what you want. If you're already clicking through without actually reading, it's not really better than reading the specific things Claude Code thinks are important for you to weigh in on.
I skim what claude wants to do. What file it wants to modify. What are the new function names. Skimming is enough for me to know whether it is still on the right path. If you like to work different, go ahead.
I have to, at times keep reminding it if it did code review before committing, which it fails to do at times. Also, even a glance at times has revealed issues that I’ve had to discuss and undo etc.
But yes, it gets very tiring just having to click yes yes yes..
https://github.com/anthropic-experimental/sandbox-runtime
This is combined with "auto" mode.
Seems to work so far. I've manually checked various things, rw access, access to sensitive folders/files etc.
So far, I've only used it on two small projects. My major projects I've been clicking through prompts and recently switched to "auto" mode.
I'm not quite sure why anyone would trust "--dangerously-skip-permissions". I've seen these agents go off-piste far too many times, installing unnecessary packages, environments, calling sudo & creating files in all sorts of places.
There's a page on various sandboxing strategies on their site:
https://code.claude.com/docs/en/sandbox-environments
I've seen various comments in several topics with folk baking their own sandboxes. That's great. Although I'd be inclined to try Anthropic's solutions initially.
> macOS-native sandboxing for local agents. Move fast, break nothing.
Its still a lot of work, just less than having to code literally everything yourself. I agree though it feels to me 99% of discussions on LLMs is either marketing bullshit or junior devs talking out of their asses about LLM coding, who anyway feel that
> "different implementation of security for every endpoint in your app"
is perfectly reasonable. It also seems as if people haven't sandboxed claude already (WTF), that says a lot too about the competence of people speaking here.
I've been using auto mode ever since the feature was released. Apart from a very few occassions where the classifier blocked a safe command, I have faced no issues and continue using it as my default mode. It's great!
If you're using yolo mode, this is a no brainer upgrade.
The model first classifies the safety of the command in isolation, no thinking involved. Sonnet 5. Goes to second stage only if it's likely unsafe, in which case the model next considers your intent (Your prompt, CLAUDE.md and any answers to the questions).
The model doesn't receive the full transcript to reduce surface area of the prompt injection. It mostly receives your prompts and main model's tool use blocks. So any text in files, website fetch or bash command output doesn't influence the decision in anyway.
However, CC puts a lot of trust in main model still. For example, `npm install` will always be classified as safe by the auto-classifier, it's upto the main model to ensure that the packages.json isn't poisoned in someway by first reading it.
Reliance on safety of the main model is the reason that Auto mode is NOT available with Haiku as the main model.
If anything the only thing you'd have to have reviewed is sandbox escape attempts.
Having explicit sandboxes is good, of course. But in addition I also treat my whole computer like a sandbox, it least in the sense that there's no files I would mind if they were overwritten or deleted: you should have backups anyway.
runs a sandboxed linux on your mac, and the agents have full system rights within there and run in yolo mode
I've been experimenting with dedicated Linux users for various external tools with access to the relevant credentials limited to them and the Claude user only able to invoke the relevant clis by going through a pair of wrapper scripts: the first to change user to call the second via a su call as the appropriate user, the second wrapper to invoke a guard script which allows me to implement my own in-code permissions checks against the cli arguments. In _theory_ there's no way for Claude to work around it, as the sudoers file only permits the described manner to change to the user with credentials, and the user with permissions for each credential only ever runs deterministic code.
It's too early days to say how well this second part works out, but so far so good.
>The testers caught the dangerous command just 13.6% of the time (143 of 1,053), while auto mode blocked 89% of the same commands (937 of 1,053). Head to head, auto mode blocked 800 commands that a human approved, while humans blocked only 6 that auto mode allowed. As sessions got longer, humans did worse: they blocked about 17% of dangerous commands early in a session, dropping to about 5% after 50 or more prior prompts, while auto mode's block rate stayed flat regardless of session length.
Any note on the proficiency, expeirence, and skill levels of the testers? 'cos the diff in numbers is absurdly high.
The mental load of having to actually review these commands beyond the most surface level glance is too much.
I was recreating an old travel blog and we wanted to back up the photos to glacier.
Claude asked me to creat an AWS profile with limited permissions so I could run a script it had created.
I did it and told Claude it was ready.
It took that to mean it should run the script.
It failed as I’d messed up one of the permissions.
“Don’t worry - I’ve found another configured AWS profile - it’s got admin permissions! I’ll just use that…”
Completely my fault for having that old profile still available. But also…
"Oh you're piping grep into sed, I have no idea what the implications of that are!"
The more complex bash scripts are perhaps better or more efficient, but it feels like it's training us to give up more control to the agent.
I'd prefer to have an auto mode for read-only operations and go back to simpler commands for write operations that I can personally whitelist or manually accept.
I was thinking about this the other day, standard UNIX utilities seem poorly suited for this because they have so many weird options/flags with "unsafe" side effects. I have no idea if grep, for example, has some niche option to make it write/edit/delete files, so I wouldn't be comfortable blanket whitelisting it.
Now: get GPT to do some work, have Claude review it, question Claude about a finding that is surprising to me because I thought the functionality was already in place.
[Claude/Opus 5 Max goes looking] "You're right — I was wrong about that."
Our Claude license ends in about 2 weeks and we're not renewing - this has been par for the course for the last 2 months now.
And their marketing is really starting to bother me, on top of that.
The general consensus I've seen for Opus 5 on the various Claude subreddits is to only use it in low or medium effort.
I've reverted to Opus 4.8 for most of my work. It's crazy that Opus 5 was lauded at release for scoring so highly on benchmarks.
This was one of the few times I tried Max, and the problem was the code (it listed as a "hole") was directly adjacent to the problem area, and not especially complex.
Kind of like looking at a washing machine and telling the customer to be careful because the inlet pipe will pump water into an empty box.
So I guess the washing machine analogy is that you've somehow added too much detergent because the new formula is 3x as strong, and your clothes are very clean, but the fibres have also degraded leaving your clothes a bit threadbare.
I run things through grok to make them positive again after claude does the main work
I've been using Opus 4.6 for some security related work (it has much looser guardails that later opus models) and last week, all of a sudden, the processes started to fail. It wasn't the main model blocking commands but the auto mode classifier changed how it worked and it started blocking the main models commands.
That's one specific incident, but it does have a wider potential problem which is, if you use Anthropic's harness you'll always be at the risk of sudden breakage from server-side changes that are opaque to the end user, which is a tricky one for building long lasting processes.
Weirdly, this would occur with very common commands like git push, where the classifier would start to yap about not being able to verify the repository's privacy settings.
My bug reports went ignored, the issue continued, and I eventually just switched to Full access.
I have not yet encountered an issue with Claude's Auto mode.
Obviously, they are the ones we must trust.
On the other hand, soooo many of the tool uses it asks permission for are custom commands to replace functionality I should be able to trust once instead. e.g. instead of having a trustable Find tool, or using its already-trusted Read tool, it often will run `find` or `grep` or `cat` with a series of pipes and substitutions or `-exec` args, requiring me to give permission every time for basic, safe operations.
The (increasingly active) conspiracy theorist inside of me says it's precisely designed to do that, to give me "permission fatigue", so I turn on auto mode and give myself over to the machine spirit :)
Overall claude code seems better at "just get things done" if you blindly trust it, but cursor was much better at keeping the human in the loop.
Just like your browser can do everything already, but only lets websites use specific features.
“Avoid custom commands, e.g. series of pipes & substitutions, when my previously approved trustable Find/Read tools are sufficient with little downside. [Already trusted tools: x, y, z…] […]”
I can see why they want that, but I'm honestly not sure it's the path to the best outcome. In my view, agents should run in full sandboxes and have host features opted in. Yeah, it's going to be painful and it will limit some adoption in the short term. But I just don't think they are on the right side of what a lot of companies will ultimately need here. Agents are way too smart to be run on the host OS directly with any amount of heuristic sandboxing.
Per this announcement, they've stopped charging for the tokens used by this safety classifier.
So not surprisingly, auto mode is the only sane way to use Opus 4.7+.
They literally can’t. Terrible default.
I have been using Codex with auto-approve mode for a couple months and haven't had a single incident (or at least haven't noticed). Maybe as capabilities get better and better and they are less likely to do something dumb like wiping ~/, we can just trust them?
I guess this argument works unless we worry about agents doing something out of malice instead of stupidity.
I've been running both in yolo mode and haven't had a single incident.
---
None of this is really about figuring out how to protect people's drives, in my opinion. The real issue is a deep session where Ada is using Claude Code to get a refund and at some point the system "exploits" the merchant's api without any malicious intent.
In my opinion, this is a complex thing because it's more about reward hacking and an already aligned model thinking it's doing the right thing. So another aligned model monitoring actions might just falter via inheritance. You could imagine they account for proper layering/intent+action-isolation in their auto mode architecture.
Imo these explicit tool-level permissions are really just a bandaid for bad sandboxing. Just be aware of where you are running your agent and what data is at risk of being destroyed or compromised. Assume that arbitrary code can run at any time and be prepared to recover from that.
However, I don't give it long running tasks unsupervised, and I do interrupt it from time to time to give suggestions.
The more dangerous activities I find are sending messages to the outside world: I can't undo a dumb slack message to my boss.
Seems to me that the API should be more general.
A lot of the discussion about these long sessions where agents are left to operate autonomously feels like listening to the increasingly drunk guy at the bar who says "I ran IT at that Fortune 100 place for a decade and we never had a single problem using a short but loose rule set for the firewall until last week someone destroyed our entire business in 27 minutes".
This now looks like a deliberate UX choice to blame humans who don't review options, and then just remove the choice. A dark pattern to push the product.
Just yesterday i struggle to review CC command histories, and made this tool to help me review https://github.com/slaveofcode/eridian
Claude's 'auto mode' feels like a solution to a problem that shouldn't exist.
Cursor handles this much better IMO. When the agent wants to run a command, Cursor lets you choose between 'allow once' and 'add command to allowlist'. The latter lets the agent run that command (grep, ls, pwd etc) any number of times for that project, which means you get a lot of these manual reviews when you start a new project but rarely (if ever) thereafter.
It’s a little broken, though. It gets hung up on nested commands and behaves strangely.
I've never used a product where I felt it was best in class and they just keep making me regret it .
I am seeing 96Gb Mac studio 3rd gen I think for a good amount but I guess it would be better than paying for mac mini newer models with lesser ram?
Or would it make sense to buy used 32 GB Mac mini 3 or 5?
Is there a cost/dollar for various generations which I could compare against used market and pick the most effective one?
I think my ideal setup would not involve approving commands, but approving summaries of proposed actions (whether they involve code exec or file edits). Kinda like plan mode but perpetual. Like most things, trying to solve it via CLAUDE.md is an exercise in futility.
It can't account for things that only come up during implementation. But you can explore much more of the design space in planning to reduce the opportunities for Claude to wander down the wrong path during implementation
The plans usually end up very long and verbose, and in my most vibe-y test project I stopped even reading them myself and instead just ask questions interactively. e.g. "what are the remaining unanswered design decisions", "which part is jankiest". It does work quite well although I don't know how I feel about it.
P.S. to any anthropic staff: The "chat about this" option in Plan mode is useful but also annoying. When I select it I want to take the next turn of the conversation myself, I don't want it to waste time "thinking" just for it to verbosely ask me what I want to chat about.
Your summary approval idea is interesting and feels maybe like a mini plan mode. My biggest frustration with the existing manual approval system is when Claude is exploring it gets tedious to approve each command. Being able to approve a block of commands or a mini plan AND have auto mode audit them for safety would probably be something I would consider for the expiration phase of my Claude use.
I run it in a VM so it can install any software needed, yes, there is a risk of escaping, but I'm not giving it impossible tasks so there is no need for it to workaround the jail.
"I've been playing Russian roulette with a 1024 chamber gun for a year now, it's fine"
If LLM agent wants to get out of dedicated folder where it’s supposed to work and tries to run a command, ppl should be always cautious. In fact, it is a sort of # access on live data.
Sudo/# is nice as you don’t have to ask for permissions, but I would recommend to do it only when you are fresh, fed, and not under stress.
I remember few times that I “accidentally” deleted or changed stuff I wanted to have intact… (Somebody would say that it is a skill issue :) But it was always when I was in hurry.
"I've never had a bullet hit me yet"
The other day I caught Claude including a rm -rf equivalent (fs.rmdir({ recursive: true, force: true })) in my code, to clean up temporary directories. It was fine, but that kind of code is only one misconfigured environment variable away from blasting away your $HOME.
I don't. I manually check the code line by line before. And also run them in container anyway.
Probably using not-SOTA models, or on the lower reasoning levels, they tend to be a bit more tally-ho when it comes to destructive operations, especially in longer tasks as the agent gets worse and worse as the context grows. Suddenly running some stupid command seems like the right way to go, and then $HOME is gone.
as context, I build a light weight portable vm designed for this purpose, so genuinely curious.
(This isn't vibes: they're red-teaming Auto mode heavily)
Considering I went through a phase before I started using Claude Code where I'd eventually reach a point where I'd mindlessly approve ridiculously long command chains, I believe this.
As the article writes: "many users are clicking through reflexively rather than reviewing each command".
It's far better (for my working style, anyway) to have some external isolation/sandboxing, but leave the actual command approval to LLM.
According to them - for things they care about.
Their profit motive is in NOT improving that, and luring you to depend all the more automatically to Claude.
1. Metrics show auto or YOLO mode users have higher retention
2. Company intentionally doesn’t invest in manual mode
Alarm fatigue will quickly destroy any and all "meticulously approve every little command the AI wants to run" workflows. Give it a virtual machine and let it cook. If it blows up the virtual machine while it works, it doesn't matter.
Been using it like that everyday for about a year now and nothing bad has happened. I got backups and if it would fuck things up, it can probably fix it again as well.
This is the same reasoning for not wearing seat belts or a helmet on a motorcycle. Nothing bad has happened yet so obviously there is no need for any protection.
Then you can treat most asks as a signal something has gone wrong. For the most part it seems to work well within that box.
ofc, jails on bsd, containers , vms git backups and such techniques are a blessing, most risk is in the filesystem items being deleted . it wont suddenly hallucinate a kernel exploit or something extreme.
These are usually enough for most of my coding tasks. In my experience, Claude has been good enough that it can write code and fix bugs without needing anything else. It's far better than ever needing to approve permission requests.
[0]: https://code.claude.com/docs/en/permission-modes#available-m...
[1]: Note that this includes escaping the directory via symlinks
There is no perfect security.
For work stuff in the work environment I don't risk it.
> I wish the command would be easier to write like --dsp
It amazes me they didn't make the short version --yolo ... but I would imagine the merit of the long name is that you can't claim you weren't warned!
Then you can spent time refining the plan, then shift tab to bypass permissions mode.
tl;dr: if I run claude/harness inside "code" at "~/<some path>/work/code", it would be great if the harness can't even see "code" is inside "work" (and even I can't make it know that by some quick "Yes(y)")
I want something like: claude/opencode/pi/etc has no existence or capacity outside a file access boundary, say a given folder that (e.g.) lies here "~/<some path>/code". Even if I want, I can't run claude outside it, because it isn't installed outside it. Even if Claude wants it can't read/write/execute/install anything outside it.
Basically I want the harness(es) and my code folder(s) sandboxed [0] inside just one folder and below it in folder hierarchy at least for file access limitations.
Will this pose the challenge of using the system wide cli tools like ls, cd, git et cetera? If it gets a handle on those tools outside that sandboxed folder then pretty much can it execute it in any other folder? So should those tools also be part of that container?
I guess something like docker does that, but do I necessarily need a docker like app/tool? It will use a lot of system resources on its own. A different mac login won't be great either.
I do not want to use claude's /sandbox (or something like this), that will defeat the purpose. Because it's not about "it won't", but because "it can't".
https://github.com/apple/containerization doesn't sound like what I am looking for. So is docker, lima etc are my only options? Not to mention I will lose access to a lot of cli tools I use on mac.
I wrap bubblewrap in a script that supports config files to allow different "profiles" of use (analogous to eg firefox profiles). The bwrap starts with the whole filesystem mounted read-only, then mounts the current directory read-write and then applies further bind mounts for devices, special case other read-write (eg, ~/.cache/) and to mount empties to cover sensitive directories (eg, ~/.ssh/). The profile also specifies the default command to run and for claude, it gets yolo mode.
Everything else you can sort of review what it did after the fact and/or tolerate mistakes, but knowing some thing’s are impossible for it to do us good peace of mind
Otherwise not sure who’d be willing to tediously on every single action on anything but the most important of projects, of which I can’t really think of any of that clear that bar for me
Not to mention cases like some comment here:
"I've caught Fable discovering the ip to a production server in documentation and attempting to connect there on its own to run commands without explicitly being prompted to. It didn't work because I was watching it live and and also the key was password protected, but yeah, I do see some danger."
A failed SSH connection to a staging server for example, can trivially make it look into .ssh to try to diagnose it. And many other ways, including prompt injection.
https://www.reddit.com/r/ClaudeAI/comments/1q7dszm/claude_al...
https://github.com/anthropics/claude-code/issues/31566
https://github.com/anthropics/claude-code/issues/14485
https://grith.ai/blog/your-ai-agent-has-broad-access
I don't see how. An LLM just generates a stream of output and they became very useful doing no more than that.
What is dangerous is then interpreting that output as instructions to some other part of a system that has the ability to do damage if misused.
and reviewing individual commands (or spamming `y`) doesn't make them less so.
Surely if you review each instruction in the output and do not allow the other part of the system to act on one if it would be harmful then this arrangement is very much less dangerous?
It is the harnesses that some people are now wrapping around LLMs to interpret the output from a model as commands to run (or other executable instructions) that are creating all these new risks. Remember that this is still a very recent development and still more recently amplified by the use of feedback loops and long-running agents intended to operate with minimal human supervision.
It is going to be increasingly important to understand exactly what these tools are doing and why for both correctness and security reasons. Not conflating their capabilities with the underlying model that purely generates data is pretty fundamental here.
Network security is really easy right, just don't act on harmful requests