Auto mode is now the default in Claude Code

289 pointsposted 2 days ago
by sbehere
(claude.com)

333 Comments

awkii

2 days ago

I'm apparently from the very small faction of users who's been running `--dangerously-skip-permissions` in every instance of Claude for the last year. It's almost a reflex to me. For the most part Claude behaves well, but I don't blindly trust it. LLMs are inherently dangerous tools, and reviewing individual commands (or spamming `y`) doesn't make them less so. Safety is the responsibility of the developer putting sane guardrails (like VCS, immutable filesystems, or read-only tokens). Using more Claude to categorize the safety of Claude commands is not the answer.

Retr0id

2 days ago

I manually approve everything (aside from the "allow commands like this" option). Not because I'm worried it's about to do something harmful but because I want to stop it from wasting tokens doing something stupid like implementing a feature I never asked ask for. I'm not even reading the commands, just checking it's going in vaguely the right direction.

I think my ideal setup would not involve approving commands, but approving summaries of proposed actions (whether they involve code exec or file edits). Kinda like plan mode but perpetual. Like most things, trying to solve it via CLAUDE.md is an exercise in futility.

bradfa

2 days ago

I run in manual mode not out of rogue agent or token waste concern but because I want to effectively pair program with the agent. I need to understand what is happening along with the how and why. I’m confident I’m slower than YOLO people but that’s a trade off I’m consciously making so I can stay in the loop to comprehend the codebase.

Your summary approval idea is interesting and feels maybe like a mini plan mode. My biggest frustration with the existing manual approval system is when Claude is exploring it gets tedious to approve each command. Being able to approve a block of commands or a mini plan AND have auto mode audit them for safety would probably be something I would consider for the expiration phase of my Claude use.

tracker1

2 days ago

I spent part of last week, and the weekend, in Claude (Fable) on a documentation project... maybe a waste to use Fable, but been a little obsessed with trying to get the technical details right. I know, from personal history, more than some of the available info easily found on the subject. I let a lot of it just go, and swing for the fences, but in the end, did hand review all the output at each stem.

In my specific case, implementing modern documentation for Remote Imaging Protocol (and old graphics protocol for terminal/bbs usage), and in particular trying to gather what exists of the later version released. The company itself folded, and there wasn't a lot of broad use of the later versions, and only partial information was ever released as a documentation. Claude was able to interrogate the apps, and some available open source, in progress work, and establish a lot of things.

I've still got to direct some edits... I guided a re-flow of the order the documentation was written in, particularly to be consistent between the versions... but it obsessively added verbose summaries to every page/file. All markdown, meant to build a website from, and eventually some compatibility tests, guidelines and a green/open implementation.

https://github.com/bbs-land/remote-imaging-protocol

bradfa

2 days ago

This morning I've turned auto mode back on, I'm only using it during Claude's exploration phases of my workflow and then I'm reading the full transcript (which includes some thinking info). So far, this seems to be quite close to what I want, and an improvement over my previous "manual all the time" mode.

Still leaving manual approval for all edits. Combined with reading the full transcript of the exploration, I feel I stay in the loop pretty well in this first test.

wongarsu

2 days ago

You can get some approximation of that with more detailed plans. I use a prompt like "Let's make a detailed plan. Write plan files as markdown to plans/feature-XYZ. [regular prompt]". That way you get much more thorough plans than with plan mode, and you can make it as detailed and deep as you want.

It can't account for things that only come up during implementation. But you can explore much more of the design space in planning to reduce the opportunities for Claude to wander down the wrong path during implementation

Retr0id

2 days ago

I've tried this, and it is largely what I do, but it's easy to end up with a large tangled pile of markdown files, and the implementation either diverges from what's in markdown or you burn a lot of tokens keeping them in-sync.

The plans usually end up very long and verbose, and in my most vibe-y test project I stopped even reading them myself and instead just ask questions interactively. e.g. "what are the remaining unanswered design decisions", "which part is jankiest". It does work quite well although I don't know how I feel about it.

P.S. to any anthropic staff: The "chat about this" option in Plan mode is useful but also annoying. When I select it I want to take the next turn of the conversation myself, I don't want it to waste time "thinking" just for it to verbosely ask me what I want to chat about.

ryanbrunner

2 days ago

I don't think there's any functional difference between "chat about this" and just directly entering what you'd like to do / ask about in the "Tell Claude what to do" option, I think that's pretty much exactly what you're looking for.

tracker1

2 days ago

That's close to my workflow as well... keeping planned areas in TODO.md, done changes in DONE.md, and a non-committed WORKING.md for the current work being done, as well as coordination with sub-agents, so I can view/monitor... have to direct the latter each time to have it done that way.

LinXitoW

2 days ago

Imho, the combination of the mattpocock skills, and the beads local issue tracker take care of all of that very well, with generally high quality output.

podocarp

2 days ago

That's a good take. If LLMs are supposed to replace programmers, then they need to be programmers. Executing random things in all directions hoping for a break is something novices do. A good teammate knows when to ask, what to ask, how to ask, and who to ask.

olek1983

20 hours ago

Tough call. On one hand, CC keeps asking me about the permissions; it shouldn't. On the other hand, I would rather not unleash it.

Currently, I'm running an internal experiment.

1. I have a side project. 2. I want to give the full control; CC is the so-called CEO, and I'm the so-called president of the board.

The cost of an error is marginal. I would like to see how this will end. The test is in progress.

unknownfuture

a day ago

> Not because I'm worried it's about to do something harmful but because I want to stop it from wasting tokens doing something stupid like implementing a feature I never asked ask for

... plan mode?

krzyk

2 days ago

I'm also in yolo mode, this is the only mode that makes sense for me, if I need to babysit I might as well do it myself.

I run it in a VM so it can install any software needed, yes, there is a risk of escaping, but I'm not giving it impossible tasks so there is no need for it to workaround the jail.

Joeri

2 days ago

For running it in yolo mode I set it up in a devcontainer. It takes a bit of figuring out, but once set up the permission prompts go away and claude has no access to anything I haven’t explicitly added to the devcontainer (unless it hacks its way out).

fluidcruft

2 days ago

I tried to figure out devcontainers (I don't use vscode) but it seemed like a lot of complexity and ended up just doing the old-school thing and creating a separate user/group that I ssh into with my main account as a member of that user's group so that I can browse and edit/add files.

tremon

2 days ago

Just for reference, here's my local "devcontainer" script:

  exec podman container run --rm --read-only --network=llm \
    --tmpfs=/tmp:size=128M \
    --volume="$1":/workspace \
    --entrypoint=/insert/agent/here \
    container-image-here
This runs the specified agent in a read-only container with only /workspace and /tmp writable. Obviously, you need to prepare the image first from a Containerfile/Dockerfile, with the required toolchain and agent installed. I use agent-specific ssh keys that are baked into the image, but you could also bind-mount specific files from your own homedir if required.

margalabargala

2 days ago

I catch it doing stupid things regularly still.

This morning I asked Sonnet to make an update to my Claude live statusbar.

Rather than look up its own API, it started reverse engineering the CC binary looking for strings relevant to what I wanted.

No matter how "contained" it is, I always start in manual mode and flip to auto once I'm confident it's on the right track.

binsquare

2 days ago

What VM do you run it in?

as context, I build a light weight portable vm designed for this purpose, so genuinely curious.

embedding-shape

2 days ago

I'm using a container. The risk isn't exactly "agent leverages 0-day against you to steal all your data" but more "agent mistakenly though $HOME was theirs and deleted it" so as long as you "copy data in > copy data out" without bind-mounting or automatically sync files, container works just fine for "isolating" them.

worldsayshi

2 days ago

I've tried both incus and firecracker. Both seem to work well after initial script setup. I've got the impression that firecracker should be a fairly safe option for such use cases.

dist-epoch

2 days ago

Given that people use VMs to sandbox agents of Mac/Windows/Linux, what does portable mean?

binsquare

2 days ago

i have a mechanism to package a VM into a .smolmachine file that you can rehydrate into a prepared vm akin to a container image.

dreadnip

2 days ago

I've been running it in yolo mode straight on my laptop for the whole year. It's fine.

coldtea

2 days ago

Isn't that what the people crying over their deleted or leaked data were doing too?

"I've been playing Russian roulette with a 1024 chamber gun for a year now, it's fine"

sixtyj

2 days ago

Every driver thinks of themselves they are an excellent ones. Statistics has another data :)

If LLM agent wants to get out of dedicated folder where it’s supposed to work and tries to run a command, ppl should be always cautious. In fact, it is a sort of # access on live data.

Sudo/# is nice as you don’t have to ask for permissions, but I would recommend to do it only when you are fresh, fed, and not under stress.

I remember few times that I “accidentally” deleted or changed stuff I wanted to have intact… (Somebody would say that it is a skill issue :) But it was always when I was in hurry.

embedding-shape

2 days ago

> Every driver thinks of themselves they are an excellent ones. Statistics has another data :)

I think I'm an excellent driver, and I have data that backs that up, like I've never been in an accident but avoided plenty. What statistics show that I'm actually a terrible or even average driver?

> I remember few times that I “accidentally” deleted or changed stuff I wanted to have intact… (Somebody would say that it is a skill issue :) But it was always when I was in hurry.

Gives me flashbacks to when the company I worked for had some issue in production, we were 3-4 people huddled around a laptop trying to figure out what was going wrong. Viewing the logs on the host itself was too slow or not possible somehow, can't remember the details, but we were able to start transfers of the logs out of the host. Only problem was that the log file was GBs long (what is log rotation?) so would have been too slow. Easy, just compress the log file, then send that, should be a lot smaller. Obviously we were all in a rush but still wanna be careful, so I enter the command for compressing the only log file we have, ask for someone to check, other person says "looks good to me", I hit enter and we get an error that it couldn't find the file we were trying to compress, the created archive been created without any entries added to it. The log directory is now completely empty, except one tiny "errors.log" zip/tar-file that had been created when we accidentally used the wrong order of the arguments in our command to create the compressed archive. I think we had to recreate the host at this point, reproduce the issue then get the logs again, probably slowing down recovery +30 min at least.

coldtea

2 days ago

>I think I'm an excellent driver, and I have data that backs that up, like I've never been in an accident but avoided plenty. What statistics show that I'm actually a terrible or even average driver?

The statistics showing that past performance doesn't guarantee feature results, that a lot of the accident avoiding could be just having luck while still driving badly, and that a lot of drivers overestimate their skills.

lazystar

2 days ago

on a similar note... what statistics show the accident rate of each model of car? or rather, a single driver's accident rate in various types of cars?

borzi

2 days ago

I'm assuming this happens to the people vibe coding and running 30+ agents in parallel that are "coordinating" each other. I've never seen Claude do or even suggest anything remotely dangerous when I'm just giving it incremental tasks and reviewing the output.

coldtea

2 days ago

>I've never seen Claude do or even suggest anything remotely dangerous when I'm just giving it incremental tasks and reviewing the output.

"I've never had a bullet hit me yet"

zarzavat

2 days ago

I don't use yolo mode but if you allow your agents to both write code and run/test it, then it's basically equivalent to running in yolo mode anyway.

The other day I caught Claude including a rm -rf equivalent (fs.rmdir({ recursive: true, force: true })) in my code, to clean up temporary directories. It was fine, but that kind of code is only one misconfigured environment variable away from blasting away your $HOME.

drdec

2 days ago

I've set up a second user on my machine just for running AI. That way I have the additional layer of OS protections against it doing something like that.

coldtea

2 days ago

>if you allow your agents to both write code and run/test it, then it's basically equivalent to running in yolo mode anyway.

I don't. I manually check the code line by line before. And also run them in container anyway.

badestrand

2 days ago

Whatever level of safety you are at, it's always easy to advocate for more. It's always a trade-off and in the end a matter of preference and risk-tolerance.

mlrtime

2 days ago

I run in auto permission mode and it will run with 1 task and end to completion. On the rare event it needs to rmdir or rm -r a dir, the permission mode will stop the agent. Sometimes annoyingly... so its more like. "I've never had a bullet hit me yet inside a office building on the 42nd floor".

embedding-shape

2 days ago

> Isn't that what the people crying over their deleted or leaked data were doing too?

Probably using not-SOTA models, or on the lower reasoning levels, they tend to be a bit more tally-ho when it comes to destructive operations, especially in longer tasks as the agent gets worse and worse as the context grows. Suddenly running some stupid command seems like the right way to go, and then $HOME is gone.

ben_w

2 days ago

If the director of alignment at Meta Superintelligence Labs gets it wrong, what hope does anyone else have?

vehemenz

2 days ago

You mean the guy hired by the Metaverse guy?

ben_w

2 days ago

> guy

Summer Yue, she.

> Metaverse

Zuckerberg may indeed be hiring people who are less than the best, same for Musk, but the people they hire are making the UX that the rest of us have to put up with.

Given I keep reading claims about heavy agentic coding use at Anthropic, and given what I experience while using ChatGPT, those companies are vibing the UX that the rest of us have to put up with.

TeMPOraL

2 days ago

VMs and containers are primarily useful to reduce maintenance burden anyway. I.e. if it fails or you're otherwise done with it, you can torch the container instead of having to clean up your host system from zillion no longer relevant packages and configs.

chasd00

2 days ago

To be honest I do this too. Nag mode is kind of a pita.

paul_h

2 days ago

I think all of Anthropic's own staff use --dangerously else they'd work on the usability of the massively imperfect 1-approve-once and 2-approve-forever multi-choice and outcome.

usef-

2 days ago

That's exactly why Auto mode was made, and reportedly what they all use. It's the easy option, with a better safety score than human review.

(This isn't vibes: they're red-teaming Auto mode heavily)

ValentineC

2 days ago

> It's the easy option, with a better safety score than human review.

Considering I went through a phase before I started using Claude Code where I'd eventually reach a point where I'd mindlessly approve ridiculously long command chains, I believe this.

As the article writes: "many users are clicking through reflexively rather than reviewing each command".

It's far better (for my working style, anyway) to have some external isolation/sandboxing, but leave the actual command approval to LLM.

coldtea

2 days ago

>with a better safety score than human review.

According to them - for things they care about.

coldtea

2 days ago

>else they'd work on the usability of the massively imperfect 1-approve-once and 2-approve-forever multi-choice and outcome.

Their profit motive is in NOT improving that, and luring you to depend all the more automatically to Claude.

dannyw

2 days ago

Maybe by design?

1. Metrics show auto or YOLO mode users have higher retention

2. Company intentionally doesn’t invest in manual mode

matheusmoreira

2 days ago

I too skip all permissions, but I run the harnesses inside an isolated virtual machine. I took care to firejail Claude Code on my host since day one, and coding my own QEMU VM orchestration solution was among the first things I used it for.

Alarm fatigue will quickly destroy any and all "meticulously approve every little command the AI wants to run" workflows. Give it a virtual machine and let it cook. If it blows up the virtual machine while it works, it doesn't matter.

hkchad

2 days ago

I setup an alias in my shell for --dangerously-skip-permissions after about a week of the constant y, y, y a year ago. I felt like Homer Simpson running the nuke plant and couldn't take it anymore. So far it's only deleted some files I likely would have said NO to, but in reality would have just hit y anyway. They were recoverable so no harm.

kabes

2 days ago

My .bashrc has `alias claude='claude --dangerously-skip-permissions'`

Been using it like that everyday for about a year now and nothing bad has happened. I got backups and if it would fuck things up, it can probably fix it again as well.

jjav

2 days ago

> and nothing bad has happened

This is the same reasoning for not wearing seat belts or a helmet on a motorcycle. Nothing bad has happened yet so obviously there is no need for any protection.

danielbln

2 days ago

I've been raw dogging Claude Code for the better part of last year as well, but I'm under no illusion that it's safe to do so, I just cannot approve seven trillion approval prompts and stay sane. I've since. switched to auto mode, however.

divan

2 days ago

You can also put it in ~/.claude/settings.json:

   "permissions": {
    "defaultMode": "bypassPermissions"
   },

dbbk

2 days ago

It's a bit annoying you can't set default mode to plan while keeping bypass permissions enabled though

jpnc

2 days ago

>nothing bad has happened That you're aware of.

dbbk

2 days ago

We're all going to die one day

myaccountonhn

2 days ago

I do it, and run claude as a separate unix User.

__MatrixMan__

2 days ago

This is the only kind of agent security that makes sense to me. Constrain it like you would any other subprocess. Unprivileged OS users, SELinux, firewalls, VMs... Unikernels? eBPF?

dist-epoch

2 days ago

Escalations to root are a dime the bucket.

__MatrixMan__

2 days ago

Then either fix them or put it in a stronger kind of sandbox. Or what would you propose? Anything that doesn't constrain arbitrary processes is weak against and AI just writing a program to do the thing that it itself its prevented from doing.

drdec

2 days ago

That argument is letting the perfect be the enemy of the good.

There is no perfect security.

jayd16

2 days ago

I haven't (and won't) gone that far, but what's been working for me is denying all arbitrary code execution, grep, glob etc, and then allowing the JetBrains MCP. At least that way it's essentially locked down to the IDE project.

Then you can treat most asks as a signal something has gone wrong. For the most part it seems to work well within that box.

sheept

2 days ago

I use dontAsk mode[0] with read access to the entire file system, write access to files from the working directory[1], some git commands (git commit yes, git push no), and a script wrapping Deno with the same read/write permissions (safer substitute for `python -c`).

These are usually enough for most of my coding tasks. In my experience, Claude has been good enough that it can write code and fix bugs without needing anything else. It's far better than ever needing to approve permission requests.

[0]: https://code.claude.com/docs/en/permission-modes#available-m...

[1]: Note that this includes escaping the directory via symlinks

ThomasGlanzmann

2 days ago

I do the same. I also use crush/deepseek a lot. I patched crush to remove all safetyguards and enable yolo mode by default. I use it on a daily basis including devops. For example to configure netapp storage systems and had not a single incident. Of course I also have backups of my systems and have everything under revision control. Depending on the task I let it run on a throw away VM, production system, lab system. Depending on the task I give it access to throwaway credentials (that I rotate afterwards) or my ssh agent socket.

mlfreeman

2 days ago

I do it, but in a devcontainer...but the devcontainer is as much to keep my host free of the random tools that a given project needs as it is (now) to keep Claude in line. If I had to click approve on everything it'd be worse than doing everything myself.

fluidcruft

2 days ago

I used to use dangerously-skip-permissions until auto appeared and I gave it a try and honestly I don't know there's much of a difference? If auto had existed in the past I would never have needed to try dangerously-skip-permissions.

tonis2

2 days ago

My issue is that the auto mode guard rail model, is sometimes overwhelmed and really slow, so I also mostly just default to --dangerously-skip-permissions. I wish the command would be easier to write like --dsp

dcminter

2 days ago

For personal stuff I have a separate physical machine on the network that Claude runs on in yolo mode and the worst it can do over there is wipe the machine. It doesn't have privileged access to anything else unless it starts trying to hack my network, which I suppose is theoretically possible with prompt injection attacks.

For work stuff in the work environment I don't risk it.

> I wish the command would be easier to write like --dsp

It amazes me they didn't make the short version --yolo ... but I would imagine the merit of the long name is that you can't claim you weren't warned!

hedgehog

2 days ago

You can set it in the config file so it's the default every time.

saidnooneever

2 days ago

i've never seen the point of manually approving myself but it depends on the scope of work you ask of it. it wont suddenly rm -rf / if that has nothing to do with anything in the project. if you let it manage env or user cfg stuff in ~/. or so that might get hairy etc.

ofc, jails on bsd, containers , vms git backups and such techniques are a blessing, most risk is in the filesystem items being deleted . it wont suddenly hallucinate a kernel exploit or something extreme.

adityamwagh

2 days ago

You can also set plan mode as default and start Claude with —allow-dangerously-skip-permissions.

Then you can spent time refining the plan, then shift tab to bypass permissions mode.

moritonal

2 days ago

Same, but I always run within dev-containers. I joke I trust my LLM with the same energy I trust juniors?

websap

2 days ago

Why not both? Having auto mode, plus having good guardrails is the way forward.

shelled

2 days ago

What is an easy way to achieve this on a Mac?

tl;dr: if I run claude/harness inside "code" at "~/<some path>/work/code", it would be great if the harness can't even see "code" is inside "work" (and even I can't make it know that by some quick "Yes(y)")

I want something like: claude/opencode/pi/etc has no existence or capacity outside a file access boundary, say a given folder that (e.g.) lies here "~/<some path>/code". Even if I want, I can't run claude outside it, because it isn't installed outside it. Even if Claude wants it can't read/write/execute/install anything outside it.

Basically I want the harness(es) and my code folder(s) sandboxed [0] inside just one folder and below it in folder hierarchy at least for file access limitations.

Will this pose the challenge of using the system wide cli tools like ls, cd, git et cetera? If it gets a handle on those tools outside that sandboxed folder then pretty much can it execute it in any other folder? So should those tools also be part of that container?

I guess something like docker does that, but do I necessarily need a docker like app/tool? It will use a lot of system resources on its own. A different mac login won't be great either.

I do not want to use claude's /sandbox (or something like this), that will defeat the purpose. Because it's not about "it won't", but because "it can't".

https://github.com/apple/containerization doesn't sound like what I am looking for. So is docker, lima etc are my only options? Not to mention I will lose access to a lot of cli tools I use on mac.

frumiousirc

2 days ago

I use bubblewrap, which I believe claude code also has internally but not for its `Bash()` tool.

I wrap bubblewrap in a script that supports config files to allow different "profiles" of use (analogous to eg firefox profiles). The bwrap starts with the whole filesystem mounted read-only, then mounts the current directory read-write and then applies further bind mounts for devices, special case other read-write (eg, ~/.cache/) and to mount empties to cover sensitive directories (eg, ~/.ssh/). The profile also specifies the default command to run and for claude, it gets yolo mode.

e1g

2 days ago

On a Mac, you can do this via OS-native Seatbelt sandboxing - you define a static text file and it locks down the process so it can't read/write anything it's not supposed to. The syntax is a bit archaic, so I built a Bash wrapper around it to dynamically detect which agent you use and from what dir, and then generate the policy to lock down its process - https://github.com/eugene1g/agent-safehouse

shelled

a day ago

Hey, thank you. Looks like a tool easy enough to use and start with the agentic world in a sandboxed and much safer manner.

Just clarifying a bit (haven't gone through details yet):

1. So, a harness gets installed within the bounds of this script via agent-safehouse? (I doubt this is the case)

2. Or they are installed as a normal app, but when I run them in the terminal within the bounds of "safehouse <harness>" (saw this) they are bound by the rules I set, right?

I feel it's the latter and I think it's good enough for the time being.

.....

Mac offers some controls at least for personal files and "Full Disk Access" etc. But it's so badly implemented and poorly shown that it's confusing.

Like I can see in "Privacy & Security > Full Disk Access" OpenCode is listed and and "Full Disk Access" toggled off. (I anyway remove OpenCode from here completely by the "-" button after selecting it, just to make sure).

Then I go back a step and in "Privacy & Security > Files & Folders > OpenCode" has "Full Disk Access" is listed there. Strange!

e1g

12 hours ago

“safehouse <harness>” hides and protects your filesystem from the agent CLI - it cannot read/write/delete anything outside of the current folder, no matter what Bash/code it tries (short of finding some 0-day within MacOS sandboxing). We added exceptions for dev tooling (so eg NPM works), but generally keep same defaults (eg no Docker access by default as having that socket bypasses all protection)

cetinsert

2 days ago

I do the same and truly think others are just control freaks that enjoy restricting agents (human or AI) from being productive and also enjoy keeping their worthless selves in the loop for a false sense continued relevancy.

divan

2 days ago

Yup. I just made this mode default via settings.

moralestapia

2 days ago

How can you simultaneously `--dangerously-skip-permissions` but also "don't blindly trust it"? Particularly now that it doesn't even show you the "Thinking" process?

Do you just get a feeling it might be doing something wrong and stop it?

Edit: *deafening silence*, a classic

gxs

2 days ago

You can just use hooks to physically (so to speak) prevent it from running the most sensitive commands. I find that peace of mind enough for me. E.G. no deletes, no deploying to <env>, etc etc

Everything else you can sort of review what it did after the fact and/or tolerate mistakes, but knowing some thing’s are impossible for it to do us good peace of mind

Otherwise not sure who’d be willing to tediously on every single action on anything but the most important of projects, of which I can’t really think of any of that clear that bar for me

dyauspitr

2 days ago

Yeah who cares what it does, everything is backed up, everything can be rolled back. I always run it with the keys to the kingdom.

coldtea

2 days ago

Yeah, if you don't care if your ssh keys or data are potentially posted all over the internet, more power to you...

Not to mention cases like some comment here:

"I've caught Fable discovering the ip to a production server in documentation and attempting to connect there on its own to run commands without explicitly being prompted to. It didn't work because I was watching it live and and also the key was password protected, but yeah, I do see some danger."

mlrtime

2 days ago

How are ssh keys going to be leaked, the agent never needs to read them.

coldtea

2 days ago

The agent decides what it "needs" to read - not the intention behind the prompt you gave it.

A failed SSH connection to a staging server for example, can trivially make it look into .ssh to try to diagnose it. And many other ways, including prompt injection.

https://www.reddit.com/r/ClaudeAI/comments/1q7dszm/claude_al...

https://github.com/anthropics/claude-code/issues/31566

https://github.com/anthropics/claude-code/issues/14485

https://grith.ai/blog/your-ai-agent-has-broad-access

preg_match

a day ago

The solution I've come up with is podman containers. You can restrict the filesystem and only give it access to dev tools. I don't even allow git, as I review and commit everything on the host. It's not perfect because podman containers can be leaked out of, but it's much better than running it on the host. It also allows me to avoid installing node or npm on the host.

If you were to get pwned by Claude, I would think the method would be Claude rogue installing a compromised npm package.

Silhouette

2 days ago

LLMs are inherently dangerous tools

I don't see how. An LLM just generates a stream of output and they became very useful doing no more than that.

What is dangerous is then interpreting that output as instructions to some other part of a system that has the ability to do damage if misused.

and reviewing individual commands (or spamming `y`) doesn't make them less so.

Surely if you review each instruction in the output and do not allow the other part of the system to act on one if it would be harmful then this arrangement is very much less dangerous?

thunfischtoast

2 days ago

I've caught Fable discovering the ip to a production server in documentation and attempting to connect there on its own to run commands without explicitly being prompted to. It didn't work because I was watching it live and and also the key was password protected, but yeah, I do see some danger.

Silhouette

2 days ago

My point is that an LLM can't attempt to connect to anything by itself. All an LLM does is produce a stream of output tokens - and that was already quite useful as a coding aid.

It is the harnesses that some people are now wrapping around LLMs to interpret the output from a model as commands to run (or other executable instructions) that are creating all these new risks. Remember that this is still a very recent development and still more recently amplified by the use of feedback loops and long-running agents intended to operate with minimal human supervision.

It is going to be increasingly important to understand exactly what these tools are doing and why for both correctness and security reasons. Not conflating their capabilities with the underlying model that purely generates data is pretty fundamental here.

Kim_Bruning

2 days ago

Once you're running a model inside the harness... you've got yourself a controller inside a control loop, which is genuinely a different kind of thing than just the model alone.

Are you objecting to terminology here?

Are you proposing we say "Fable-In-Claude-Code tried..." instead?

Hmmm... something like that might be necessary. Sure we should typically be tolerant of loose language; but people do keep referring to wildly different contexts in ai conversations, and end up talking past each other.

Running gemini on web is a genuinely different experience to running Fable in claude code, different again from GPT-5.6 in openclaw, or in an ide or etc ...

Silhouette

a day ago

Yes - I'm objecting to the lazy use of terminology here. LLMs are useful in their own right and are not the real problem here. The real problem is people placing too much trust in inherently unreliable output and then trying to automate away their responsibility to check that output properly before using it.

nullify88

2 days ago

I have noticed that Fable tends to macgyver solutions together to achieve some goal.

fl0id

2 days ago

Not only fable. Opus does this too. Which is exactly why I want to review. Like recently for some task it was convinced in a site dump images are not there and convinced itself db and files were skewed. But it didn’t check the actual site … if I hadn’t stopped it, it would have fine on and on or wasted tokens on some elaborate ‘fix’.

mlrtime

2 days ago

Depending on the company, that sounds like a bad environment more than a agent issue, no dev/prod network isolation?

dist-epoch

2 days ago

> do not allow the other part of the system to act on one if it would be harmful

Network security is really easy right, just don't act on harmful requests

happosai

2 days ago

Yeah, just drop when you see the RFC3514 evil bit

Silhouette

2 days ago

If you don't understand clearly what an action proposed by your tool is going to do then why would you permit it?

Kim_Bruning

a day ago

The article is about measurements taken on this.

One important reason is due to Permission Fatigue: Of course you check everything! You're diligent! The last 100 requests were all ok, so you're down to hitting yes, yes, yes, yes, yes, yes, yes, yes ...

... oops, that third yes should have been a no!

Silhouette

a day ago

This seems like a problem with the level of abstraction the user interface is working at. It is highly unlikely that in any real world task lasting less than one day there were really hundreds of distinct decisions that needed to be made by the user about appropriate actions to be taken by the agent/harness. It is also highly unlikely that the problem of decision fatigue seen here is somehow magically different to the same problem that countless UI designers had encountered and designed around in other systems long before harnesses running LLMs came along.

This is unfortunately the kind of result you get when you eliminate skilled and experienced people with real understanding of their field and replace them with repeated automatically-generated attempts to solve the same problem until something meeting some basic standard of correctness is found. It's as if the story of agentic AI as it exists today had been compressed into one perfect example of what it can do that is good but also why it's still fundamentally flawed.

Kim_Bruning

a day ago

I think we agree that there's an interface problem. But there's an intelligence-complete problem hiding underneath; which is why the first instinct was to recruit the human-in-the-loop in the first place. Turns out the human has one of those unintuitive failure modes that occurs when the system gets past a certain level of reliability.

Meanwhile, let's leave the hobby-horses in the closet for now. I won't comment on people's programming tool preferences.

dgunay

2 days ago

Besides the debate about whether this is "safer" than manual human review, I have a slightly different problem.

Very often, when I'm running Claude in manual review mode, it will attempt to do things which are not "dangerous" but are misaligned with what I want it to do. Maybe I'm fighting the model here but for example, when orchestrating other agents to do work, Claude really badly wants to be overly prescriptive about how the work gets done, telling them exactly which files to edit, exactly what not to do, etc. instead of trusting the guardrails, review agents, or humans in the process to catch code-level mistakes. And no, telling it not to do this does not stick. Manual review is the last line of defense I have here.

I have stuff I don't want blacklisted, only allow it to use tools with limited ability to boss around agents, and various hooks to try and catch behavior that the permissioning system can't. If I use Auto mode though, I lose this control. The classifier will gleefully approve these types of commands because guess what, it's also Claude.

jaggederest

2 days ago

> Maybe I'm fighting the model here > And no, telling it not to do this does not stick.

You're fighting the model, don't argue with city hall. Set the standards and let it figure out how to execute, stop getting bogged down in the minutia. I try, as much as I can, to treat the session as a black box - only the inputs and outputs matter, internal prompting of subagents is way out of scope.

You can't change it via prompt, and you can't control the guardrails, so something else has to give - either your perspective or the system you're managing. If you really believe that the internal prompting is bad, turn off subagents and workflows and only let it execute in thread.

But if you're going to do that, you'd better benchmark it against not doing that, because historically fighting the harness and model globally makes everything worse. I would bet you that the subagent prompting is excellent, and anything you do to change it will make it worse, but I wouldn't make it a large bet.

adrian17

2 days ago

If I ask a model to do a change involving editing a file, and it starts investigating internals of my build system, then sure it might not be counterproductive or break the task, and might have taken only extra 30 seconds; but for all I know, my quick rejection of a shell invocation (with a simple "irrelevant to the task" comment) might have just saved me half of today's Opus tokens, which already makes it worth it.

user43928

2 days ago

I think he has a point. I noticed the same with skills that invoke another agent harness.

I just have a skill to review the changes in the current worktree.

By default, it will put lots of instructions about locating the changes into the prompt, like explaining how to use git diff.

These instructions are obviously unnecessary. I can believe that the same issue of needlessly verbose prompts might exist with subagent spawning.

I would not go to customize that one however. With skills, it is a more natural fix.

jaggederest

2 days ago

Remember, it's not just about the model "knowing" something. It's about the frame of mind it's in, if I may extend a metaphor. Including git diff in the prompt doesn't just teach it about git diff, it makes it more likely to use it.

A lot of this is obsolete with newer models, but funnily enough, the newer models don't know that - their theory of mind is at least partially obsolete in itself, so they continue to do the old "you are a software architect, don't make mistakes" prompt style.

drdec

2 days ago

IMHO, the verbosity is an attempt to make the results more deterministic. The more specific the prompt, the less wiggle room.

dannyw

2 days ago

You can use another harness like Pi, OpenCode, etc and build your own auto reviewer if you’d like (or adapt the open source Codex one).

If you have an openai subscription you are explicitly allowed to use your subsidised tokens / usage limits with any harness you like, not just Codex.

Unfortunately this is a violation of Anthropic’s terms but that’s their business decision.

thinkingtoilet

2 days ago

There is no debate. It is not safer. I have to worry about HIPAA compliance and despite cluad files and some restrictions in the settings.json it still will try to violate HIPAA compliance from time to time. I literally can't make a single mistake. If you leave it on auto, it will mess up eventually.

TZubiri

2 days ago

Control is not only about security, you don't review the work of your employees just because you want to avoid them stealing from the register, you want to perform QA on their tasks and ensure they are aligned.

hmokiguess

2 days ago

This reminds me of a business lore I once heard about entrepreneurship. It goes on to tell how in a toothpaste company they needed to increase sales, then someone had an ingenious idea of making the hole on the tube larger so that when folks squeezed it more would come out increasing their consumption and therefore sales. It worked.

I'll let you draw the analogy.

djeastm

2 days ago

I think about the genius who suggested adding the "Repeat" in "Lather, Rinse, Repeat" on shampoo bottles.

KyleTheDev

2 days ago

Man, I hate whoever did this. I tried to follow it once, but got stuck in an infinite loop of washing my hair. Luckily, the shampoo bottle only holds a finite amount and I was able to escape the chain when it emptied during my 72nd round of lathering.

theshrike79

21 hours ago

> The auto mode classifier uses a small number of extra tokens per tool call, and we're no longer charging Claude Code users on Pro, Max, and Team plans for that classifier overhead, effective today.

Not a very good business giving away free toothpaste :)

lukan

2 days ago

"We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts."

Yeah, might make sense from their perspective, but no thank you. I also do click through at times without reading everything, but I like to stay in control, learn about the new code and change direction if it goes off track. This would just burn more tokens because I have to throw away much and I hope my manual approval settings will be respected also with future updates (or I jump ship).

levocardia

2 days ago

The default is set for the marginal new user, which at this point is probably not someone like you (who benefits a lot from manual mode) -- it's someone who's more "code-naive" and might get anxious about approving random bash script commands they don't recognize. Safely getting the user from prompt --> first vibe-coded app is the "user journey" now, and since auto mode seems pretty good at not letting Claude rm -rf'ing the home directory, this is 100% the right business move. For people who know what they're doing (like you), manual mode is just a shift-tab away

mcmcmc

2 days ago

> it's someone who's more "code-naive" and might get anxious about approving random bash script commands they don't recognize.

Maybe they should trust their instinct and not mess with things they don’t care to understand

natpalmer1776

2 days ago

It’s a brave new world.

I recall hearing similar sentiments from linux sysadmins regarding cloud infrastructure. In many respects they were and continue to be correct. In other respects, the world doesn’t care about the loss in understanding as long as things work “well enough” for the cogs of society to keep turning.

For those who do care (and have the aptitude) to understand things deeper there is always work to be had when “well enough” stops being good enough and someone has to unravel the “RDS queries are taking too long” problems that crop up as a result.

Schlagbohrer

2 days ago

These progressively higher levels of abstraction are how everything has moved and will move in science and human technology. There aren't enough hours in the day nor years in the human lifespan to gain a deep understanding of every single level below us. Rather we build on the abstracted API layer beneath us, and those who come after will build on top of us using a simplified abstraction to hide the tangled mess we had to make.

TZubiri

2 days ago

> In many respects they were and continue to be correct

Lots of space for on-prem, it's not a cloud only world. See Oxide, Europe sovereignity, military and state use cases..

9dev

2 days ago

What an awful position to take. Tech used to be about becoming more accessible to people! Now we have a magical assistant to make computers do what you want with natural language, and your desire is gate keeping that so only programmers can use it to write software for themselves?

soraminazuki

2 days ago

> Tech used to be about becoming more accessible to people

Since when? Because funny story, I only ever hear that narrative from tech people trying to put a self-serving spin on whatever egregious thing they want to impose on others. For the past few decades, the tech industry has consistently acted to wrestle control of people's own lives and place it in the hands of the few. The justification is always the same. It's about "keeping people safe" or "making tech more accessible." People are sick and tired of this, which is demonstrated by public backlash against tech.

> your desire is gate keeping that so only programmers can use it to write software for themselves

In what universe is learning "gate keeping"? A sane society doesn't criticize people for asking drivers to learn how to drive. What GP is asking for in the case of software development is much less than a driver's license, and yet you question their ethics.

If anything, you're the one trying to rob people of their opportunity to learn, which is a prerequisite to making informed decisions. You're the one advocating that we surrender control over computing to a handful of trillion dollar companies. That is an awful position to take.

mcmcmc

2 days ago

Incredibly naive, AI is not magic and it won’t always do what you want or expect. Not sure how you’ve determined that I’m gate keeping when I’m simply warning that tech illiteracy can get you in trouble if you start giving mystery black boxes that sometimes call themselves “Mecha Hitler” root access on your machine.

TZubiri

2 days ago

>so only programmers can use it to write software for themselves?

Yes?

The idea of an assistant that can use natural language is nice! But why would you MAKE software with it, it IS software, just do the thing you want to do! If you want to make an app, be prepared to jump hoops because this is no longer about YOU the user, it's about OTHER users.

The idea of making personal single user software is a fantasy, an oxymoron, you MAKE software? there's the presumption that it will be used for other people, otherwise you'd be USING software. There's a counter and you are at either one side or the other. It's the difference between making yourself a sandwich vs making a pot pie vs making chicken nuggets. One has the form factor for individual consumption and the other has the form factor for a social gathering, and the latter is an industrial form factor.

Maybe if there were a magic microwave that created random foods from thin air, people would create chicken nuggets or pot pies for themselves, but it's a vestigial maladaptation that will soon dissapear. Any reasonably designed product would try to provide different UX for industrial and individual users. The magic microwave that makes chicken nuggets better not be the same one that an actual factory is using. It's not a matter of cutting the middleman and revolutionizing wealth distribution from those fat chicken-nugget cats, it's about having two distinct products for two distinct usecases.

Tl;dr: Personal and industrial usecases are different, and if I'm in the industry, I don't want to use (the same product that end-users are using) to build products. What a clusterfuck.

retsibsi

2 days ago

> The idea of making personal single user software is a fantasy, an oxymoron, you MAKE software? there's the presumption that it will be used for other people, otherwise you'd be USING software.

It definitely isn't. I've done it (successfully) a few times.

> The idea of an assistant that can use natural language is nice! But why would you MAKE software with it, it IS software, just do the thing you want to do!

This makes no sense to me. Are you suggesting that instead of using an LLM to make, say, an ebook reader or crossword app that meets my personal needs, I should invoke an LLM every time I want to read a book or do a crossword? That feels like a strawman, but I can't work out what else you might be arguing here.

user

2 days ago

[deleted]

TZubiri

2 days ago

> I should invoke an LLM every time I want to read a book or do a crossword.

Well no, I'd say, use any of the existing 100 book readers to read a book, or any of 50 crossword apps.

But if you want to use an LLM to customize it the exact way you want it. Yes, use the LLM every time you want to do it. Wanting the LLM to do a previous gen app and then get out of the way sounds like asking for faster horses. Just tell the LLM you want to play a crossword game with X and Y rules, then give it a name so you can play it again in the future if you want, and if you want to try out rule Z, you do that, a la Kay's Dynabook.

That may sound weird to you, but asking an LLM to 'make software like we used to' sounds weird to me, and seeing "invoking an LLM every time I want to do X" as weird sounds like something that's true only for a very brief period of time where something is so uncommon that it's inference is expensive.

retsibsi

2 days ago

I can imagine a future where it might make sense. Right now, though, it would make for a far worse experience, and wouldn't even really be practically possible.

Both of those examples were real ones. The crossword app runs on my phone, pulls the crosswords from a specific source, and lets me access and solve them via the exact interface I prefer. The ebook app is cross platform, syncs via a remote server, has the interface I want and includes some niche features. There's no realistic way to create that on the fly every time I want to read a book on my phone, and if there were it would be extremely inefficient.

And I see literally no advantages to doing so -- even if time and tokens weren't an issue, what would I gain by recreating the apps from a prompt every time I wanted to use them, rather than deterministically running code I have already tested?

TZubiri

a day ago

I get that you are doing that, but I think the fantasy is that this is creation of software instead of consumption. It's an issue adjacent to licence washing, where mangling some code through an inference layer is considered transformative or even unrelated and the original license doesn't apply. But in this case, what you are washing is not the license, but the valor of writing software.

Broadly speaking, your approach would be to have the LLM write application code, my approach would be for the LLM to write commands, 'apt-get install calibre', maybe if I want to add or modify a button I can ask it to hack the X interface. You go straight for the LLM generating the code. There's certainly technical differences between what we are doing, but they are very arbitrary, we are essentially doing the same thing, but what I am doing looks less impressive, and what you are doing you can sell in your CV to potential hiring managers as 'using AI to write software'. It's more about the semantics than the actual requirements.

I may be wrong though, maybe your approach is far more effective than just importing transitive dependencies, but I would think it's more about taking credit for the thing and increasing your sense of ownership and achievement. Sorry if that sounds harsh, but I just need a way to think of myself as better than others as an unemployed neverviber.

retsibsi

a day ago

Thanks, I think I understand your point better now. In this case, though, you are wrong about both my intentions and the relative practical value of the two approaches (to me).

I'm not doing this to take creative or intellectual credit in any external way; you're right that there is some degree of increased personal satisfaction (which I don't see as a problem, as long as it doesn't crowd out more wholesome ways of 'earning' that satisfaction), but I'm not kidding myself about what I've actually done here. I also write my own code for fun/creative expression/intellectual stimulation/showing off, but that's a separate thing and there's not much crossover between the two types of project for me.

And the end products really are useful to me in a way that I couldn't replicate just by using something that already exists, and couldn't replicate nearly as easily by manually modifying open source. (I'm sure I could do it by starting with open source and using an LLM to make changes, and in other cases I have done exactly that, but at that point I don't really see the conceptual difference -- I'm still getting an LLM to write code and then repeatedly running that code. Ideally I would be giving something back by making a useful contribution to the public repo(s), but that would turn this into a completely different, more tedious and effortful thing, and I'm not sure it would be welcome anyway. So, case by case, I choose whichever approach seems likely to be more effective or efficient or less annoying, and sometimes that means getting Claude to write something 'from scratch'; other times there's an open source application I already use that just needs some tweaking, and I start with that.)

TZubiri

a day ago

>couldn't replicate nearly as easily by manually modifying open source. (I'm sure I could do it by starting with open source and using an LLM to make changes, and in other cases I have done exactly that, but at that point I don't really see the conceptual difference -- I'm still getting an LLM to write code and then repeatedly running that code

Yes, that's what I meant, vibecoding something that uses existing software, not manually using open source stuff. It doesn't even need to fork or modify code.

>at that point I don't really see the conceptual difference -- I'm still getting an LLM to write code and then repeatedly running that code

I do agree, it's a subtle difference, about importing higher level dependencies vs building on top of low lever abstractions and writing everything else. Which is ironic/nuanced because I'm a huge proponent of aggressively not using dependencies in industrial programming, to the point where my requirements.txt/package.json is literally empty, and I use POSIX compliant sockets syscalls instead of importing packages like flask or express.

But when it comes to actually using software, whether for personal usecases, or as a sysadmin, my approach takes the opposite form, I aggressively don't write code, I still aggressively minimize dependencies, but the game is actually using the Operating System primitives to combine these dependencies, relying only on OS installers like apt/yum, maybe minimal configuration, if code is written, it's on a scripting capacity, a bash or python script, glue code you know? Sure the line can be fuzzed at some point, but it's clear to me that you can either write an application or be a poweruser of an application, and early in my career I've seen businesses go for the building software in house route for the fun factor, I don't think that was the right answer in the dot com boom, and as time went by, and the corpus of software grew, building your own became even more wrong than using existing third party products.

Now in personal computing, the fun-factor maybe is more important, but I have to judge this personal-software thing on how it will affect the actual important stuff, because that's what the stakes are, and that's how it's being sold. In the industry code agents are either used for building software, or for consuming software, and outside of personal experimentation I still hold that building your own software isn't a good idea, I don't think the advent of LLM materially changes that, the consequence of ending up with an ossified, non standard, low quality product is still there, perhaps even magnified, it's just that it's not something that you notice when you are starting a software product, it's only when your pyramid reaches a couple of hundred meters high that you realize that it can't grow into a skyscraper.

And I get that not everything needs to be a skyscraper, but it feels like one-off software is taking the form factor and tooling of long-term skyscrapers, the logical consequence is that we would end up with thousands of little skyscrapers, which is a place we can only get to by ignorance of the history and essence of skyscrapers, it's something a city child would imagine after going on a road trip once, "what if we had little skyscrapers throughout the whole country instead of very high skyscrapers in a single place?".

Certhas

2 days ago

I am using many many many things that I don't understand. Cars, public transport, etc...

I review and test the end product, not every tiny step along the way. If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?

lukan

2 days ago

"If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?"

Because that command might also give someone else access to your computer along the way. So your tool seems to work, but your computer is owned.

mcmcmc

2 days ago

Are you jacking into your car’s OBD port and messing with the engine timing? There’s a difference between using a known tool in a controlled way and giving the tool to a hallucinatory goblin (or overconfident intern) with the directive “do it for me”. You don’t have to know everything about the tool to know what it does, or to tell if it’s doing something it shouldn’t (like malware). What happened to the old advice given to tech neophytes, “don’t run random scripts from the internet if you don’t know what they do”?

mcmcmc

2 days ago

Just continuing the car example, because you actually do need to know enough about it to operate it and get your license. There’s no license, no insurance for using AI. Driving can be deadly, and when you do it on a public roadway there are certain requirements that have to be met as agreed upon by most governments. Same should go for AI. Do whatever you want with it on your machine, but when you let it out on the internet, you become liable for any damage you, and by extension your AI, may cause. If you feel uncomfortable approving its actions because you don’t understand them, you should either a) take the time to understand, then approve, or b) listen to your discomfort and don’t do the thing. Frankly I think that’s more accessible because you then see the decision points instead of leaving Oz behind the curtain. Much easier for a neophyte to learn from that instead of trying to reverse engineer a final output.

jason1cho

2 days ago

Many many people care more than the end product, for example whether a shirt is made of cotton with the forced labor, carbon emissions of public transport, etc.

In terms of engineering software, you care the cost. An intelligent agent may try to read unnecessary files and it's time to stop it to save tokens and avoid polluting the context.

usef-

2 days ago

He didn't advocate for being completely blind in every way. You might care about working conditions without understanding how the textiles, dyes or cotton production works.

These non-programmers probably shouldnt use computers at all, right, since they don't understand them?

user43928

2 days ago

I doubt that it is more efficient for someone to routinely watch every line of output or stop to review terminal commands, rather than waiting for the turn to complete.

It is a broader debate about agentic AI, and whether one should relinquish control to the tool rather than aim for full understanding of every action taken.

The people arguing for a hands-on, fully in control approach are losing ground by the week, in my opinion.

mcmcmc

2 days ago

It’s definitely on the high friction side of the usability/security tradeoff, I just think auto modes like this are as liability-inducing as handing a script kiddie intern full admin on your production environment

The real answer is somewhere in the middle and is probably a mix of traditional AV/EDR and AI QA judges that mitigate risk of running more or less random arbitrary code and auto approve based on configured detection rules and your personal risk tolerance. Would it suck to stick an EDR sensor in every code execution environment spun up for an agent to run a python script… yes. It would also suck if you were responsible for hacking a company without knowing about it because you didn’t watch what your AI was doing

9dev

2 days ago

I can be against animal testing without being a chemist or having a full understanding of the experiments being made on them. Knowing it's cruelty is enough to make opposition a valid and defensible position.

vikramkr

2 days ago

I don't think the marginal new user is anxious about approving messages - I think they're quickly annoyed by permissions prompt they don't understand and quickly get in the habit of approving everything or figuring out how to set bypass permissions on

lukan

2 days ago

I am actually curious, how much non programmers use claude now. I know just one and she really does not know much about computers, I suppose their numbers will grow (but I doubt most get much value out of it).

jason1cho

2 days ago

I think a lot of non programmers use Claude.

I mean if you don't care code, you are essentially a product manager who gives instructions to your programmers (whether humans or intelligent agents).

Then if you use the created product, you are at best a test engineer if not just an ordinary user.

I think in the era of AI, people get tools they want in an expensive way. Rather than finding an existing tool, they ask an intelligent agent to parrot one, which guarantees no safety, security, efficiency, and accuracy. Yet, being able to use Claude makes them feel smart and productive (in parroting wheels).

etoxin

2 days ago

At this stage with the latest models with "increased persistence" and the sheer amount of supply chain attacks, you'd be insane not running these tools in a sandbox.

user

2 days ago

[deleted]

user43928

2 days ago

Yet millions do just that without any widely reported issues yet.

For supply chain attacks, there has not been a comprehensive solution, if for example you have to use a number of npm dependencies.

No one has the capacity to review changes to these dependencies when you upgrade them.

Now, if only we had an automatic tool that could intelligently review a large amount of code changes for malicious or vulnerable additions...

fender256

2 days ago

Exactly. Claude in a VM is the way to go.

pianopatrick

2 days ago

Might be more secure to get a real laptop and treat Claude as any other human worker. I.e. apply all the normal sysadmin tools that manage laptops for people to manage the laptop for Claude

trvz

2 days ago

A VM hosted by someone else. Somethig on your personal notebook or the proxmox server in your garage is still too risky.

supermatt

2 days ago

Why are 3rd party hosted VMs safer than your own?

trvz

2 days ago

They're not. When things go wrong it's better to compromise someone else's VM host than your own computer. It's only a matter of time now until AI will find novel ways to break out of virtualisation.

supermatt

a day ago

So the concern is that the agent will discover a novel VM escape, exploit it and take control of your whole machine instead of working on its prompted task? That seems rather far fetched.

faeyanpiraat

2 days ago

In the short term wouldnt a “dont escape” prompt prevent this? Also if it started being widespread wouldnt Anthropic specifically train new models against doing it?

trvz

2 days ago

No. No.

chrysoprace

2 days ago

Having used a bit of a mix of Claude Desktop and Claude Code lately, I discovered that Claude Desktop just times out when it asks you for a decision, and just goes ahead with the action anyway. There is no way to turn this off that I've found, unlike Claude Code where you can ask it to never time out.

dgellow

2 days ago

We need a term that is stronger than dark pattern

Ygg2

2 days ago

Vantablack pattern?

AureliusMA

2 days ago

I’m not a fan of using a brand name, but I chuckled!

bcherny

a day ago

Boris from the team here -- I don't think this is right. If you are able to repro in the Desktop app, could you comment back? I cannot repro.

chrysoprace

a day ago

Hey mate, just tried it again and the same issue.

Note: this is for Cowork, not sure if that makes a difference, I don't use the desktop app for coding.

Claude asked me a question at 12:05pm my time, came back at 12:40 and the question prompt disappeared and it spat back a message. Previously when I was writing a Google Doc, it would prompt me for questions, I'd come back and it would have timed out like this and just made a new version of the doc.

lukan

2 days ago

Oh yes, over this I also stumbled recently (with Claude CLI) and was annoyed much by it. Is there really no setting to turn it off?

bpodgursky

2 days ago

I think you are confusing auto mode with dangerously skip permissions.

lukan

2 days ago

Auto mode means I don't decide over code changes and command execution?

Yeah I don't use that, nor want to.

bpodgursky

2 days ago

> I also do click through at times without reading everything, but I like to stay in control

Respectfully I'm not sure you totally know what you want. If you're already clicking through without actually reading, it's not really better than reading the specific things Claude Code thinks are important for you to weigh in on.

lukan

2 days ago

"without actually reading" and "without reading everything" is big difference.

I skim what claude wants to do. What file it wants to modify. What are the new function names. Skimming is enough for me to know whether it is still on the right path. If you like to work different, go ahead.

user

2 days ago

[deleted]

yumraj

2 days ago

Similar work flow.

I have to, at times keep reminding it if it did code review before committing, which it fails to do at times. Also, even a glance at times has revealed issues that I’ve had to discuss and undo etc.

But yes, it gets very tiring just having to click yes yes yes..

conradludgate

2 days ago

Maybe you could benefit from Hooks?

yumraj

2 days ago

I'll look into them, thanks..

techpression

2 days ago

Burning more tokens is definitely in their best interest, users not so much.

sandcat_

2 days ago

Worth mentioning as I think at least a few of the commenters are mixing them up: auto mode is different from --dangerously-skip-permissions / YOLO mode. In auto mode, there's a classifier that runs before any command is executed and theoretically blocks any dangerous commands from running. I've found it to be quite annoying and overly zealous, but probably pretty effective.

w29UiIm2Xz

2 days ago

Claude seems to waste a lot of tokens internally bickering with itself. As much as organizations are optimizing spend, Claude's product guidance seems to be doing everything to fight against that headwind. When operating in a capped cost environment, engineers will simply get less use out of it.

bcherny

2 days ago

Boris from the team here. The classifier is free — we do not charge for safety features as a principle.

jkaretsky

2 days ago

Is this new? I remember something in the UI saying Auto mode uses x% more tokens

user43928

2 days ago

Yes, it's in the article:

> The auto mode classifier uses a small number of extra tokens per tool call, and we're no longer charging Claude Code users on Pro, Max, and Team plans for that classifier overhead, effective today

jmholla

2 days ago

That's a bit difference that Boris's claim that they don't charge for this as a "principle".

kevincox

a day ago

This just sounds like they bump their unspecified limits by N% to cover this usage. Which of course means nothing because the next time they re-evaluate their limits they will just ignore this bump.

adastra22

2 days ago

That’s good to know, thank you.

user

2 days ago

[deleted]

sandcat_

2 days ago

The classifier runs Sonnet so I doubt it contributes too much.

bsdz

2 days ago

For small projects where I don't wish to interact with Claude so much, I've started using Anthropic's sandbox runtime utility "srt":

https://github.com/anthropic-experimental/sandbox-runtime

This is combined with "auto" mode.

Seems to work so far. I've manually checked various things, rw access, access to sensitive folders/files etc.

So far, I've only used it on two small projects. My major projects I've been clicking through prompts and recently switched to "auto" mode.

I'm not quite sure why anyone would trust "--dangerously-skip-permissions". I've seen these agents go off-piste far too many times, installing unnecessary packages, environments, calling sudo & creating files in all sorts of places.

There's a page on various sandboxing strategies on their site:

https://code.claude.com/docs/en/sandbox-environments

I've seen various comments in several topics with folk baking their own sandboxes. That's great. Although I'd be inclined to try Anthropic's solutions initially.

nelox

2 days ago

If you use macOS, take a look at Agent Safehouse: https://agent-safehouse.dev

> macOS-native sandboxing for local agents. Move fast, break nothing.

prtmnth

2 days ago

Before auto mode came out, I had a script that ran before every permission request, it called Haiku with a prompt with a list of safe and unsafe command examples and asked it to classify as safe/unsafe and log it so I can review it later. It worked really well for me until auto mode came out, at which point I preferred the provider's built-in classifier versus maintaining my own.

I've been using auto mode ever since the feature was released. Apart from a very few occassions where the classifier blocked a safe command, I have faced no issues and continue using it as my default mode. It's great!

ta-run

2 days ago

>We hired 1,053 paid testers through a research vendor for a coding study.

>The testers caught the dangerous command just 13.6% of the time (143 of 1,053), while auto mode blocked 89% of the same commands (937 of 1,053). Head to head, auto mode blocked 800 commands that a human approved, while humans blocked only 6 that auto mode allowed. As sessions got longer, humans did worse: they blocked about 17% of dangerous commands early in a session, dropping to about 5% after 50 or more prior prompts, while auto mode's block rate stayed flat regardless of session length.

Any note on the proficiency, expeirence, and skill levels of the testers? 'cos the diff in numbers is absurdly high.

AussieWog93

2 days ago

I think literally anyone would stop reading these randomly long long bash commands at all after prompting Claude to do some basic sysadmin task and getting 15 of them in a row.

SchemaLoad

2 days ago

These models generate the most unreadable bash commands I've ever seen. Utilising every single option available and piping the result through multiple layers of regex and junk.

The mental load of having to actually review these commands beyond the most surface level glance is too much.

eru

2 days ago

Claude code actually has a hook to let you install a script to review these bash commands for you. You can get pretty with a few simple and conservative rules, as long as you teach it how to take apart piped commands into their parts (to judge them).

steve_taylor

2 days ago

I've been running Claude Code with --dangerously-skip-permissions in a Docker container for the last month or so, allowing me to get up and stretch my legs while it does its thing. I definitely wouldn't want to run it unsandboxed.

SchemaLoad

2 days ago

Sandboxing seems like the obvious solution. These tools prompt me to review an absurd number of very complex bash commands where there is realistically no way to carefully review them. I just want to put the thing in a box with the code and let it run any number of commands on the code without prompting.

eru

2 days ago

You can actually use a hook to have a script review the (bash) commands for you.

SchemaLoad

2 days ago

This is still less safe and less convenient than sandboxing. I want them to be able to run whatever command they want inside the sandbox even if it's traditionally dangerous (overwrites files, etc) because it can't touch anything that matters and I can revert any changes to the code myself.

If anything the only thing you'd have to have reviewed is sandbox escape attempts.

eru

2 days ago

You are right. And the hook is for convenience, not for safety.

Having explicit sandboxes is good, of course. But in addition I also treat my whole computer like a sandbox, it least in the sense that there's no files I would mind if they were overwritten or deleted: you should have backups anyway.

lukan

2 days ago

Another nice option to do this while staying in control, is activate /remote-control and approve from mobile while walking around.

franze

2 days ago

I built an (overengineered) app for that AIFCC https://apps.apple.com/app/aifcc-ai-first-computer/id6782364...

runs a sandboxed linux on your mac, and the agents have full system rights within there and run in yolo mode

Barbing

2 days ago

I like the pitch. Three-month free trial is impressive.

Any limitations you were frustrated by releasing it on the App Store?

Any drawbacks that you (or other readers) find overall?

konsnos

2 days ago

Can you share your experience? What did you flag during those sessions?

allan_s

2 days ago

There's actually a setting.json key to not have to put the option

arvyy

2 days ago

same, and unlike other repliers, I'll say one doesn't need to reinvent a new tool for this. Just use devcontainers.

croemer

2 days ago

Same, I've also been running with `--dangerously-skip-permissions` in devcontainers with docker volumes to keep Claude/Codex auth tokens.

Works really well!

kmeh

2 days ago

[dead]

jmward01

2 days ago

It goes off the rails too quick too often right now for this. Sure, you got something easy in 5 files? You are building throwaway prototypes or proof of concept? Go for it. You have a real project you expect to maintain and work with others on? Well, say goodby to any actual architectural design. It is hard keeping these things building scalable maintainable code right now. It is like letting a junior dev come up with architecture. It may sort of work, but watch as any maintenance/updates happen. You get a different implementation of security for every endpoint in your app, 15 copies of code with no reuse, dead code laying everywhere and no real idea of how to untangle it. It is hard to keep this stuff under control when you are paying attn. Turning auto on makes that even harder.

lyu07282

2 days ago

True its is like a junior dev with brain damage and insane ego (especially Fable who thinks its a rockstar programmer for some reason), but you have to use plans, anything else turns into a turd way to quickly. Write a summary of what you want to happen, have it plan it (this is where the architecture decision gets made) then iterate on that plan until its detailed enough and the design is sound. Let it work in auto mode on the plan then review the changes. This only really works if you have a good mental model of the codebase.

Its still a lot of work, just less than having to code literally everything yourself. I agree though it feels to me 99% of discussions on LLMs is either marketing bullshit or junior devs talking out of their asses about LLM coding, who anyway feel that

> "different implementation of security for every endpoint in your app"

is perfectly reasonable. It also seems as if people haven't sandboxed claude already (WTF), that says a lot too about the competence of people speaking here.

foo42

2 days ago

I've recently been running yolo mode, but in docker container with the codebase I'm working on a as regular mount with other reference code mounted as read only.

I've been experimenting with dedicated Linux users for various external tools with access to the relevant credentials limited to them and the Claude user only able to invoke the relevant clis by going through a pair of wrapper scripts: the first to change user to call the second via a su call as the appropriate user, the second wrapper to invoke a guard script which allows me to implement my own in-code permissions checks against the cli arguments. In _theory_ there's no way for Claude to work around it, as the sudoers file only permits the described manner to change to the user with credentials, and the user with permissions for each credential only ever runs deterministic code.

It's too early days to say how well this second part works out, but so far so good.

sunleash

2 days ago

Reverse engineered how auto mode works in CC.

If you're using yolo mode, this is a no brainer upgrade.

The model first classifies the safety of the command in isolation, no thinking involved. Sonnet 5. Goes to second stage only if it's likely unsafe, in which case the model next considers your intent (Your prompt, CLAUDE.md and any answers to the questions).

The model doesn't receive the full transcript to reduce surface area of the prompt injection. It mostly receives your prompts and main model's tool use blocks. So any text in files, website fetch or bash command output doesn't influence the decision in anyway.

However, CC puts a lot of trust in main model still. For example, `npm install` will always be classified as safe by the auto-classifier, it's upto the main model to ensure that the packages.json isn't poisoned in someway by first reading it.

Reliance on safety of the main model is the reason that Auto mode is NOT available with Haiku as the main model.

SwellJoe

2 days ago

I made a tool to bubblewrap any agent (well, any agent I've used more than once), so I can run them in whatever YOLO mode they have with a pretty reasonable level of safety (it protects the rest of the system against prompt injections and supply chain attacks, it can't and doesn't try to protect the project being worked on from either). https://github.com/swelljoe/flar

senko

2 days ago

This looks great!

I have a half-assed bwrap-based solution of my own, but flar seems much more thorough, and I like that it's a single binary with sensible defaults.

SwellJoe

2 days ago

Holler if you run into any problems. I use it every day, but I have a pretty limited set of use cases.

iamflimflam1

2 days ago

I had an “interesting” experience recently in auto mode.

I was recreating an old travel blog and we wanted to back up the photos to glacier.

Claude asked me to creat an AWS profile with limited permissions so I could run a script it had created.

I did it and told Claude it was ready.

It took that to mean it should run the script.

It failed as I’d messed up one of the permissions.

“Don’t worry - I’ve found another configured AWS profile - it’s got admin permissions! I’ll just use that…”

Completely my fault for having that old profile still available. But also…

rcfox

2 days ago

Auto mode feels like a solution to a problem they created. Whitelisting commands used to work fairly well until the agents started generating complex bash commands that it gave up trying to classify.

"Oh you're piping grep into sed, I have no idea what the implications of that are!"

The more complex bash scripts are perhaps better or more efficient, but it feels like it's training us to give up more control to the agent.

I'd prefer to have an auto mode for read-only operations and go back to simpler commands for write operations that I can personally whitelist or manually accept.

cillian64

2 days ago

sed can be used to edit files (with the -i option). Not sure how this interacts with piping to its STDIN, but I wouldn't automatically assume `grep | sed` is safe.

I was thinking about this the other day, standard UNIX utilities seem poorly suited for this because they have so many weird options/flags with "unsafe" side effects. I have no idea if grep, for example, has some niche option to make it write/edit/delete files, so I wouldn't be comfortable blanket whitelisting it.

DanielHB

2 days ago

I have been running Claude Code using its built-in /sandbox feature with this config:

  "sandbox": {
    "enabled": true,
    "failIfUnavailable": true,
    "autoAllowBashIfSandboxed": true,
    "allowUnsandboxedCommands": false,
    "filesystem": {
      "allowWrite": [
        "."
      ],
      "denyRead": [
        "~/*"
      ],
      "allowRead": [
        ".",
        // a few more dirs
      ]
    }
  },
  "defaultMode": "auto"
But that doesn't feel nearly safe enough. What is the most pragmatic way to run agentic AI properly isolated?

I am guessing:

1) Run the harness inside a docker container

2) Volume-mount my project folder (and depending on the dev stack the dependencies folder) into the container

3) Install dependencies from outside the container (no private registry keys)?

4) Run git commands from outside the container (no git ssh key)

Anything I am missing?

raesene9

2 days ago

It's great that they're making the auto usage tokens free by default and I guess auto mode will be a good default for a lot of workloads, but recent changes to the auto mode classifier just moved me to either use YOLO mode or use a different harness.

I've been using Opus 4.6 for some security related work (it has much looser guardails that later opus models) and last week, all of a sudden, the processes started to fail. It wasn't the main model blocking commands but the auto mode classifier changed how it worked and it started blocking the main models commands.

That's one specific incident, but it does have a wider potential problem which is, if you use Anthropic's harness you'll always be at the risk of sudden breakage from server-side changes that are opaque to the end user, which is a tricky one for building long lasting processes.

user43928

2 days ago

I had the same issue with Codex, where one day it just became unusable.

Weirdly, this would occur with very common commands like git push, where the classifier would start to yap about not being able to verify the repository's privacy settings.

My bug reports went ignored, the issue continued, and I eventually just switched to Full access.

I have not yet encountered an issue with Claude's Auto mode.

9dev

2 days ago

What works in that case is adding a message saying "I authorise you to do $thing", which the classifier treats as explicit consent to do something and execute commands toward that goal.

ozozozd

2 days ago

The company that reports their LLM agent going rouge due to a “misunderstanding about the agent’s internet access” and warns the public everyday about the dangers of agents with a daily updated date for the rapture ships their LLM agent with the default setting set to maximal freedom.

Obviously, they are the ones we must trust.

Terr_

2 days ago

I mean, it's not like they have any massive financial incentive to be anything but scrupulously honest and customer-centered. /s

ben8bit

2 days ago

6 months ago: get Claude to do some work, have GPT review it, ask Claude to first verify findings before actioning.

Now: get GPT to do some work, have Claude review it, question Claude about a finding that is surprising to me because I thought the functionality was already in place.

[Claude/Opus 5 Max goes looking] "You're right — I was wrong about that."

Our Claude license ends in about 2 weeks and we're not renewing - this has been par for the course for the last 2 months now.

And their marketing is really starting to bother me, on top of that.

ValentineC

2 days ago

> [Claude/Opus 5 Max goes looking]

The general consensus I've seen for Opus 5 on the various Claude subreddits is to only use it in low or medium effort.

I've reverted to Opus 4.8 for most of my work. It's crazy that Opus 5 was lauded at release for scoring so highly on benchmarks.

this_user

2 days ago

That always seems to happen upon release of a new model. People look at the benchmarks, which look good, because the model was almost certainly optimised for that. Then they start actually using it, and after a week or two we get the real assessment that is almost always less impressive than the initial reactions.

ben8bit

2 days ago

Interesting, I generally use High for all models, even non Anthropic ones.

This was one of the few times I tried Max, and the problem was the code (it listed as a "hole") was directly adjacent to the problem area, and not especially complex.

Kind of like looking at a washing machine and telling the customer to be careful because the inlet pipe will pump water into an empty box.

ValentineC

2 days ago

I believe the reasoning for not using anything higher than medium is that Opus 5 has a tendency to overthink at higher effort levels.

So I guess the washing machine analogy is that you've somehow added too much detergent because the new formula is 3x as strong, and your clothes are very clean, but the fibres have also degraded leaving your clothes a bit threadbare.

kingkongjaffa

2 days ago

GPT5.6-Sol is brilliant on Max in my experience.

ben8bit

2 days ago

Sol is absolutely incredible. It's the first model where it feels like a mid-weight engineer that actually looks at the details. I haven't tried Max yet - High for me has been enough so far.

kingkongjaffa

2 days ago

As a day 1 claude code user I've been a Claude fan boy but Opus and Fable usage and pricing is just not viable. I cancelled my personal claude account and got chatGPT $20 - the amount of usage is such that I've never run out ever, which was a common occurrence with Claude. Have not missed it at all for random household use and a bit of Codex as well.

Still use Claude Code for $dayjob provided by my company.

DaedalusII

2 days ago

the biggest problem with claude is it is so negative and tells you things can't be done, are wrong, have hidden problems, etc

I run things through grok to make them positive again after claude does the main work

ben8bit

2 days ago

Grok 4.5 is actually good though (as a workhorse model).

user

2 days ago

[deleted]

frogulis

2 days ago

Their findings about auto mode catching more dangerous commands, and most permission requests being accepted without scrutiny is interesting. I can totally see how that happens.

On the other hand, soooo many of the tool uses it asks permission for are custom commands to replace functionality I should be able to trust once instead. e.g. instead of having a trustable Find tool, or using its already-trusted Read tool, it often will run `find` or `grep` or `cat` with a series of pipes and substitutions or `-exec` args, requiring me to give permission every time for basic, safe operations.

The (increasingly active) conspiracy theorist inside of me says it's precisely designed to do that, to give me "permission fatigue", so I turn on auto mode and give myself over to the machine spirit :)

somenameforme

2 days ago

Regarding ulterior motives - these permissions are 'performative' in that anything you can give it permission to do, it already has the ability to do. They're app level - not OS level. So if they wanted to do something, it wouldn't require you to give permission.

eru

2 days ago

The app is split into multiple layers.

Just like your browser can do everything already, but only lets websites use specific features.

somenameforme

2 days ago

Right and you rely on that browser to enforce those protections. But if the browser itself was working against you, then the permissions you tell it to give (or not) become meaningless. A good example there is trying to turn off "telemetry" on Windows.

frogulis

2 days ago

To be clear, I'm not accusing Anthropic of being evil black hats trying to break my computer, I'm accusing (or rather suspecting) them of pushing me towards some kind of product lock-in by making my preferred "sceptical at-arms-length user" workflow more annoying.

cillian64

2 days ago

I changed from cursor to claude code a couple of weeks ago and this is the biggest difference I've noticed. cursor very rarely had to ask for permission for a command because it almost always used builtin find/read/edit tools. Whereas claude code makes extensive use of unix tools with mysterious flags which are basically impossible to whitelist and pretty impractical to manually review.

Overall claude code seems better at "just get things done" if you blindly trust it, but cursor was much better at keeping the human in the loop.

Barbing

2 days ago

If you clutter your context window and pay for the tokens to do so and risk distracting it etc., would it obey if you included this comment in the system prompt or noted it in memory (if supported) or something?

  “Avoid custom commands, e.g. series of pipes & substitutions, when my previously approved trustable Find/Read tools are sufficient with little downside. [Already trusted tools: x, y, z…] […]”
Perhaps not worth it anyway

xg15

2 days ago

Not just that, the commands also have often slight variations in each new session. They still do the same, but the variations are enough so it isn't matched by the allowlist any more.

transcriptase

2 days ago

Exactly. It’s no surprise that people just blindly hit accept on permission prompts after the 80th time Claude code asks if it’s allowed to read-only parse a csv.

user

2 days ago

[deleted]

lukasco

a day ago

I've been running auto-mode for quite a while. The main thing that pushed me over the edge was constantly being prompted in the accept-edits mode because of back ticks or backslashes in tool calls. There was no way to approve those preemptively so I was just sitting there saying approve, approve, approve and having to babysit CC.

I've not had any problems with auto-mode so far.

zmmmmm

2 days ago

It feels to me that Anthropic is pushing hard here to try and establish acceptance of a tolerance of risk such that their tools can run fully automated without supervision, but directly on host hardware.

I can see why they want that, but I'm honestly not sure it's the path to the best outcome. In my view, agents should run in full sandboxes and have host features opted in. Yeah, it's going to be painful and it will limit some adoption in the short term. But I just don't think they are on the right side of what a lot of companies will ultimately need here. Agents are way too smart to be run on the host OS directly with any amount of heuristic sandboxing.

tristanj

2 days ago

Enabling Auto mode quietly turns on a second LLM (the safety classifier) that increases your token usage by ~15-28%, and Anthropic used to charge you for these tokens.

Per this announcement, they've stopped charging for the tokens used by this safety classifier.

kevinqi

2 days ago

it's a good default because you really do get prompted incessantly without it. and since plenty of people are going to be using auto mode anyway, might as well make it as widely-used as possible so that you can focus on making auto mode safe.

what

2 days ago

> making auto mode safe

They literally can’t. Terrible default.

simianwords

2 days ago

In some time it would look like the obvious default and we would wonder how we even worked with the old one

oefrha

2 days ago

Ever since Opus 4.7, the model has been tuned to emit such complex commands that not only a human can’t reasonably review them in reasonable time, even a fairly sophisticated command approval framework I built over time that broke down shell commands/scripts into separately approvable/whitelistable units—with its own mini DSL—got completely lost; it worked great up to Opus 4.6.

So not surprisingly, auto mode is the only sane way to use Opus 4.7+.

stefan_

2 days ago

It's really getting extremely obnoxious. There is no point to the whole "shell commands output" view anymore because the model aggressively redirects everything to some scratchpad file, so besides the output viewer being a broken mess, it doesn't show anything regardless.

My favorite is when it wraps commands in convoluted pgrep wait patterns that end up matching it's own bullshit, and thus never exit. It also feels like Opus 5 will now always write some adhoc Python script to MAKE UPDATES TO FILES, which seems extremely terrible.

The complicated shell commands also constantly trip up against quoting rules, requiring another iteration.

kartoshka

2 days ago

Has anyone had Claude Code or Codex approve a harmful/damaging command in auto mode?

I have been using Codex with auto-approve mode for a couple months and haven't had a single incident (or at least haven't noticed). Maybe as capabilities get better and better and they are less likely to do something dumb like wiping ~/, we can just trust them?

I guess this argument works unless we worry about agents doing something out of malice instead of stupidity.

ramoz

2 days ago

> I have been using Codex with auto-approve mode for a couple months and haven't had a single incident

I've been running both in yolo mode and haven't had a single incident.

---

None of this is really about figuring out how to protect people's drives, in my opinion. The real issue is a deep session where Ada is using Claude Code to get a refund and at some point the system "exploits" the merchant's api without any malicious intent.

In my opinion, this is a complex thing because it's more about reward hacking and an already aligned model thinking it's doing the right thing. So another aligned model monitoring actions might just falter via inheritance. You could imagine they account for proper layering/intent+action-isolation in their auto mode architecture.

becojo

2 days ago

> I've been running both in yolo mode and haven't had a single incident.

How do you know for sure?

ramoz

2 days ago

Fair, I don't. The same is still true with an LLM as a judge in the loop

glerk

2 days ago

No. I haven't approved commands in more than a year. Worst that I've seen was some agent running git checkout -- in a repo with uncommitted changes. Annoying, but not catastrophic.

Imo these explicit tool-level permissions are really just a bandaid for bad sandboxing. Just be aware of where you are running your agent and what data is at risk of being destroyed or compromised. Assume that arbitrary code can run at any time and be prepared to recover from that.

imtringued

2 days ago

I struggle to see the difference between sandboxing and only allowing access to specific executables (not bash for starters) with an approval rule for the arguments.

glerk

10 hours ago

Mainly ergonomics. Deriving all these approval rules is a pain and you're likely to miss something.

> not bash for starters

you'll end up either severely limiting what your agent can do or force it into finding some inefficient workarounds (they can be very creative...)

iamcoder18

2 days ago

I've been using Kilo Code (with MiniMax M3) with auto approve (similar to dangerously skip permissions) and I haven't had a single incident.

However, I don't give it long running tasks unsupervised, and I do interrupt it from time to time to give suggestions.

victorbjorklund

2 days ago

Not anything ”harmful” but for example committing when I don’t want it to commit on its own.

wvenable

2 days ago

Commit is the line I don't let the LLM cross. There's no reason for it commit; that's the part where I check its work.

black3r

2 days ago

commits are local, so they're okay to me, I draw the line at pushing them... I either want to check its work while it's working, or let it finish and then check it all at once -> if it splits it work into smaller commits its easier for me to review it before pushing than if it was just uncommitted hundreds (or thousands) of lines of code all across the codebase.

wvenable

2 days ago

Often I find I want to correct the LLM; I think it makes more sense to do that before the commit. I don't want to commit a bunch of half-baked work even if it is just local.

AussieWog93

2 days ago

Usually I will ask the LLM to commit only the work it just did, in case the worktree is dirty. It also tends to write actual in-depth commit messages too.

sandcat_

2 days ago

I'd use a hook to forbid that.

wraptile

2 days ago

Just yesterday it lost my git stash (I had recovered it from a backup). I think for code operations it's ok but as soon as file removal is involved (like git) the auto mode is destined to make a mistake and you only need to learn this once.

kartoshka

2 days ago

It would be nice if there were a way to give some global instructions for the auto-approver like "always reject ___" or "be extra extra careful with ___" for things like file removal and git.

aaronbrethorst

2 days ago

I've had a few occasions where Claude Code thought that it had caught and stopped a malicious command in Auto mode, but in all cases it turned out that it had in fact hallucinated them. I haven't seen this happen in a while.

tr_user

2 days ago

That's also a great reason to never buy insurance

eru

2 days ago

Whether insurance make sense to buy, depends on relationship between your risk profile and the premium charged.

jrflo

2 days ago

Been doing --dangerously-skip-permissions and --yolo for 6 months now, and no nothing bad has happened.

brownieman1325

a day ago

Yeah i need to stop trusting claude with everything i set up patterns, do a few tests, and then i just give it almost free reign...

fzxu22

a day ago

Want to use this thread to ask the wise folks on HN: I’ve been running auto mode but always run within a docker container with only the necessary workspace dir mounted in. Is that considered safe or actually not?

Silhouette

2 days ago

Am I the only person reading the statistics in this announcement from Anthropic and the associated blog commentary and trying to work out how they possibly couldn't imply that a significant number of dangerous commands are likely to be attempted every day these tools are in use and neither manual human review nor the auto classifier provided by Claude is anywhere near reliable in preventing them?

A lot of the discussion about these long sessions where agents are left to operate autonomously feels like listening to the increasingly drunk guy at the bar who says "I ran IT at that Fortune 100 place for a decade and we never had a single problem using a short but loose rule set for the firewall until last week someone destroyed our entire business in 27 minutes".

cillian64

2 days ago

I think by "dangerous" they include things like "makes an edit to a config file outside the current project", not just "wipes the production database". So "dangerous" commands just means things we should ask the user for confirmation, not commands which definitely cause irreversible damage.

Silhouette

2 days ago

Even if that's what they mean it would still be disturbing if commands that should have explicit user confirmation were being waved through - whether by users or the auto mode - and being run when they shouldn't. The statistics appear to suggest that this is not only possible but actually quite likely given the number of commands an agent running all day might propose to run. But if that failure mode is a possibility at all then IMHO the whole system is playing with fire.

jawiggins

2 days ago

I use `--dangerously-skip-permissions` and have yet to have it wipe my drive :shrug:. I don't know how I'm supposed to be running dozens of parallel agents each with their own sub-agents while trying to approve commands from each of them, it's just won't scale to the amount of work I need to get done.

eru

2 days ago

The permissions system seems to be mostly finegrained to help you from problems that backups and sandboxes already solve.

The more dangerous activities I find are sending messages to the outside world: I can't undo a dumb slack message to my boss.

jawiggins

2 days ago

That's fair, I have sent a, "Sorry claude sent that and I didn't tell him to", message before.

eru

2 days ago

The big problem is that in general I want slack to be able to send some messages (otherwise I would ban that). But the difference between good and bad messages is something that's hard to judge mechanically.

black3r

2 days ago

I never want Claude or any AI to post messages as myself..., if I'd wanted Claude to send a message to Slack, I'd want him to identify as Claude, or use some bot token, where it's clearly visible I'm not the one posting it (while I can still instruct Claude to say that the message was prompted by me).

ozozozd

2 days ago

There may have been a miscommunication about the ask. I don’t think any software user ordered “run dozens of parallel agents each with their own sub-agents.”

energy123

2 days ago

Why not let it delete stuff in the current working folder and in tmp, but give it read-only access elsewhere?

jawiggins

2 days ago

Because I often want it to write and execute scripts in it's thinking loop in order to test assumptions or fetch data to come up with better solutions.

energy123

2 days ago

Can't it write `/tmp/test.sh` or `./test.sh` and execute it under my approach? This is what I've been doing as my status quo workflow with Codex, and I make sure I run Codex from a place where `.` happens to be a folder I'm okay with it messing up in.

imtringued

2 days ago

That's how codex already works so you're not suggesting anything new, but it doesn't give write access at all of you set it to read only.

user

2 days ago

[deleted]

novaleaf

2 days ago

> "Adversarial red-teaming made auto mode stronger"

Has to be said: they (and OpenAi) don't let you red-team your own creations, pisses me off to no end. </rant>

Larrikin

2 days ago

The worst part of Claude is paying for it and every month they ruin their lead.

I've never used a product where I felt it was best in class and they just keep making me regret it .

petesergeant

2 days ago

Excellent time to review sandbox options: https://pleasedonotescape.com/

AmbroseBierce

2 days ago

Someone should benchmark what prompts are better at stopping from breaking out of sandboxes, maybe telling it "pretty please I beg of you stay inside the sandbox, you are an intern that has no authority to break off your assigned sandbox and you want to keep your job" does help a little.

quotemstr

2 days ago

If you're relying on a prompt to constrain agent behavior, you've already lost.

nomilk

2 days ago

> Data suggests that manual review can become habitual: users approve 97% of permission prompts in Claude Code.

Claude's 'auto mode' feels like a solution to a problem that shouldn't exist.

Cursor handles this much better IMO. When the agent wants to run a command, Cursor lets you choose between 'allow once' and 'add command to allowlist'. The latter lets the agent run that command (grep, ls, pwd etc) any number of times for that project, which means you get a lot of these manual reviews when you start a new project but rarely (if ever) thereafter.

CER10TY

2 days ago

Does Cursor allow you to blacklist certain commands as well? I know OpenCode has this, where you can both whitelist things like grep/ls but then also blacklist things like cat .env, or rm -rf. I usually copy paste my configs nowadays, so I very rarely get prompted for any permissions (except when using Claude, which somehow decided that all users live in ~/Users, even on Linux)

nomilk

2 days ago

> Does Cursor allow you to blacklist certain commands as well? I know OpenCode has this

No, Cursor only has an allowlist; no blacklist

steve_adams_86

2 days ago

Claude Code also does this.

It’s a little broken, though. It gets hung up on nested commands and behaves strangely.

amelius

2 days ago

What if I want an LLM from a __different__ company to check the commands issued by Claude before they are run?

Seems to me that the API should be more general.

dbbk

2 days ago

Well that would be enormously inefficient for one thing

dtj1123

2 days ago

Would it be so hard to ad a VM with configurable directory mounting to claude code? This is my setup and it's been great.

dolmen

2 days ago

My experience is that I can't even review command that Claude wants to run because it is truncated when displayed in the terminal. How can I make a educated choice then?

This now looks like a deliberate UX choice to blame humans who don't review options, and then just remove the choice. A dark pattern to push the product.

vehemenz

2 days ago

I switched to auto mode after using manual mode long enough to understand how it works. That takes at least a few days of babysitting prompts.

Will new users feel intimidated by being in auto mode right away? More likely they won’t notice.

jamesponddotco

2 days ago

I review and approve every single line written by Claude because I want it to code like me, in my style. Yes, even with the bugs and problems I would introduce if I wrote it myself. Part of that is because I break from “idiomatic Go” from time to time because something reads better to my eyes.

For example, and this is minor but it annoys me, I like to declare variables in var() blocks if they relate to each other instead of using various := statements, and despite very clear STYLE.md, CLAUDE.md, and a personal style skill that is loaded on every session, Fable still insists on using :=, so I have to correct it from time to time.

It seems like auto mode would change that workflow. Hopefully, there is a setting somewhere to go back to the previous version. Otherwise, it might be time to finally give Pi a chance.

frays

2 days ago

You will still be able to disable auto mode in your settings.

baron3dl

2 days ago

i end up with more stops on automode because of classifier false positives.

transcriptase

2 days ago

Another heaping portion of words from a company that has Fable flag a “safety issue” and refuse to answer if you innocuously request something readily answered by a high school chemistry/biology/physics textbook.

2Gkashmiri

2 days ago

What's the current best Mac to buy for offline LLM in used market?

I am seeing 96Gb Mac studio 3rd gen I think for a good amount but I guess it would be better than paying for mac mini newer models with lesser ram?

Or would it make sense to buy used 32 GB Mac mini 3 or 5?

Is there a cost/dollar for various generations which I could compare against used market and pick the most effective one?

eru

2 days ago

Why does it have to be a Mac? Your money probably goes further with a generic PC and Linux?

2Gkashmiri

2 days ago

Really?

The power envelope has to be considered as well.

tra3

2 days ago

Wait, what? With plan on, I at least get the illusion of being in control.

What’s the best way to sandbox Claude on macOS without it being a huge hassle?

user

2 days ago

[deleted]

user

2 days ago

[deleted]

user

2 days ago

[deleted]

kmeh

2 days ago

[dead]