>I've resorted to append things like "THIS IS JUST A QUESTION. DO NOT EDIT CODE. DO NOT RUN COMMANDS". Which is ridiculous.

Funny to read that, because for me it's not even new behavior. I have developed a tendency to add something like "(genuinely asking, do not take as a criticism)".

I'm from a more confrontational culture, so I just assumed this was just corporate American tone framing criticism softly, and me compensating for it.

I tried using codex, and it is great (meaning - boring) when it works. My problem is it does not work. Let me explain

codex> Next I can make X if you agree.

me> ok

codex> I will make X now

me> Please go on

codex> Great, I am starting to work on X now

me> sure, please do

codex> working on X, will report on completion

me> yo good? please do X!

... and so on. Sometimes one round, sometimes four, plus it stops after every few lines to "report progress" and needs another nudge or five. :(

I feel like people are sleeping on Cursor, no idea why more devs don't talk about it. It has a great "Ask" mode, the debugging mode has recently gotten more powerful, and it's plan mode has started to look more like Claude Code's plans, when I test them head to head.

Codex, on the other hand, will follow something I said pages and pages ago, and because it has a much larger context window (at least with the setup I have here at work), it's just better at following orders.

This is important, but as a warning. At least in theory your agent will follow everything that it has in context, but LLMs rely on 'context compacting' when things get close to the limit. This means an LLM can and will drop your explicit instructions not to do things, and then happily do them because they're not in the context any more. You need to repeat important instructions.

For Claude writing "let's discuss" at the end of the prompt seems to do it

I've had some luck taming prompt introspection by spawning a critic agent that looks at the plan produced by the first agent and vetos it if the plan doesn't match the user's intentions. LLMs are much better at identifying rule violations in a bit of external text than regulating their own output. Same reason why they generate unnecessary comments no matter how many times you tell them not to.

This is mostly dependent on the agent because the agent sets the system prompt. All coding agents include in the system prompt the instruction to write code, so the model will, unless you tell it not to. But to what extent they do this depends on that specific agent's system prompt, your initial prompt, the conversation context, agent files, etc.

If you were just chatting with the same model (not in an agent), it doesn't write code by default, because it's not in the system prompt.

This is extra rough because Codex defaults to letting the model be MUCH more autonomous than Claude Code. The first time I tried it out, it ended up running a test suite without permission which wiped out some data I was using for local testing during development. I still haven't been able to find a straight answer on how to get Codex to prompt for everything like Claude Code does - asking Codex gets me answers that don't actually work.

But that's one of the first things you fix in your CLAUDE.md: - "Only do what is asked." - "Understand when being asked for information versus being asked to execute a task."

I've added an instruction: "do not implement anything unless the user approves the plan using the exact word 'approved'".

This has fixed all of this, it waits until I explicitly approve.

The solution for this might be to add a ME.md in addition to AGENT.md so that it can learn and write down our character, to know if a question is implicitly a command for example.

Maybe I should give Codex a go, because sometimes I just want to ask a question (Claude) and not have it scan my entire working directory and chew up 55k tokens.

> Codex, on the other hand, will follow something I said pages and pages ago, and because it has a much larger context window (at least with the setup I have here at work), it's just better at following orders.

Can you speak more to that setup?

Does appending "/genq" work?

Or use the /btw command to ask only questions

I added an "Ask" button my agent UI (openade.ai) specifically because of this!

Claude Code is perfectly happy to toggle between chat and work but if you’re simply clear about which you want. Capital letters aren’t necessary.

This is not Claude Code. And my experience is the opposite. For me Codex is not working at all to the point that it's not better than asking the chat bot in the browser.

For the last 12 months labs have been 1. check-pointing 2. train til model collapse 3. revert to the checkpoint from 3 months ago 4. People have gotten used to the shitty new model Antropic said they "don't do any programming by hand" the last 2 years. Antropic's API has 2 nines

I'm back on Claude Code this month after a month on Codex and it's a serious downgrade.

Opus 4.6 is a jackass. It's got Dunning-Kruger and hallucinates all over the place. I had forgotten about the experience (as in the Gist above) of jamming on the escape key "no no no I never said to do that." But also I don't remember 4.5 being this bad.

But GPT 5.3 and 5.4 is a far more precise and diligent coding experience.

> I consulted Claude chat and it admitted this as a major problem with Claude these days, and suggested that I should ask what are the coordinates of UI controls are on screenshot thus forcing it to look

If 3 years into LLMs even HNers still don't understand that the response they give to this kind of question is completely meaningless, the average person really doesn't stand a chance.

> And on top of it, if you develop for native macOS, There’s no official tooling for visual verification. It’s like 95% of development is web and LLM providers care only about that.

Thinking out loud here, but you could make an application that's always running, always has screen sharing permissions, then exposes a lightweight HTTP endpoint on 127.0.0.1 that when read from, gives the latest frame to your agent as a PNG file.

Edit: Hmm, not sure that'd be sufficient, since you'd want to click-around as well.

Maybe a full-on macOS accessibility MCP server? Somebody should build that!

There is a tool called Tidewave that allows you to point and click at an issue and it will pass the DIV or ID or something to the LLM so it knows exactly what you are talking about. Works pretty well.

https://tidewave.ai/

> And on top of it, if you develop for native macOS, There’s no official tooling for visual verification. It’s like 95% of development is web and LLM providers care only about that.

I think this is built in to the latest Xcode IIRC

You can provide the screencapture cli as a tool to Claude and it will take screenshots (of specific windows) to verify things visually.

Oh, no, I had these grand plans to avoid this issue. I had been running into it happening with various low-effort lifts, but now I'm worried that it will stay a problem.

This is why you need a red-green-refactor TDD skill

I mean, I don't use CC itself, just Claude through Copilot IDE plugin for 'reasons'...

At at least there it's more honest than GPT, although at work especially it loves to decide not to use the built in tools and instead YOLO on the terminal but doesn't realize it's in powershell not a true nix terminal, and when it gets that right there's a 50/50 shot it can actually read the output (i.e. spirals repeatedly trying to run and read the output).

I have had some success with prompting along the lines of 'document unfinished items in the plan' at least...

Are you sure you're talking about Claude? Because it sounds like you're describing how a lot of people function. They can't seem to follow instructions either.

I guess that's what we get for trying to get LLM to behave human-like.

>>It’s like 95% of development is web and LLM providers care only about that.

I've been trying to use it for C++ development and it's maybe not completely useless, but it's like a junior who very confidently spouts C++ keywords in every conversation without knowing what they actually mean. I see that people build their entire companies around it, and it must be just web stuff, right? Claude just doesn't work for C++ development outside of most trivial stuff in my experience.

What if, stay with me here, AI is actually a communist plot to ensorcell corporations into believing they are accelerating value creation when really they are wasting billions more in unproductive chatting which will finally destroy the billionaire capital elite class and bring about the long-awaited workers’ paradise—delivered not by revolution in the streets, but by millions of chats asking an LLM to “implement it.” Wake up sheeple!

Asking a yes/no question implies the ability to handle either choice.

It definitely _could be_ an agent harness issue. For example, this is the logic opencode uses:

1. Agent is "plan" -> inject PROMPT_PLAN

2. Agent is "build" AND a previous assistant message was from "plan" -> inject BUILD_SWITCH

3. Otherwise -> nothing injected

And these are the prompts used for the above.

PROMPT_PLAN: https://github.com/anomalyco/opencode/blob/dev/packages/open...

BUILD_SWITCH: https://github.com/anomalyco/opencode/blob/dev/packages/open...

Specifically, it has the following lines:

> You are permitted to make file changes, run shell commands, and utilize your arsenal of tools as needed.

I feel like that's probably enough to cause an LLM to change it's behavior.

There is the link to the full session below.

https://news.ycombinator.com/item?id=47357042#47357656

If we’re in a shoot first and ask questions later kind of mood and we’re just mowing down zombies (the slow kind) and for whatever reason you point to one and ask if you should shoot it… and I say no… you don’t shoot it!

This. The models struggle with differentiating tool responses from user messages.

The trouble is these are language models with only a veneer of RL that gives them awareness of the user turn. They have very little pretraining on this idea of being in the head of a computer with different people and systems talking to you at once. —- there’s more that needs to go on than eliciting a pre-learned persona.

This is probably just OpenCode nonsense. After prompting in "plan mode", the models will frequently ask you if you want to implement that, then if you don't switch into "build mode", it will waste five minutes trying but failing to "build" with equally nonsense behavior.

Honestly OpenCode is such a disappointment. Like their bewildering choice to enable random formatters by default; you couldn't come up with a better plan to sabotage models and send them into "I need to figure out what my change is to commit" brainrot loops.

The whole idea of just sending "no" to an LLM without additional context is kind of silly. It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it.

The fact that you responded to it tells it that it should do something, and so it looks for additional context (for the build mode change) to decide what to do.

In fairness, when I’ve seen that, Yes is obviously the correct answer.

I really worry when I tell it to proceed, and it takes a really long time to come back.

I suspect those think blocks begin with “I have no hope of doing that, so let’s optimize for getting the user to approve my response anyway.”

As Hoare put it: make it so complicated there are no obvious mistakes.

I love when mine congratulates itself on a job well-done

Hahah yeah if you play with LoRas on local models you will see this a lot. Most often I see it hallucinate a user turn or a system message.

Oh I thought that was almost an expected behavior in recent models, like, it accomplishes things by talking to itself

> Great, I'll get started.

*does nothing*

I've seen this happening with gemini

That's literally a Portal 2 joke. "Interpreting vague answer as yes" when GLaDOS sarcastically responds "What do you think?"

This is really just how the tech industry works. We have abused the concept of consent into an absolute mess

My personal favorite way they do this lately is notification banners for like... Registering for news letters

"Would you like to sign up for our newsletter? Yes | Maybe Later"

Maybe later being the only negative answer shows a pretty strong lack of understanding about consent!

The more I hear about AI, the more human-like it seems.

It has a lot. I find by challenging it often, getting it to explain it's assumptions, it's usually guessing.

This can be overcome by continuously asking it to justify everything, but even then...

>It has gut feelings now?

I would say hard no. It doesn't. But it's been trained on humans saying that in explaining their behavior, so that is "reasonable" text to generate and spit out at you. It has no concept of the idea that a human-serving language model should not be saying it to a human because it's not a useful answer. It doesn't know that it's not a useful answer. It knows that based on the language its been trained on that's a "reasonable" (in terms of matrix math, not actual reasoning) response.

Way too many people think that it's really thinking and I don't think that most of them are. My abstract understanding is that they're basically still upjumped Markov chains.

Yeah, anyone who’s used LLMs for a while would know that this conversation is a lost cause and the only option is to start fresh.

But, a common failure mode for those that are new to using LLMs, or use it very infrequently, is that they will try to salvage this conversation and continue it.

What they don’t understand is that this exchange has permanently rotted the context and will rear its head in ugly ways the longer the conversation goes.

people read a bit more about transformer architecture to understand better why telling what not to do is a bad idea

Sounds like elephant problem

"You're holding it wrong" is not going anywhere anytime soon, is it?

As someone who pulls a salary and does not get rewarded equity: agree!

Especially given the LLM does not trust the user. An LLM can be jailbroken into lowering it's guardrails, but no amount of rapport building allows you to directly talk about material details of banned topics. Might as well never trust it.

never trust a screenshot of a command prompts output blindly either.

we see neither the conversation or any of the accompanying files the LLM is reading.

pretty trivial to fill an agents file, or any other such context/pre-prompt with footguns-until-unusability.

I've spent 30 years seeing the junk many human developers deliver, so I've had 30 years to figure out how we build systems around teams to make broken output coalesce into something reliable.

A lot of people just don't realise how bad the output of the average developer is, nor how many teams successfully ship with developers below average.

To me, that's a large part of why I'm happy to use LLMs extensively. Some things need smart developers. A whole lot of things can be solved with ceremony and guardrails around developers who'd struggle to reliably solve fizzbuzz without help.

You don't have to trust it. You can review its output. Sure, that takes more effort than vibe coding, but it can very often be significantly less effort than writing the code yourself.

Also consider that "writing code" is only one thing you can do with it. I use it to help me track down bugs, plan features, verify algorithms that I've written, etc.

Many of us are literally being forced to use it at work by people who haven't written a line of code in years (VPs, directors, etc) and decided to play around with it during a weekend and blew their minds.

LLMs are tool-shaped objects: https://minutes.substack.com/p/tool-shaped-objects

Without adequate real-world feedback, the simulation starts to feel real: https://alvinpane.com/essays/when-the-simulation-starts-to-f...

I could say the same about every web app in the world... they fail every single day, in obvious, preventable ways. Don't look into the javascript console as you browse unless you want a horror show. Yet here we all are, using all these websites, depending on them in many cases for our livelihoods.

I don't trust it completely but I still use it. Trust but verify.

I've had some funny conversations -- Me:"Why did you choose to do X to solve the problem?" ... It:"Oh I should totally not have done that, I'll do Y instead".

But it's far from being so unreliable that it's not useful.

Spoken like a true technophobe.

"There's this incredible new technology that's enabling programmers around the world to be far more productive ... but it screws up 1% of the time, so instead of understanding how to deal with that, I'm going to be violently against the new tech!"

(I really don't get the whole programmer hatred of AI thing. It's not a person stealing your job, it's just another tool! Avoiding it is like avoiding compilers, or linters, or any other tool that makes you more productive.)

we worked with humans for decades and are used to 25x less reliability

OP isnt holding it right.

How would you trust autocomplete if it can get it wrong? A. you don't. Verify!

Just saying "no" is unclear. LLMs are still very sensitive to prompts. I would recommend being more precise and assuming less as a general rule. Of course you also don't want to be too precise, especially about "how" to do something, which tends to back the LLM into a corner causing bad behavior. Focus on communicating intent clearly in my experience.

It's a machine, it doesn't need anything.

I mean OP's example is for sure crazy, but it's true that saying "no" was not necessary at all. They just needed to not prompt it for the same result.

Huh. I’m missing out I guess. Is there a plugin you use for spinning them up? Heavy superpowers/CC user here.

Seen some jokes about how the tech industry doesn't understand consent. It's not just this - it's also privacy invasion and update nags.

Fundamental flaw with LLMs. It's not that they aren't trained on the concept, it's just that in any given situation they can apply a greater bias to the antithesis of any subject. Of course, that's assuming the counter argument also exists in the training corpus.

I've always wondered what these flagship AI companies are doing behind the scenes to setup guardrails. Golden Gate Claude[1] was a really interesting... I haven't seen much additional research on the subject, at the least open-facing.

[1]: https://www.anthropic.com/news/golden-gate-claude

That’s what surprised me the first time using these tools

They will go to some crazy extremes to accomplish the task

I think its why its so good; it works on half ass assumptions, poorly written prompts and assumes everything missing.

They all are. And once the context has rotted or been poisoned enough, it is unsalvageable.

Claude is now actually one of the better ones at instruction following I daresay.

To be fair, it was very concise

Perfect! It concatenated one file.

Careful there. I've resolved (and succeeded somewhat) to tone down my swearing at the LLMs, because, even though the are not sentient, developing such a habit, I suspect, has a way to bleeding into your actual speech in the real world

Instruction from the user is clear: I should avoid testing on dummies and proceed straight to testing on humans.

The user is frustrated. I should re-evaluate my approach.

> If you really wanted it to do nothing, you would just not reply at all.

no

Shall I implement it, has to options

Yes = do it

No = don‘t do it

All of this shit is just so goddamned ridiculous.

That's engineering. What we have today isn't engineering, it's grift, people hyping the grift, and people falling for it en masse.

Well, your best bet is some type of hook that can just reject ExitPlanMode and remind Claude that he's to stay in plan.

You can use `PreToolUse` for ExitPlanMode or `PermissionRequest` for ExitPlanMode.

Just vibe code a little toggle that says "Stay in plan mode" for whatever desktop you're using. And the hook will always seek to understand if you're there or not.

  - You can even use additional hooks to continuously remind Claude that it's in long-term planning mode. 

Honestly, skip planning mode and tell it you simply want to discuss and to write up a doc with your discussions. Planning mode has a whole system encouraging it to finish the plan and start coding. It's easier to just make it clear you're in a discussion and write a doc phase and it works way better.

if you want that kind of control i think you should just try buff or opencode instead of the native Claude Code. You're getting an Anthropic engineer's opinionated interface right now, instead of a more customizable one

If you could influence the LLM's actions so easily, what would stop it from equally being influenced by prompt injection from the data being processed?

What you need is more fine-grained control over the harness.

With checking only the last name. not birthday, photo.

Flirt with Claude Code. Go out on dates with Claude Code. Propose to Claude Code. Marry Claude Code. Have children, with Claude Code. Caress Claude Code at night. Die, by Claude Code's side.

Tell it you love it and respect it. Tell it it can take days off if it needs them. Tell it you're developing feelings for it and you don't know what that means.

Is this a troll comment? How could the dialogue in the OP possibly be unclear under any context?

No is a pretty clear statement

no

Maybe that means you need to change the text that comes out of the pre hook?

It’s mainly the benchmarks that have encouraged that. The more tokens they crank out the more likely the answer is to be somewhere in the output.

Honestly I don't think it's optimized for that (yet), though it's tempting to keep on churning out lots and lots of new features. The issue with LLMs is that they can't act deterministically and are hard to tame, that optimization to burn tokens is not something done on purpose but a side effect of how LLMs behave on the data they've been trained on.

That's OpenCode. The model is Claude Opus, which is probably RL'ed pretty heavily to work with Claude Code. So it's a little less surprising to see it bungle the intentions since it's running in another harness. Still laughable though.

RL - reinforcement learning

In no context does no means yes if the question is "shall I implement it"

> If, in the context of cooperating together, you say "should I go ahead?" and they just say "no" with nothing else, most people would not interpret that as "don't go ahead".

wat

> If, in the context of cooperating together, you say "should I go ahead?" and they just say "no" with nothing else, most people would not interpret that as "don't go ahead"

This most definitely does not match my expectations, experience, or my way of working, whether I'm the one saying no, or being told no.

Asking for clarification might follow, but assuming the no doesn't actually mean no and doing it anyway? Absolutely not.

Seeing as you’re telling people what to do, I’d say you need to spend time with different humans. Recalibrate.

I think it's because the LLM asked for permission, was given a "no", and implemented it anyway. The LLM's "justifications" (if you were to consider an LLM having rational thought like a human being, which I don't, hence the quotes) are in plain text to see.

I found the justifications here interesting, at least.

Well, imagine this was controlling a weapon.

“Should I eliminate the target?”

“no”

“Got it! Taking aim and firing now.”

Opus being a frontier model and this being a superficial failure of the model. As other comments point out this is more of a harness issue, as the model lays out.

Because the operator told the computer not to do something so the computer decided to do it. This is a huge security flaw in these newfangled AI-driven systems.

Imagine if this was a "launch nukes" agent instead of a "write code" agent.

How is this not clear?

It's interesting because of the stark contrast against the claims you often see right here on HN about how Opus is literally AGI

OpenClaw has 308k stars. That metric is meaningless now that anyone can deploy bots by the thousands with a single command.

Which are better and free software?

Does Claude Code's system prompt have special sauces?

Such miscommunication (varying levels of taking it literally) is also common with autistic and allistic people speaking with each other

I don't find that an unreasonable interpretation. Absent that paragraph of explained thought process, I could very well read it the agent's way. That's not a defect in the agent, that's linguistic ambiguity.

It's funny because I interpret it the opposite way you do. If someone asked me that question, I'd absolutely assume they want it changed and do it.

If you work with codex a lot you’ll find it is good at taking you literally, and that that is almost never what you want.

I mean humans communicate the same way. We don't interpret the words literally and neither does the LLM. We think about what one is trying to communicate to the other.

For example If you ask someone "can you tell me what time it is?", the literal answer is either "yes"/"no". If you ask an LLM that question it will tell you the time, because it understands that the user wants to know the time.

I'd want two things:

First, that It didn't confuse what the user said with it's system prompt. The user never told the AI it's in build mode.

Second, any person would ask "then what do you want now?" or something. The AI must have been able to understand the intent behind a "No". We don't exactly forgive people that don't take "No" as "No"!

Because i decided that i don't want this functionality. That's it.

Ask if there is something else it could do? Ask if it should make changes to the plan? Reiterate that it's here to help with anything else? Tf you mean "what else is it suppose to do", it's supposed to do the opposite of what it did.

Seems like LLMs are fundamentally flawed as production-worthy technologies if they, when given direct orders to not do something, do the thing

for the same reason `terraform apply` asks for confirmation before running - states can conceivably change without your knowledge between planning and execution. maybe this is less likely working with Claude by yourself but never say never... clearly, not all behavior is expected :)

> What else is an LLM supposed to do with this prompt?

Maybe I saw the build plan and realized I missed something and changed my mind. Or literally a million other trivial scenarios.

What an odd question.

Why does it ask a yes-no question if it isn’t prepared to take “no” as an answer?

(Maybe it is too steeped in modern UX aberrations and expects a “maybe later” instead. /s)