Yeah, I feel like what these files do is not that difficult to understand; it's just context that the agent will pick up and use pretty much the same as any other context it has. No, it won't deterministically prevent things with a static check, but it will work about as well as just manually telling the agent "don't do X" as part of the prompt. The fact that it's a "mostly works" mechanism rather than a "guaranteed to always work" mechanism is pretty much the same experience that using an LLM gives in general, and while that requires a bit of thought about how to use it, it's still good enough to be useful for a lot of things.

Before LLMs, I found "mostly works" systems like this to be incredibly sketchy and not worth using. The main thing I've had to learn in this past year is that what I thought was an ironclad rule turned out to be only a heuristic that was useful before but not always helpful, because empircally as much as I might find the lack of determinism jarring, in practice these tools are genuinely good enough at what they do to be worthwhile to use, as long as you're making sure not to use them in ways that the occasional failure costs more than just some wasted time.

I generally set up the rules of engagement for a project in the AGENTS.md file and if it starts to get large break out to more specific files referenced from AGENTS.md to keep the constant per-request token overhead lower.

For Python, I have the agent use `uv` (even for direct script invocations) so that the agent doesn't need to burn tokens concerning itself with the details that `uv` is managing behind the scenes.

This is what the majority of mine look like as well. Just simple instructions for things I need to do repeatedly.

I have not even really needed a formal memory system. If I see an error happen more than once, I just say "hey add a note on this to agents.md". Tends to be verbose but overall works quite well for the projects I am doing.

Can you share which model and harness created this situation?

Yeah, something immediately felt wrong about the entire framing of the article.

I work on several projects on the side, and they all have agents files that give the context about what the project is, what the elements of it are, and what things we're typically working on. This allows my initial prompt to reference things that would otherwise not be in the context at all.

I mean, they're not magic, they're just some automatic context that's supplied. I feel like the study is trying to say that context with an LLM doesn't matter, which is obviously not a tractable position to hold.

The fact that they generated all the agents files instead of curating them with a human is probably part of the problem.

Why articles of people who make living by being pro-AI never get comments like this?

The science is the source linked in the second paragraph.

That's what I took away from this as well. Headline is clickbait but at the end of the day there are uses for these instructions.

My agents.md usually becomes do’s don’ts and shortcuts to get to where it needs to be in the code so it doesn’t waste time and context.

I haven't really been doing this yet, but for what it's worth, I think explaining how to use tools doesn't really belong in your agents.md. I think it's better to use it to explain the what tools are available (information about the environment, like you said), and then provide details on the how in skills files. That way you're not bloating your context with how to use `uv` in a session where you're not doing anything Python related. At worst, you're just saying that `uv` exists on your system.

I just have it comment on scripts, I've now just started for an app super-run.sh consisting of this pipeline: dev, test, build, e2e, deploy.

All the scripts are designed to bail quickly. They all create a log in the background instead of blocking. They're all annotated with the necessary comments to keep it on path and locate files, track pratfalls, etc.

So any entrypoint to whatever I'm doing typically starts with one of these scripts. It's heavy handed but orientating your agent for the specific task is better than just dumping a whole set of context that will be ignored if it has nothing to do with your very next command.

Telling it how to do a pull request isn't going to help if you're trying to debug a technical issue.

I have in mine to not do any write operations with git, to only use the CLI to explore the history and the current state. I don't trust my agent to commit and push on my behalf. Using my agents.md has worked well for that kind of thing.

I have no historical context (heh) for this person, but as someone who uses LLMs regularly, I feel like there are reasonable takeaways from the article. Namely:

- Don't vibe you agents.md file, it won't capture any intuition about the project that the model doesn't already have.

- Keep your agents.md file short. Long ones mostly bloat context for minimal difference in behavior.

- Writing for a human audience is probably better. Any LLM can read docs made for humans anyways.

Incredible how much of an audience you can get by just being pro "the latest hype"