> Reasoning about l1/l2/l3 cache hit ratios and their implications basically throwing darts at the wall, in the wrong room. If you give it a benchmark feedback loop then it might get there eventually but still massive alpha for low level systems engineers who instinctively know how this stuff works and can now automate 99% of the grind.

I suspect a lot of the training set for this sort of thing is people online speculating about cache performance incorrectly.

Can you give it the valgrind suite to loop over? I have yet to try that with AI, but maybe the cachegrind tool is enough to help it.

I see the same thing, except I was working on high level performance reasoning. Whenever a piece of code has multiple steps that require multiple algorithms to work, AI almost always fails to guess which step is the slowest and what causes that step to be slow. Even Fable makes wrong guesses. You definitely need to give them a benchmark feedback loop.

Are there actually any humans who can reason about things like cache performance from first principles? I know there are some people who think they can, and I suspect they're fooling themselves. The one iron principle of micro-optimization at the level of cache hits is "measure, measure, measure", you just cannot think your way to the right answer on the first try. Processors and instruction sets today are too complicated, and tips that worked on one generation might be neutral or worse on the very next revision, making all the cargo cult knowledge passed around on this topic at best useless. I'll echo one of the sibling commenters here and say that LLMs probably bullshit their way to answers on questions like this because that's what humans online do as well.

If you give an LLM a proper testing harness and feedback loop to actually generate hypotheses, test them and revise them, I suspect it will do much better.

Agree it's amazing how much low-hanging performance fruit AI can trivially find. On the other hand though, once you get through the obvious no-brainer stuff, there's a lot of non-trivial tradeoffs in performance and I think that still demands a good amount of expertise to guide the AI in the right direction. Obviously AI will continue working it's way up the value chain, but I think there's a glass ceiling for AI where the right macro tradeoffs and perspectives on how software should work will bump into the hard and often articulated reality that different stakeholders want different things and often have either magical thinking or even self-deception about how those desires can co-exist with what everyone else wants.

This isn't a new problem by any means, but now that code is cheap, it means instead of getting frustrated with engineering and their pesky unimportant details, people will get frustrated with the AI and it's pesky unimportant details.

I think they can be useful for quickly iterating through benchmarks and trying lots of ideas, but they won't come up with them on their own. Also, I'm not sure why, maybe some mean reversion thing, but they will never, ever suggest writing a tool to make their own life easier, get more accurate information, or anything. Once I point it at a tool, it can be ok at using it (I say ok because they seem to skim the help docs, which is truly ironic, considering I seem to read it more thoroughly even though I'm 100x slower at it. I assume this is some token saving system prompt), but they won't suggest it for you.

This is why I'm not worried about being replaced for now or the forseeable future. For all of the improvements they've made, this part just never seems to change. They could slap another heuristic prompt for the edge case, but eventually it'll revert to the mean again.

I think there is a way to use LLMs to help with programming, but not when I'm not the driver in the seat writing the tests and deciding the architecture. Also I would never ship code written by them as the final product for anything I care about. Since I, like most people, find reading code to be arduous. The more fun thing to do is to force yourself to rewrite it all, treating the LLM's work as a rough draft.

> If it can be measured, then LLMs can optimize it

Then they can start attempting to optimize it. They can also spin round and round making the numbers worse because they don't actually know what to do.

I ran into this exact issue and had Claude write a test run scheduler to avoid cargo process contention and migrated to nextest within it. I have --release build cycles on my local machine down to about three seconds from what was for the same project 60+ seconds before investing in the changes.

"Rank the top findings/solutions by impact vs confidence" continues to be one of my best quickwins to add to all sorts of prompts. Bam, now you have a reasoned priority list.

Not really. The LLM is the data model. The part doing the language manipulation. An agent is the combination of an LLM with an interface it knows how to use to take actions with, though it may just refer to the interface or connection or tool itself.

The LLM itself is primarily a transformer that handles tokenization, mapping, and neural-network reasoning (in simple terms). It's the engine. It takes in language, interprets it's meaning, and based on other language that supplies instructions, decides how to respond. The agent typically takes the form of an instruction for how to use a tool, combined with how to access it.

Technically, the chat session is itself an agentic tool because it's taking input and returning output through that interface. But it's kind of the default tool. Stdout, if you will.

Say you're in Gmail and you get a message that tells you about a meeting, but doesn't include an invite. If you tell Gemini to add an event for that meeting to your calendar, the LLM interprets your instruction, then uses an agent tool to read your email, interprets it, and then uses another agent tool to create a meeting on your Google Calendar.

From an editing perspective, this went mostly down to a) attributing the change to a model rather than the harness and b) vibes.

In my mind no; an agent is an llm wrapped in more software (most trivially a while loop, but probably more then that)

They were with tera 2.1.0, however using the default settings and not enabling the `fast` feature (which I did not know about, TIL).

The comparison benchmark results in the post were older and prior to further passes. After updating to tera 2.4.0 and enabling the `fast` feature set, I reran the benchmarks against the current codebase and the difference between my Rust crate and Tera is now about ~1.5x.

Some of the technical differences are (which in disclosure were surfaced by GPT 5.6 Sol): context and loop values are borrowed while Tera clones, cached static positions/lookups, less per-render heap setup, escaping chunks directly into the final String instead of bytes, and more granular bytecode functions which avoid stacks.

Maybe LLM's should be integrated with the software, so they could optimize to the real hardware and workload ?

There is an entire paragraph about benchmaxxing and how to mitigate that.