Adding verification steps to AI agents made them worse. I tested it 29 times

3 pointsposted 8 hours ago
by chrisdudek
(substack.com)

5 Comments

Bjartr

27 minutes ago

> What works... External mechanical checks

This is what I expected verification to mean going in.

For some reason verification in this article means "try to convince the AI even harder than before"

nareyko

8 hours ago

[dead]

chrisdudek

8 hours ago

I suspected that given some responses the agent was giving during the experimentation. Looked sometimes tendentious when measuring.

nareyko

7 hours ago

[dead]

chrisdudek

6 hours ago

I tried to also avoid that, and the main orchestrating agent was running subagents running experiments, and others were checking the results. Somewhat it worked as it was not imposing only successes, there were more failures.