Reflections on Trusting Trust, Revisited: Poisoning Self-Modifying AI Coding

10 pointsposted 9 hours ago
by sbulaev

1 Comments

cyanydeez

3 hours ago

perhaps take the adversary out and just ask: how does the system evolve without duplicating errors side effects of rate but "horrific" consequences.

What you're talking about is evolution and we've seen lots of problematic outcomes with just genetic diversity alone.

You dont need an adversary, and it's kind of suggesting that these systems even can modify themselves in a coherent manner which I dont see proven.