proc0
2 hours ago
Whenever we discover how the human mind does it, and then we apply it to AI systems. Currently AI just has a surface level interface that makes it seem like it can think of all these things humans think about, i.e. is this going to work properly, is this a good idea given the objectives, does this achieve the wider goal etc., etc.
It's ultimately hard to articulate and hard to measure. We are only measuring very narrow tasks (even if the tasks are large, there is no test that is measuring how well the AI took initiative or was able to solve unforeseen problems).
People like Yann LeCun understand this and are trying to build more wholistic level intelligence that would achieve this, while the majority of the current AI industry is closing their eyes and betting on scale and hoping that with enough compute these properties will emerge. I think we're going to be stuck with "dumb intelligence" until those other breakthroughs happen.
abalgir
2 hours ago
Don't you think we should build infrastructure around the AI with a watcher (yes, another AI) that makes sure AIs don't deviate or drift? I.e., we can ask the dev AI to submit its test ledger as a contract, the reviewer AI to submit its own as another contract, and the watcher to monitor contract executions for deviations.
Then I see what you might say: Who monitors the watcher? -:(
proc0
2 hours ago
Exactly. We can sure try, but I would bet against it working well. It might work a little, maybe it would be worth it, but the underlying assumption is that AI can have good judgement and so far that is mostly an illusion and it's closer to random. Sometimes it makes good decisions, sometimes not, and that is a huge problem that keeps it from being good agentic systems. They still need the harness and all this other traditional code around it to get it to work.
abalgir
33 minutes ago
Totally agree, they need a strong harness to make them work