Two Reminders about AI

1 pointsposted 6 hours ago
by slackfan

Item id: 49700513

8 Comments

k310

6 hours ago

Three?

AFAICT, LLM's do no checking of data fed to them, accepting lies.

Correct me if this is wrong, but IMO, it's a fatal flaw in any software.

I'm just trying to be objective.

verdverm

5 hours ago

What is a "lie" vs what is the "truth"? We don't have consistent answers as living beings, why do we expect probability to distinguish, even if as a rhetorical argument?

These are still token probability machines, and though the models in the last 6 months do more "thinking", they use certain tokens (actually, but wait) to "intentionally" inject path bifurcation so the next most likely token is conversation that questions preceding statements, like statistical BFS/DFS over the token / embedding graph, but you want to make certain path choices closer in equality so both are explored.

adastra22

6 hours ago

This is simply not true. People have been running autonomous systems for at least a year now.

slackfan

6 hours ago

Show me any system without an initial prompt or goal set.

adastra22

5 hours ago

OpenClaw? Ok literally there's an initial prompt during bootstrapping, but the agent rewrites its starting prompts, in a way that is nondeterministic even with same inputs. It doesn't take very long for the random walk to have it quite distant from the initial starting prompt.

gus_massa

6 hours ago

Generate a list of 100 English words at random, and then use that as the instructions. (Not that I think it's a good idea...)

slackfan

5 hours ago

Seems a stretch to go from that to hacking rubygems don't you think?

I don't believe anthropic or openai are doing anything similar to get their "hack a website" results. The fact that they are prompting dangerous prompts in non-airgapped environments is evidence of at minimum negligence, if not malicious intent on the behalf of those companies.

ben_w

5 hours ago

Reminder: it's not the offline un-prompted weights sitting alone on a hard drive that anyone is worried about.

It's what they do when prompted (both when it is and when it isn't what the user asked for), that we get the most out of them by connecting them to the internet, and that some people are giving them control over robots, that worries people.