I sometimes feel a peculiar resonance with these models: they catch the faintest hints of irony and return astoundingly witty remarks, almost as if they were another version of myself. Yet all of the problems, inconsistencies, and surprises that arise in human thought stem from something profoundly differen, which is our embodied experience of the world. Humans integrate sensory feedback, form goals, navigate uncertainty, and make countless micro-decisions in real time, all while reasoning causally and contextually. Cognition is active, multimodal, and adaptive; it is not merely a reflection of prior experience a continual construction of understanding.

And then there are some brilliant friends of mine, people with whom a conversation can unfold for days, rewarding me with the same rapid, incisive exchange we now associate with language models. There is, clearly, an intellectual and environmental element to it.

Whenever we're testing LLM's against people we need to ask "which people?" Testing a chess bot against random undergrads versus chess grandmasters tells us different things.

From an economics perspective, maybe a relevant comparison is to people who do that task professionally.

This has been my experience as well. I see very bright people lampooning LLMs because it doesn't perform up to their expectations when they are easily in the top 1% of talent in their field. I don't think they understand the cognitive load in your average F500 role is NOT very high. Most people are doing jack shit.

Few things enrage me like the smell of cannabis on the highway after it was legalized in my state. Sure, hypothetically, that's the passenger. But more likely than not, it's DUI.

> However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things

do you have examples of this?

asking because this is not what happens to me. one of the main things i worry about when interacting with the llm is that they agree with me too easily.

This is why i simply do not bother with them unless the task i need is so specific that theres no room for argument, like yesterday i asked it to generate me a bash script that ran aws ssm commands for all the following instance IDs. It did that as a two shot.

But long conversations are never worth it.

>>> … However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things …

This is something I have not experienced. Can you provide examples ?

Yeah this is exactly opposite my issue with LLMs. They often take what you say as the truth when it absolutely could not be.

I have taken the stance to not argue with LLMs, don't give them any clues, and don't ask them to roleplay. Tell them no more than what they need to know.

And if they get the answer wrong, don't try to correct them or guide them, there is a high chance they don't have the answer and what follow will be hallucinations. You can ask for details, but don't try to go against it, it will just assume you are right (even if you are not) and hallucinate around that. Keep what you already know to yourself.

As for the "you are an expert" prompts, it will mostly just make the LLM speak more authoritatively, but it doesn't mean it will be more correct. My strategy is now to give the LLM as much freedom as it can get, it may not be the best way to extract all the knowledge it has, but it helps spot hallucinations.

You can argue with actual people, if both of you are open enough, something greater make come out of it, but if not, it is useless, and with LLMs it is always useless, they are pretrained, they won't get better in the future because that little conversation sparked their interest. And on your side, you will just have your own points rephrased and sent back to you, and that will just put you deeper in your own bubble.

> However, for some reason, both Gemini and ChatGPT tend to argue with me

The trick here is: "Be succinct. No commentary."

And sometimes a healthy dose of expressing frustration or anger (cursing, berating, threatening) also gets them to STFU and do the thing. As in literally: "I don't give a fuck about your stupid fucking opinions on the matter. Do it exactly as I specified"

Also generally the very first time it expresses any of that weird shit, your context is toast. So even correcting it is reinforcing. Just regenerate the response.

> ChatGPT tend to argue with me so heavily

I have found that quite often when ChatGPT digs in on something, that it is in fact right, and I was the one that was wrong. Not always, maybe not even most of the time, but enough that it does give me pause and make me double check.

Also, when you have an LLM that is too agreeable, that is how it gets into a folie a deux situation and starts participating in user's delusions, with disastrous outcomes.

Sadly that’s a personality trait that’s far too common in the field, and it can get pretty annoying.

Jumping from "the author uses language I dislike" straight to "also, he has no theory of mind" is a bit of a leap. Like world record winning long jump kinda stuff.

Also, what big words? 'Proliferation'? 'Incoherent'? The whole article is written at a high school reading level. There's some embedded clauses in longer sentences, but we're not exactly slogging our way through Proust, here.

Once it’s in the context window the model invariably steers crazy. Llms cannot handle the “don’t think of an elephant” requirement.

It depends.

It's a natural inclination for all LLMs, rooted in pre-training. But you can train them out of it some. Or not.

Google doesn't know how to do it to save their lives. Other frontier labs are better at it, but none are perfect as of yet.

Yea I find it a bit condescending. Humans ain't robots, duh!

And the world wouldn't function if everyone operated at the exact same abstraction level of ideas.

> when humans have no idea what they're talking about, it's usually more obvious

Is it?

That's not my experience.

Yeah, shared context over time is the answer to all these problems and has been for both history and prehistory. Patience appears to be the scarcest resource of all these days.

It's sadly also an attitude I'm not surprised to see coming out of tech, given how many people don't seem to get that "I got into this field so I could interact with computers, not people" is supposed to be a joke.

This is exactly why I stopped reading after that section.

Concur. Alas, that ship has sailed.

Please define "confabulation" (for us stupid non-AI, non-native speakers)

There is no the Turing test. I think ELIZA was the first program to pass a Turing test, around 60 years ago.

You're absolutely right.

I have an idea for a reverse turing test where humans have to convince an LLM that they are an LLM. I suspect that most people would fail, proving that humans lack intelligence.

... or circumstances now select for "high agency" humans, and those humans propagate.

Not sure why we think normal evolution wouldn't just route around such problems.

> reward hacking, where social approval or harmony is optimized at the expense of truth or usefulness

> Is he saying humans have become this way because of the influence of LLMs?

No. The first paragraph explains it quite clearly, IMO: "While some are still discussing why computers will never be able to pass the Turing test, I find myself repeatedly facing the idea that as the models improve and humans don’t, the bar for the test gets raised and eventually humans won’t pass the test themselves."

The point is not that the problems exist more in humans now vs before. It's that they can be observed more significantly in humans than in LLMs (and moreso over time) if one cares to look because LLMs improve and humans do not on sub-evolutionary timescales. And perhaps our patience with them in humans is now diminished because of our experiences with them in LLMs and so people may notice them in humans more than before.

Humans have always been this way, what we lack now is the patience to put up with it.

Sadly, we have n=1 for intelligence and that's humans. The "second best" of intelligence is already LLMs. And it's hard to expect imitation learning on data that wasn't produced by anything intelligent to yield intelligence - although there are some curious finds.

Even for human behavior: we don't have that much data. The current datasets don't capture all of human behavior - only the facets of it that can be glimpsed from text, or from video. And video is notoriously hard to use well in LLM training pipelines.

That LLMs can learn so much from so little is quite impressive in itself. Text being this powerful was, at its time, an extremely counterintuitive finding.

Although some of the power of modern LLMs already comes from nonhuman sources. RLVR and RLAIF are major parts of training recipes for frontier labs.

The datasets going into LLMs have to have an element of human-ness to it.

For example I can’t just feed it weather data from the past decade and expect it to understand weather. It needs input and output pairs with the output being human language. So you can feed it weather data but it has to be paired with human description of said data. So if we give it data of a rain storm there has to be an english description paired with it saying it’s a rainstorm.

Why would this be satire? I'm guessing you don't pay that much attention to humans, or you have a very curated social group you're around.

I have definitely, absolutely, positively had conversations where details have fallen out of the context window of my conversation partner(or mine, for that matter), without the person in question realizing this has happened, and have only via LLMs found a vocabulary to give a name to the phenomenon.

Arguments like this make me suspect that the proponents have simply a malformed theory of mind. If I'm being really catty, I'll say it's because they have below average levels of self-awareness.

The good old "kilogram of steel" test is something that the children, the mentally infirm and the less capable LLMs all fail in the same way.

Some failures like that are simply human failures reproduced faithfully. Some are rooted deeper than that.

And yes, it's true that children don't get bored in the same way adults do, which often leads to repetitive behavior. Boredom is an important heuristic for behavior, it seems.

I think the author might not have been speaking 100% with his tongue not in his cheek there.