So, the exact stuff Google used to be good at.

What is "it". Gpt-5 auto? Gpt-5 pro? Deep research? These have wildly different hallucination rates.

I gave it a PDF recently and asked it to help me generate some tables based on the information there in. I thought I'd be saving myself time. I spent easily twice as long as I would have if it I had done it myself. It kept making trivial mistakes, misunderstanding what was in the PDF, hallucinating, etc.

It is excellent when just finding something is enough. Most often in my practice, I am dealing with questions that have no written-down answers, meaning the probability of finding a book/article that provides one is negligible. Instead, I am looking for indirect answers or proofs before I make a final engineering decision. Yet another problem is that the language itself changes over time. For instance, at the beginning of the 20th century, the integers were called integral numbers. IMHO, LLMs poorly handle such cases when considered as a substitute for search engines. For full-text vector search, I am using https://www.recoll.org/ a real time saver for me, especially for desktop search.

I think you have a very good point here: a semantic search would be the best option for such a search. The items would have unique identifiers so the language variations can be avoided. But unfortunately, I am not aware of any of these kinds of publicly available projects, except DBpedia and some biology-oriented ontologies that would massively analyze scientific reports.

Currently, I am applying RDF/OWL to describe some factual information and contradictions in the scientific literature. On an amateur level. Thus I do it mostly manually. The GPT-discourse somehow brings up not only the human-related perception problems, such as cognitive biases, but also truly philosophical questions of epistemology that should be resolved beforehand. LLM developers cannot solve this because it is not under their control. They can only choose what to learn from. For instance, when we consider a scientific text, it is not an absolute truth but rather a carefully verified and reviewed opinion that is based on the previous authorized opinions and subject to change in the future. So the same author may have various opinions over time. More recent opinions are not necessarily more "truthful" ones. Now imagine a corresponding RDF triple (subject-predicate-object tuple) that describes that. Pretty heavy thing, and no NLTK can decide for us what the truth is and what is not.

https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

The Gell-Mann Amnesia effect. And you're absolutely right, it's extremely pronounced in LLM users.

Thank you for sharing! I like your dendrogram-like circular graphs! They are way more intuitive. That could be a nice companion for a bibliometrix/biblioshiny library for bibliometric analysis https://www.bibliometrix.org/. I tried "Deep Dive" with my own request, and ... it unfortunately stops at the end of "Organizing results". Maybe I should try again later.

Right, and I would even go a step further and say the context from SebastienBubeck is stretching "solved" past its breaking point by equating literature research with self-bootsrapped problem solving. When it's later characterized as "previously unsolved" it's doubling down on the same equivocation.

Don't get me wrong, effectively surfacing unappreciated research is great and extremely valuable. So there's a real thing here but with the wrong headline attached to it.

For literature search that might be ok. It doesn't need to replace any other tools, and if 1/10 it surfaces something you wouldn't have found otherwise it could be worth the time on the dud attempts.

That doesn't match my recollection of the AlphaEvolve release.

Some people just read the "48 multiplications for a 4x4 matrix multiplications" part, and thought they found prior art at that performance or better. But they missed that the supposed prior art had tighter requirements on the contents of the matrix, which meant those algorithms were not usable for implementing a recursive divide and conquer algorithm for much larger matrix multiplications.

Here is a HN poster claiming to be one of the authors rebutting the claim of prior art: https://news.ycombinator.com/item?id=43997136

We also had the GPT-5 presentation which featured both incorrect bar charts (likely AI generated) and an incorrect explanation of lift.

No, your claim about matrix multiplication is false. Google's new algorithm can be applied recursively to 4x4 block matrices (over the field of complex numbers). This results in an asymptotically faster algorithm for nxn matrix multiplication than Strassen's. Earlier results on 4x4 matrices by Winograd and others did not extend to block matrices..

Google's result has subsequently been generalised: https://arxiv.org/abs/2506.13242

Well, it is important that we have some technology to prevent us from going round in circles by reinventing things, such as search.

It's an interesting type of fumble too, because it's easy to (mistakenly!) read it as "LLM tries and fails to solve problem but thinks it solved it" when really it's being credited with originality for discovering or reiterating solutions already out there in the literature.

It sounds like the content of the solutions themselves are perfectly fine, so it's unfortunate that the headline will leave the impression that these are just more hallucinations. They're not hallucinations, they're not wrong, they're just wrongly assigned credit for existing work. Which, you know, where have we heard that one before? It's like the stylistic "borrowing" from artists, but in research form.

The porn pivot makes perfect sense. Porn is already quite fake and unconvincing and none of that matters.

They know where the money is.

The porn / sex-chat one is really disappointing. It seems they've given up even pretending that they are trying to do something beneficial for society. This is just a pure society-be-damned money grab.

That's not any employee, its their VP of Science.

"rogue employee"

"OpenAI Researcher Hallucinates GPT-5 Math Breakthrough" could be a headline from The Onion.

Off topic, but I saw The Onion on sale in the magazine rack of Barnes and Noble last month.

For those who miss when it was a free rag in sidewalk newsstands, and don't want to pony up for a full subscription, this is an option.

Seriously those headlines are getting DailyMail sensationalism levels of ridiculous.

I the old world we would just use the word bullshit.

retail investors? no way. The fever-dream may continue for a while but eventually it will end. Meanwhile we don't even know our full exposure to AI. It's going to be ugly and beyond burying gold in my backyard I can't even figure out how to hedge against this monster.

"The truth is usually just an excuse for a lack of imagination."

A human being informed of a mistake will usually be able to resolve it and learn something in the process, whereas an LLM is more likely to spiral into nonsense

> Humans "hallucinate" in the AI way constantly

This is more and more clearly false. Humans get things wrong certainly, but the manner in which they get things wrong is just not comparable to how the LLMs get things wrong, beyond the most superficial comparison.

Do you think the OpenAI human, when informed of their "oopsie" replied "You're right, there is existing evidence that this problem has already been solved. Blah Blah Blah ... and that's why our new model has made a huge breakthrough against previously unsolved math problems!"

Humans are a bit better at knowing which things are important and doing more research. Also better at being honest when directly pressed. And infinitely better at learning from errors.

(Yes, not everyone, but we do have some mechanisms to judge or encourage)

it's the same thing with self-driving, if you can make it safer than a good human driver it's enough. but the bar is pretty low with driving (as evidenced by the hundreds of thousands of collisions and deaths and permanent disabilities each year). and rather high in scientific publishing.

> Humans "hallucinate" in the AI way constantly

This claim is ambiguous. The use of the word "Humans" here obscures rather than clarifies the issue. Individual humans typically do not "hallucinate" constantly, especially not on the job. Any individual human who is as bad at their job as an LLM should indeed be replaced, by a more competent individual human, not by an equally incompetent LLM. This was true long before LLMs were invented.

In the movie "Bill and Ted's Excellent Adventure," the titular characters attempt to write a history report by asking questions of random strangers in a convenience store parking lot. This of course is ridiculous and more a reflection of the extreme laziness of Bill and Ted than anything else. Today, the lazy Bill and Ted would ask ChatGPT instead. It's equally ridiculous to defend the wild inaccuracy and hallucinations of LLMs by comparing them to average humans. It's not the job of humans to answer random questions on any subject.

Human subject matter experts are not perfect, but they’re much better than average and don’t hallucinate on their subjects. They also have accountability and paper trails, can be individually discounted for gross misconduct, unlike LLMs.

Hanlon's Razor

Pretty sure you can fill a room with serious researchers that at the very least will doubt about 2) being solved with LLMs, especially when talking about formal planning with pure LLMs and without a planning framwork.

PS: So just we're clear: formal planning in AI </> making a coding plan in Cursor.

> AIME is saturated (with tool use) [...]

But isn't tool use kinda the crux here?

Correct me if I'm mistaken, but wasn't the argument back then on whether LLMs could solve maths problems without e.g. writing python to solve? Cause when "Sparks of AGI" came out in March, prompting gpt-3.5-turbo to code solutions to assist solving maths problems over just solving them directly was already established and seemed like the path forward. Heck, it is still the way to go, despite major advancements.

Given that, was he truly mistaken on his assertions regarding LLMs solving maths? Same for "planning".

> LLMs can't do math. He went on to "argue" that LLMs trick you with poetry that sounds good, but is highly subjective, and when tested on hard verifiable problems like math, they fail.

They really can’t. Token prediction based on context does not reason. You can scramble to submit PRs to ChatGPT to keep up with the “how many Rs in blueberry” kind of problems but it’s clear they can’t even keep up with shitposters on reddit.

And your 2nd and third point about planning and compounding errors remain challenges.. probably unsolvable with LLM approaches.

I don’t think it’s a wordplay with the r-word, but rather a reference to the famous Shakespeare quote: “Hoist with his own petard”. It’s become an English proverb. (A petard is a smallish bomb)

It's a play on the word petard

Hoist (thrown in the air) by your own petard (bomb) is a common phrase.

Beating the Turing Test is not AGI, but it is beating the Turing Test and that was impressive enough when it happened

We're not even solving problems that humanity can solve. There's been several times where I've posed to models a geometry problem that was novel but possible for me to solve on my own, but LLMs have fallen flat on executing them every time. I'm no mathematician, these are not complex problems, but they're well beyond any AI, even when guided. Instead, they're left to me, my trusty whiteboard, and a non-negligible amount of manual brute force shuffling of terms until it comes out right.

They're good at the Turing test. But that only marks them as indistinguishable from humans in casual conversation. They are fantastic at that. And a few other things, to be clear. Quick comprehension of an entire codebase for fast queries is horribly useful. But they are a long way from human-level general intelligence.

There's a few variables which can make it much worse.

The first is how much of the capital expenditures are being fueled by debt that won't be repaid, and how much that unpaid debt harms lending institutions. This is fundamentally how a few bad debts in 2008 broke the entire financial system: bad loans felled Lehman Brothers, which caused one money market fund to break the buck, which spurred a massive exodus from the money markets rather literally overnight.

The second issue is the psychological impact of 40% of market value just evaporating. A lot of people have indirect exposure to the stock market and these stocks in particular (via 401(k)s or pensions), and seeing that much of their wealth evaporate will definitely have some repercussions on consumer confidence.

Isn't the dot com bubble a far better proxy? Notably, todays spending is both higher and more concentrated in a few companies that a large part of the population has exposure to (most dot com companies weren't publicly traded and far smaller vs MSFT, Alphabet, Meta, Oracle, NVDA making up most investment today) by way of pension funds, ETFs, etc.?

I think my theory into contagion would be that There’s been lots of talk about these companies starting to rack up debt, and I think AI is so tied into the US GDP that things like -

If the stock market crashes, there’s lots of talk about how wealth and debt are interlinked. Could the crash be general enough to start calls on debt backed by stocks.

My recollection in 2008 was that we didn’t learn about how bad it was until after. The tech companies have been so desperate for a win, I wonder if some of them are over their skis in some way, and if there are banks that are risking it all on AI. (We know for some tech bros think the bet on AI is a longtermist like bet; closer to religion than reason and that it’s worth risking everything because the payback could be in the hundreds of trillions)

Combine this with the fact that AI is like what - 30% of the US economy? Magnificent 7 are 60%?

What happens if sustainable PE ratios in tech collapse. Does it take out Tesla?

Maybe the contagion is just the impact on the US economy which, classically anyways has been intermingled with everything.

I would bet almost everything that there is some lie at the center of this thing that we aren’t really aware of yet.

> People keep spouting this, but I don't see how the AI bubble bursting would be all that devastating.

Well, an enormous amount of debt is being raised and issued for AI and US economic growth is nearly entirely AI. Crypto bros showed the other day that they were leveraged to the hilt on coins and it wouldn't surprise me if people are the same way on AI. It is pretty heavily tied to the financial system at this point.

When America sneezes, the rest of the world catches a cold. This was said after the OG 1929 crash and I can remember it said in the 80s. Nobody says it anymore.

Due to exorbitant privilege, with the dollar as the only currency that matters, every country that trades with America is swapping goods and services for 'bits of green paper'. Unless buying oil from Russia, these bits of green paper are needed to buy oil. National currencies and the Euro might as well be casino chips, mere proxies for dollars.

Just last week the IMF issued a warning regarding AI stocks and the risk they pose to the global economy if promises are not delivered.

With every hiccup, whether that be the dot com boom, 2008 or the pandemic, the way out is to print more money, with this money going in at the top, for the banks, not the masses. This amounts to devaluation.

When the Ukraine crisis started, the Russian President stopped politely going along with Western capitalism and called the West out for printing too much money during the pandemic. Cut off from SWIFT and with many sanctions, Russia started trading in other currencies with BRICS partners. We are now at a stage of the game where the BRICS countries, of which there are many, already have a backup plan for when the next US financial catastrophe happens. They just won't use the dollar anymore. Note that currently, China doesn't want any dollars making it back to its own economy, since that would cause inflation. So they invest their dollars in Belt and Road initiatives, keeping those green bits of paper safely away from China. They don't even need exports to the USA or Europe since they have a vast home market to develop.

Note that Russia's reserve of dollars and euros was confiscated. They have nothing to lose so they aren't going to come back into the Western financial system.

Hence, you are right. A market crash won't be a global systematic crisis, it just means that Shanghai becomes the financial capital of the world, with no money printing unless it is backed up by mineral, energy or other resources that have tangible value. This won't be great for the collective West, but pretty good for the rest of the world.

Except they weren't intentionally trying to deceive anyone. They made the faulty assumption that these problems were non-trivial to solve and didn't think it was simply GPT-5 aggregating solutions in the wild.

You didn't read the X replies if you believe that

Guess even the CTO of OpenAI relies on Anthropic models in a pinch...