Why wouldn’t the AI be able to answer questions about the proofs or organize a talk? Those are pretty much the things it excels at, exploring and summarizing previous knowledge. The problem is the volume of questions and data a human needs to step through to arrive at the same understanding.

Fair, “answers” is probably more precise than “knowledge.” That gap between answers and understanding is the whole point of the post, so I think we mostly agree.

> It is ultimately the understanding

But understanding to what end? I'm if it's for the pure aesthetic experience of engaging in the beauty of math than that's fine, but that's typically not the view from my experience.

The goal of understanding is to advance the field, since historically human understanding was the only way to do this.

For the vast majority of human history, the world we experienced was a strange and unintelligible space. Humans survived at the by the grace of forces beyond their control. For a brief window of time human existed in a world where they increasingly saw a relationship between their understanding of the world and there ability to steer their environment towards a desired outcome.

I believe now we're seeing that that was a historical anomaly. The world is once again entering a state where it becomes increasingly unintelligible to us. Nick Land already famously pointed out, many decades ago, that the global financial system is already a form of "artificial intelligence" we don't really control. We're seeing that spread wider and wider.

The world making sense peaked around 1960s and has continued to breakdown ever since.

Does that mean AI is not doing something useful? If it provides a proof, it gives direction to human mathematicians who can then develop an understanding, give talks, be questioned about the topic, and teach. Is that not a useful service?

I guess the issue with that is that the understanding is important to mathematicians. The majority of people outside mathematics don’t understand it whether it’s humans doing the work or AI. If AI can generate useful results more quickly, then to the general public it is more valuable.

> We’ve traded one black box for another, and still need to unpack it.

The human mind has been a black box, and largely still is.

So we traded one black box for another, in more than one way. The new black boxes are going to be able understand and keep up with their own work, much more quickly than we will.

The "alien" picnics, in this case, are a prelude to settlement and expansion, and we have only met the children.

Your point on understanding mostly holds (although our own mind's inscrutability but usefulness is a strong counter-case), but it will increasingly work against us.

Usefulness doesn’t always rely on understanding. And it’s not a new thing in mathematics to have an answer without fully understanding it. We’re talking about a change in the rate of that happening, not a new phenomenon in itself.

They are very much useful, I even have an example in the beginning. They wouldn’t be plundered from the zones and sold for crazy money if they weren’t useful. I’m also not saying that what AI is dropping now is akin to alien artifacts. I’m posing the question of whether it may get there.

I think there is a difference here; the modern telescope doesn't output answers, claims or theorems - just data that still has to be interpreted to be even remotely useful.

I tried "Monday Begins on Saturday" and had a pretty hard time getting through it. But I have to give it another chance - and check out the ones you mentioned.

Both of those are listed on the image of the book cover in the article.

Yep it's the OG literary version. Very much worth a read.

Also the game Pacific Drive.

No real spoilers in the article though. Just one of the ideas revealed very early in the book.

[flagged]

It doesn't assume, it just observes. It has the capacity to use computers better than us, more persistently, in a more scalable way, to solve problems we haven't solved before like Navier-Stokes and many more to come. You can argue all you want about whether it's intelligent or not and what it means, but the solution is now here for us to process, understand, and integrate forward into our model of the world.

That's probably not the point the author is trying to make. In the book humans are trying to make sense of the artefacts lefts by the aliens, but are only able to do so slowly. The gap between human understanding and the aliens who built the artefacts are just to large.

The book never tells us that this is actual trash, it's just a theory. If the theory is true however, the aliens might also not understand the damage they've done, it's just a bit of littering to them. Or they don't care, they just destroyed an ant hill, so what.

The author of the article might have a bit more faith in the current generation of AI than I do, I don't believe that LLMs are really capable of creating that big a gap. The discoveries they make are all based on human knowledge and understanding, so we can comprehend the foundations on which they are built. Now if models can train and iterate on their own, to a much greater extend, then maybe, some day, in the far future their discoveries could leap frog humans to the same degree as the aliens in "A Roadside Picnic" and not understand that their output is damaging to us. That's just not going to happen with our current LLMs, they aren't going to scale up to that extend. There are already articles suggesting that we're almost at the peak of what they can do, with the resources we allocate.