We will get much better at manipulation and better at people “writing” things to justify their own feelings.

What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.)

Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them.

Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.)

My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.

This reminds me a lot of the proof by construction for the 4-color theorem. It was only enabled by the advancement of computers and dissatisfied many of the computer scientists and mathematicians since it was a "brute force" approach.

I wonder if AI will end up being similar. Certain theorems get proven by AI but others do not. We haven't reached the limits of this yet and I haven't found a good argument for where those limits will be (I do doubt that there are no limits).

Sigmoidal, not exponential. It would be insane to assume an exponential curve

There are many math problems that are simply puzzles: intellectually interesting but nothing worth of value depends on it. To me it would be more impressive if we could define hard problems that need to be solved up front and see how the models deal with that.

The results OpenAI demonstrated are impressive, but it also looks like they threw a lot of compute at it just to get results. How many tokens did they waste on problems they couldn't solve? Applying inference infrastructure on a large number of math problems at scale we haven't seen before to me doesn't demonstrate an exponential curve in model abilities.

>Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation?

Is there even the tiniest reason to suspect that the people steering this progress will use it for the democratic good of all?

> The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn,

isn't it clearly split between verifiable not verifiable ? what is interesting about that question.

> but I’ve noticed Fable to be quite a big step up there

what did you notice ?

Today's models depend on inference time compute to get these results. The inference time compute available on any claude subscription is not comparable to the ones used to get some of these results (yes, in this case, it is 2000 USD total as noam confirmed, but some previous results took more).

In general, you can think of the process as generating massive rollouts in generation N, and then compiling in the verifier/human feedback("gradient") signal into generation N+1. The time taken to make the rollout in generation N, and separately the time taken to get the same rollout in generation N+1, each grows constant in some tasks, linear in more, and exponential in some.

In the end, this becomes bottlenecked by time. Today, we can make statements like "I generated all these successful trajectories with 2 weeks of compute, in the next model it will be able to do it in 7 hours of compute", but very soon you'll find yourself making statements like "I generated.... with 8 months of compute, in the next model it can do it in 6 months", which isn't really enticing the same way you can _technically_ brute force passwords but it just needs prohibitive amounts of time and money. That is the "plateau". Note that, this point is quite far away. For example, at any point if we agree it plateaus, today's known hardware techniques such as fixed function accelerators give you a 10-100x timeline reduction immediately allowing for a few more cycles of improvement. This is not to mention future innovations, but of course none of that is helping with the benchmarks where the time needed is growing superlinearly.

In many math and coding benchmarks, we are still in the constant phase. These are the massive improvements we see every few months. I'm not making any prediction of what will plateau and what will not as it's not possible to make an informed prediction about these things IMO. But the observed fact is that some have already plateaud as in, they don't improve with reasonable inference time (likely superlinear growth).

> will we need mathematicians to translate

Let's take a sudoku analogy. The model is initially just doing the random value algorithm, but lets say you the human are watching it. You make one of the usual reductions and interject "hey you can stop trying 8 here because of ....". Over enough examples, you get to a point where the model is _forced_ to learn the logical pattern. Next generation, it will skip that number. After this, you can peak the distribution using simple 1/0 RL. Doing _pure_ 1/0 RL works decent, but its not frontier as its a very sparse signal.

For that lift, human (or even a better LLM, but if you're trying to improve a frontier LLM, there is by definition no better LLM) feedback becomes necessary. This is _why_ it is crucial that these models interface in natural language and is also why the labs are hiring AI tutors by the hundreds. The "better LLM" case is what Kimi etc are doing by "distilling"(bad term for this) claude.

> But the long term is completely bewildering if you believe any of these trends can continue at a similar pace for the next few years.

For math and coding, for now we are in the phase where the times are just ... constant, so there's little reason to think it will stop soon. We still need humans to expand the frontier. It just becomes a matter of if its worth the cost of compute for running this generalized The Algorithm or not.

Given how well chess players internalized _many_ (not all) of alphazero's emergent chess knowledge, I am confident we wont have too much trouble figuring out any new math LLMs come up with, which will let us keep expanding the frontier by giving the LLM the next "lift". Only when we reach the stage where the time growth become exponential will this stop, IMO.

>Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation?

Yes.

From a mathematician who was intimately familiar with some of these problems [0]

>I don’t understand it yet. Maybe it’ll take me an afternoon to check all the calculations, but what would still be missing is why this was an approach that would’ve made sense in the first place. Is there some broader context or theory within which this would’ve been the obvious thing to do? What other results can be proven using these techniques? What is it telling us about quantum information or operator theory? I have no idea. I spent about an hour this morning asking ChatGPT these questions, but it’s somewhat frustrating because it speaks with a mishmash of physicist, operator algebraist, quantum information theorist-lingo, plus the usual LLM breezy lilt that annoys everybody.

They certainly seem to have "intuited", in a way that is not immediately obvious to experts in the field, the way to solve at least some of these problems. This was not just simply grinding away at a method that humans already knew would work and just hadn't gotten to yet.

[0] https://nitter.poast.org/henryquantum/status/208362369543662...

>Whilst current models can't 'intuit

That's how they are finding these solutions though, unless we are just going to label intuition as something only humans can do. Like a submarine being unable to swim or whatever that example is.

> Whilst current models can't 'intuit' and come up with conjectures

I disagree. I routinely let LLMs speculate or generate hypotheses along the way of helping with technical research. Sometimes they can prove the correctness of a concrete math idea but other times even an unproven conjecture helps with the numerical algorithm implementation and the result is then simply supported by additional data. I guess that any autoresearch-adjacent application has LLMs intuiting and coming up with hypotheses/conjectures—as do the steps/lemmas along a complex proof. In my opinion the modern LLMs are powerful intuitive thinkers that generate lots of conjectures of varying quality or importance.

> they can certainly disprove some of them very quickly through the kind of grind that humans can't do

Of course computers can grind in a way that humans can't. But now we have systems that convert the human-comprehensible ideas into a computer's plan of attack, in a way that greatly expands the frontier of ideas thus treatable.

> Whilst current models can't 'intuit' and come up with conjectures

People keep saying this. Why?

Surely the AI can complete the prompt “Generate new research questions based on these observations”?

When I read the reasoning traces of coding models they are constantly asking themselves questions and attempting to answer them.

Ahh, but you missed the continuation, where they get to the heart of the matter: money.

"Excuse me, We demand rigidly defined areas of doubt and uncertainty!"

DT: Might I make an observation at this point?

MT: You keep out of this metal nose.

VF: We demand that that machine not be allowed to think about this problem!

DT: If I might make an observation…

MT: We’ll go on strike!

VF: That’s right. You’ll have a national philosopher’s strike on your hands.

DT: Who will that inconvenience?

MT: Never you mind who it’ll inconvenience you box of black legging binary bits! It’ll hurt, buster! It’ll hurt!

DT: [Booming] If I might make an observation … All I wanted to say is that my circuits are now irrevocably committed to computing the answer to Life, the Universe, and Everything.

VF: That’s a -

MT: Ahhh! With -

DT: But, but the program will take me seven-and-a-half million years to run.

“All I wanted to say,” bellowed the computer, “is that my circuits are now irrevocably committed to calculating the answer to the Ultimate Question of Life, the Universe, and Everything.” He paused and satisfied himself that he now had everyone’s attention, before continuing more quietly. “But the program will take me a little while to run.”

Fook glanced impatiently at his watch.

“How long?” he said.

“Seven and a half million years,” said Deep Thought.

Lunkwill and Fook blinked at each other.

“Seven and a half million years!” they cried in chorus.

“Yes,” declaimed Deep Thought, “I said I’d have to think about it, didn’t I? And it occurs to me that running a program like this is bound to create an enormous amount of popular publicity for the whole are of philosophy in general. Everyone’s going to have their own theories about what answer I’m eventually going to come up with, and who better, to capitalize on that media market than you yourselves? So long as you can keep disagreeing with each other violently enough and maligning each other in the popular press, and so long as you have clever agents, you can keep yourselves on the gravy train for life. How does that sound?”

The two philosophers gaped at him.

“Bloody hell,” said Majikthise, “now that is what I call thinking. Here, Vroomfondel, why do we never think of things like that?”

“Dunno,” said Vroomfondel in an awed whisper; “think our brains must be too highly trained, Majikthise.”

So saying, they turned on their heels and walked out of the door and into a life-style beyond their wildest dreams.”

The first link is very sloppy and doesn't actually explain why the "certificate" proves anything about the sphere packing. Or if it did, I couldn't understand it.

The models are frequently getting worse at items that they aren’t being benchmarked for — and that’s happening more and more over time! Other people in other fields aren’t idiots, they are accurately perceiving the fact that these models are being hyper optimized for our industry, and are becoming less capable in other domains over time. Models of the same scale are massively worse at writing a broad variety of styles of prose than their equivalent from two years ago. (Models of increased scale are a mixed bag.)

Maybe you’re the one who needs breaking out of your cached beliefs.

People will be broken out of their denial by actual economic growth. That's what this is all meant to be for... I think we might start seeing some surprising numbers.

It's indeed very exciting. I'm looking forward to new advancements/predictions in physics. Preferably as beautifully explained as E = M*c^2

it is very hard for people to eat crow, as the replies will show

Never understood all this talk about moving goalposts - you understand that's how science works, right? We improve, we learn, we recalibrate our expectations based on what we've learned. If we never "moved the goalposts", we'd be stuck scoring the same goals over and over.

> The sooner people can be broken out of their denial

There is irony here

Which straw man are you arguing against?

And what does taking it seriously entail?

"AI" has limits in that it cannot invent knowledge, it can only distill and search for patterns in existing knowledge

not sure how many will get this reference but "AI" for science and math is like super-shoes for runners

at first we are blown away by the impossible improvements including sub-2-hour realworld marathon and every other PR/CR/WR is dialed down

but then the improvements slow and reach a stall point because of the limit of technology and the source of the achievement

ie. sub-2-hour marathon yes, sub-1-hour never happening (rollerblade inline-skate record is 1-hour marathon)

https://garymarcus.substack.com/p/openais-amazing-but-vastly...

https://garymarcus.substack.com/p/two-critical-updates-re-as...

As always, PR hype. Goalposts have not moved.

Guys, please use critical thinking. The haters don't hate by default, we hate because we're gaslit about this stuff every day and it's annoying. Extraordinary claims require proof, and they're not giving us information that would be essential to knowing if this is actually significant or not.

Sorry to hear you've been impacted by this AI math.

I heard that Gary Kasparov was impacted by AI chess, but at least he still seems to have a job, so don't give up.

It is a fact of experience, and indeed effectively a theorem, that the better they get at coding and math, the dumber they are. These are the wages of RLVR etc

You’ve received the expert answer several times. You just don’t seem to like the answer.

The problems from CS (CVP and circuit complexity) are very important problems that have been worked on by top researchers for 30-40 years. Some of these researchers include Turing Award winners. A solution to them would be a best-paper award at many top CS conferences.

The ones I'm familiar with are big breakthroughs, but they are both counterexamples. Examples have an advantage in that once you have the example in hand and a sketch of the proof (which they have provided), then an expert can probably work out the details themselves.

The sofic groups question was the outstanding question about sofic groups. Almost everyone thought that non-sofic groups existed, and there were plausible candidates, but proving a group was non-sofic was out of reach. Now that we know how to do it once, we can probably do it a lot more.

The Connes rigidity conjecture I think people thought was false, but it was a provocative claim to make. The significance of conjectures is frequently not that the answer to the question is "yes", but that we don't know how to answer the question. And now, apparently, we do.

> However, I am worried that the language they are using in this blog post is exaggerating for the sake of marketing

Your worry.... is because they used the word advanced? For marketing? The word is used very appropriately here. There were PhD's who spent a big part of their career tackling these problems.

"Breakthrough research" can be defined (in the citation record) as research that both (1) becomes highly cited, and (2) brings together citation chains that were previously not showing up together.

Mundane incremental research is cobbled from existing citations that already appear nearby in the record.

Basically, innovative research is a measure of bridging thought and domains that were previously not bridged. It's quite concrete as a measure in the citation record.

So we can know pretty conclusively.

Puja Ohlhaver gave a talk on this[1], and ran some experiments (that I had the pleasure to support on)

[1]: https://www.youtube.com/watch?v=guLDNMAOn24

It’s a marketing. They are a sham company. If this article was by Scientific American or something it would be worth a lot more. They are literally trying to keep the hype train on track.

Also on HN front page today: AI's debt binge can't last, hidden borrowing reaches $1.65T (fortune.com)

https://news.ycombinator.com/item?id=49160699

This starts to feel like chess engines. It’s obvious their play is superior but it’s impossible for humans to understand the moves.

It sounds like he hasn't verified the results of a problem that he has personally worked on, so how many of these problems have actually been verified?

It seems that a lot of folks misunderstand the guarantees that lean provides.

I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications) that the lean proofs are checked against. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.

I like the Lean formalizations — I hadn't thought seriously of asking for that before but might try it with some stuff I've been working on.

Exact prompts haven't mattered for about a year now.

I don’t really like AI but let’s stop kidding ourselves, no human mathematician could make progress on a dozen major open problems in a week or two. If you’re measuring it against humans then it is by far the best mathematician to ever live.

> but it seems less likely to me than before that the types of math/science discoveries will explicitly unlock better software performance.

You seem to overlook a simpler barrier. To make these advances, they have to be possible. A 15% improvement in GPU kernels doesn't evidence that significantly more improvement has been left on the table.

Correct me if I'm wrong, but all of the aforementioned advances were made in the last year? Until very recently few people had access to these tools. Most people still don't know how to use ChatGPT, and very few use tools like CC regularily. If in a few years these frontier tools become commonplace and people upskill we would should see a network effect?

This makes no sense. To believe this you have to think that the models are somehow being overfit explicitly on academic mathematics and it doesn’t carry over at all to more practical software engineering. I wouldn’t make that bet.

This will depend on the problem; I expect big algorithmic performance improvements in AI since the algorithms are still new, inefficent, and constantly being improved. But maybe not for sorting, fast fourier transforms, or other well-studied basic algorithms?

> we are also seeing incredible advances in software performance

Incredible?

> open ai announced like 15% improvement by fixing gpu kernel issue

That is... ordinary software optimization.

> novel advances in math

> we are seeing frontier level math breakthroughs (ie performance that would put it in the top 100 or 1000 mathematicians in the world if it were a human, meaning top .00001% or 800/8B)

i think you have misunderstanding of what mathematicians do

I understand the frustration with the constant PR-hype these AI labs keep spewing out, but on other hand I just can't understand this sentiment at all. These are real problems mathematicians and computer scientists have been working on and were unable to make progress on. Now they have been given a new tool and using that tool have solved those problems. And its not just one or two problems, its many very difficult problems. The mathematicians I know are saying that the latest crop of models is changing the way people do research math, I think that's a pretty big deal.

It seems like they threw it a decently large battery of open math problems and probably limited it to something like $200-500 per problem:

https://x.com/polynoamial/status/2083478171975082334

As a complete guess, it seems like they tested hundreds to thousands of problems with a relatively low per-problem budget

--

The linked tweet from Noam Brown at OpenAI reads:

> And yes we did try other major problems without success. Sadly no Millennium Prize problems (yet).

> But also, we didn’t spend a lot on each problem. It’s possible to push test-time compute much further.

I believe we're seeing a new kind of mathematics that will require completely new formats for publication, a bit similar to those used in experimental sciences. AI-powered mathematics should be fully reproducible, so it's the authors' responsibility to disclose the exact model type, inference settings/seeds and the full prompt history leading to the result. Of course that would ideally require open weights models.

It's not just about requiring to disclose AI use. AI-powered mathematics is a completely valid discipline that doesn't need to be shy, but it should develop its own publication culture.

Yeah I remember reading about something along the lines of Mathematics is now about the scaffolding around you find the problems/solutions not just the problems and solutions. For teaching purposes. This was before this ai craze

I don't think you want to bring cost into this argument.

Even if the cost was $1 mil for these 10 problems, that's maybe 10-20 math researchers for a year.

Do you really think that if you paid that to humans, they will deliver the same results?

I guess people will always find something to gripe about.

> therefore the $2000 number could be completely misleading, similar to P-value hacking by not disclosing the total experimental setup.

I don't think that comparison to p-hacking is fair. I mean not reporting price of all run is nothing like committing scientific fraud and fake results.

Also, have there been examples of researchers not affiliated with OpenAI (or another LLM creator), who have done something similar?

Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.

Not everyone works for Evil Corp. I work in the public sector and my work supports public health and safety initiatives. AI has allowed my team to get much more done than we would have otherwise which improves the quality of life of the people in my community.

So I would like to counter your cynicism with a “YMMV” depending on who you work for.

If it doesn't help average people, why do millions of them pay for it?

Downvoters mind to explain?

> I can see that a lot of technical people have ambivalent to negative feelings towards AI

My negative feelings towards AI are about energy use and inequalities, that kind of stuff. It undeniably works well, but whether or not it is better for society or the planet is a lot less clear.

The old way of establishing career credibility is being destroyed, for better or worse. Accomplishments that used to be career-defining are hard to distinguish from AI, and correlate more with access to compute. Think about Bill Gates's math paper he wrote in college. That kind of thing is gone now as a path to credibility. There's still competitions and grades, but the diversity of paths is going away. Maybe new ones will open up. This is a competitive advantage for old people who have credible pre-2025 accomplishments they can point to.

I find this whole way of looking at things weird. Did maths exist just to entertain and employ mathematicians? Surely maths is, like, useful? Not immediately, not predictably, but in the long run? In which case, whether mathematicians feel bad about it is mostly irrelevant - it's like complaining about the railway because it may put coaching inns out of business.

Given that we were nowhere near this state even two years ago, I think it’s a question of velocity more so than just distance.

The chess analogy is awful. If you simply want to know the answer to a chess problem, give it to the engine. Chess only lives on because it's a competition between humans to test their skill (just like bicycles, cars, trains didn't eliminate foot races) ... the computer is largely factored out, but not entirely -- people train with the computer, use it to check whether they played correctly, ... and they cheat. A lot. Thus there are more and more sophisticated mechanisms to detect and prevent cheating.

If you translate that to math, then all you get is math competitions, not math as a career. Of course the translation isn't nearly exact ... there's a lot more room for professional mathematicians because the math space is far more vast than the chess space and can't generally be cranked out mechanically (we have proof).

P.S. The response is nonsense ... I explained exactly why it's awful (others have too) and the response doesn't in any way refute the explanation ... rather it offers up a ridiculous strawman.

Every time someone makes a comparison to chess I die inside. Chess is a spectator sport primarily funded by a few eccentric billionaires. Players artificially constrain themselves in timed environments knowing that they will never be able to produce better moves than a smartphone because a select few people find it interesting. Only ~30 top professionals actually make enough money to have a full career playing chess, maybe a few hundred more can sustain a meager lifestyle with coaching gigs. I shudder to imagine what will happen to the tens of thousands of non-Fields medalist caliber mathematicians if math goes the way of chess. Perhaps Terence Tao and a few other famous mathematicians will be funded by Peter Thiel to report on how well humanity can keep up with the machines? How do you expect any mathematician to be optimistic about this comparison.

Do mathematicians have the right to say "no AI PRs please, the volume is too much" just like how some open source maintainers do it? I guess they feel a loss of control, there is no way to turn the hose off.

Thinking of this a little bit with the perspective of every new proof as a burden, dumped for review by actual mathematicians.

As in chess and go and also coding for the past ~year there are two groups of people: the disappointed and the enthusiastic. The disappointed are sad that they lost their advantage and that the craft they honed for years or decades has rapidly lost its value; the enthusiastic are excited about the future and what computers can bring to their domain and how it will evolve. I’m a bit of both if it comes to programming, more enthusiastic than disappointed, but also more than a bit terrified about the pace of it all. I imagine that’s how Kasparov felt back then, that’s how Lee Sedol felt and now that’s how Terry Tao feels.

The most disappointed folks will simply drop out, but the enthusiastic ones will keep going and with luck make up for the ones who decided to quit. Chess and go certainly went this way.

When you can formalize it in Lean or some such, why would this be? I can understand the desire to separate out other forms of research from the human corpus. But theoretical math that is decidable/provable, I’m not sure I see the risks.

Why?

Knowledge is knowledge, as long as it can be proven true.

https://en.wikipedia.org/wiki/Ice-nine

I’m personally hoping for the next big AI gangbanger to be theoretical physics. Boy does that field need a good reshuffle. I think when any novel mathematical theory can be done by AI you’ll see simultaneously theoretical physics getting wrecked as hard as pure math is. At that point we might see new physics or paradigm shifting technology emerging.

There's no clean line between a collection of theorems and a theory.

It will not happen with existing LLM techniques.

You mean we're still gonna be employed doing the boring part while AI gets to do the fun part?

I'd honestly rather they just automate every job at that point.

> The results were achieved by an internal version of Astra, our next major model.

surely some human regularly typed "think deeper, make no mistakes".

It seems that a lot of folks misunderstand the guarantees that lean provides.

I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications), that the lean proofs are checked against, indeed correctly encode the real theorems. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.

well, a bug in the Lean kernel was discovered last week by way of an LLM tricking itself and its handler into believing it had found a non-constructive proof of the existence of a nontrivial Collatz cycle, see https://infosec.exchange/@0xabad1dea/117002106099986943 and https://lipn.info/@mevenlennonbertrand/116997917683191056

I'm not an expert at it myself, but my understanding is there are numerous ways to "cheat" in a Lean proof (via `sorry` and similar). They're taking responsibility for fully verifying that none of these cheats were used (and that the theorem statements themselves were all correctly formalized.)

No, the correctness isn't for the "inside the Lean proofs", but for the translation of "human language math" and its formal Lean variant.

Why would you factor in salary unless they had to baby it through. You would only count the hours for setting up the harness and prompt and checking the result.

Training the model is going to be amortized over other uses.

> The cost of generating the proofs for all 10 of these breakthroughs combined was under $2,000 at Sol API prices.

https://x.com/polynoamial/status/2083470822258467194

Given that OpenAI pays their employees with stock surely a breathtaking number, but not a very meaningful number now that the infrastructure is in place and the models are trained. AI could never get better and it would still be incredibly disruptive.

Not much point to pure math being kept secret, in all honesty. There isn't really industrial value, its only purpose (to them) is showing off their model's capabilities. More realistically they'll just stop paying for it.

Edit: Oh, are you suggesting they just use it to privately improve their models? I imagine a few more correct proofs would have a very marginal benefit, if any. Also, they'll probably just get extracted, meaning it still gets out but OpenAI doesn't get to fancily announce it themselves.

What does this even mean lol. These are not solved questions. The solution never existed.

I’m starting to think the probability of RSI within 12 months is more like 99%

I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which would more likely require months/years, but algorithmic progress would likely saturate quickly. I guess it depends on how much you think further AI progress depends on hardware vs. software.

Okay, let's take it seriously. What do you propose? What can your average person do to prepare for RSI beyond bracing themselves mentally?

Found some discussion here [1] from someone who actually worked on a few of these problems.

[1] https://x.com/henryquantum/status/2083623695436623915?s=20

For OpenAI, research is marketing. I’m sure they’ve got plenty of budget for that.

No such thing as free, even internally at a company. All such use of resources is accounted for, assigned a dollar value and billed to some department. Someone ran the numbers and figured that whatever they spent on these GPU cycles was worth it.

Presumably it's a rounding error compared to their full output, and they're making sure they have enough compute set aside for research by limiting public models. The more datacenters they build the less they have to limit them.

RL training can use all of them - idk what needed means.

I love how people come up with creative ideas to prove the bubble. This one is even more ridiculous - that OpenAI had spare compute to advance mathematics proves that data centres will not be needed. WHAT.

If anything it proves more data centres are needed. That's literally the only reasonable conclusion from this news.

These is mostly mathematics, not science, and they link the paper in the post: https://cdn.openai.com/pdf/74c24085-19b0-4534-9c90-465b8e29a...

Mathematicians will tear it to pieces if any of it is fake!

This isn't really a productive way to think about these things, IMO. It's quite possible it would take hundreds of years for any specific group of PhDs to solve them. Or one individual PhD could have the correct flash of insight and solve it in a month. There's absolutely no way to predict this, besides trying to gauge the apparent simplicity of the proof or counterexample (which is likely to be misleading). Until someone actually runs an experiment like this it's not a viable metric.

A friend’s PhD advisor has been chasing non-sofic groups for 25 years (and was shown a preprint of the results by openai to verify them). He believed a solution would be Fields-worthy

This was not a problem that was for sale

You couldn't. PhDs have been working on these problems for decades. It wasn't for lack of trying that none of them could figure these solutions out!

They've been hiring mathematicians to verify this stuff themselves. They're obviously not just throwing it out there without any human review.

This is something I have been thinking about for quite a while. I readily embrace the advances AI may bring to mathematics and the hard sciences, but those advances were largely expected even predictable.

What I had hoped for was something more ambitious: using AI as an arbiter in economic, political, and social debates, one capable of weighing evidence, exposing trade-offs, and helping us make decisions that produce better outcomes over the medium and long term, even when those decisions conflict with powerful private interests.

I suspect, however, that this is not a particularly urgent goal for the people funding and directing these systems, many of whom live far removed from scarcity and its consequences.

Solving math problems doesn’t require the cooperation of rival factions.

Not really. They're an AI company: they develop AI and sell it. There's some room for pulling off flashy marketing stunts, but not all that much.

This is division of labor, and it's a good thing. I'm sure OpenAI employees, who are very well paid, donate some money from their salaries to others working on the areas you're citing: probably more than you think, I say that from having attended some EA parties back in the day.

But that isn't my point: my point is that a company which makes brushless motors should put most of its time and money into solving the "make and sell brushless motors" problem, and if they or their investors feel like they need to do more for the world, give money to the people who have the time and ability to do things about that. There are a lot of quality-of-life improvements which need brushless motors.

Next question is how useful their product (OpenAI, I mean) is to more focused do-good-in-the-world professionals. I'm sure that varies quite a bit. For getting the homeless off the street? I conjecture, not very useful. For 'complete the transition off fossil fuels'? Extremely useful, no one in that field knows how to do their job without AI in summer 2026. I'm certain of this.

Yes, but all of those are orders of magnitude harder than math.

None of these problems need technology to solve. Step 1 is trying to get different groups of people to cooperate with each other. Good luck with that.

What's stopping the non-profits that exist today from just putting more money into tokens to get the solutions they want?

Seriously?

We already know how to solve all of these issues. What we lack is collective political will.

Almost like superintelligence solves these problems...

We know how to solve food insecurity (in 1st world countries). We have plenty of food. Capitalism requires though throwing out food that can't be sold because billionaires find giving away things anathemic to their worldview. Get rid of billionaire sociopaths.

I think we previously assumed that AGI would need to come before superintelligence, but it kind of seems like that's wrong? Super intelligence in a narrow field has arguably already arrived (solving problems that were previously unsolved), but we are still pretty far behind human abilities in things like spatial reasoning or computer use.

It’s artificial, it’s general, and it’s intelligence. The people who believe “AGI” is an important and unattained goal need to start coining and defining their terms better.

It depends on your definition. For me it would have to be able to do the stuff humans can do like make a cup of coffee (Wozniak test).

Just maths isn't really general enough for the G in AGI.

The Maxwell conjecture was a conjecture in theoretical physics (though not a particularly important one)

The fundamental obstacle is that we have no conceivable way to produce the energy levels to test the predictions that new theoretical physics would produce. We are like over a dozen orders of magnitude off.

Ambient 0: Math for Airports

It solved a handful of novel esoteric problems out of hundreds fed to it. Far from the end of science.

It's just another tool that can help solve problems. It doesn't know _what_ problems to solve. It turns out that a lot of old problems are now low hanging fruit for these new models. In terms of 'expanding the frontier', we've just discovered dynamite and can now blast our way through mountains. The bottom of the ocean or space are still as hard to reach as ever.

No bud, it’s just the beginning!

The main issue was that it was submitted late Friday night SF time, meaning it was overnight or Saturday everywhere in the world when the post had its chance on the front page. The flagging was minimal relative to the vote count and had no effect, and the flamewar detector would have been turned off sooner if moderators saw it sooner (it wasn't really a flamewar, just a lot of comments). It still spent 10 hours on the front page.

None of this is anything out of the ordinary; this kind of thing has always happened. The only real story here is that moderators sleep sometimes.

> HN is where I expect to read expert comments on these topics, has this style of conversation moved elsewhere?

For people who have been paying attention to accurate predictions leading to our present state of the world, HN has collectively been reactionary, incorrectly dismissive, and incredibly behind the curve. In public, people are better informed on LessWrong and AI Twitter circles. That's where frontier researchers are. Barely here

I think that HN is particularly negative towards AI because the vast majority of users here will have their prestigious CS careers disrupted by AI advances. So, there's an inherent negative bias towards this news.

I for one am really fascinated by AI's advances in science and math and would like to talk about it somewhere without the constant flamewars...

"People weren't their strongest even when most did manual labor."

Is there good historical data on some measure of strength across representative populations over time in the modern era? I'm doubtful.

We do know that the introduction of agriculture diminished strength:

"Bone mass was around 20% higher in the foragers - the equivalent to what an average person would lose after three months of weightlessness in space.

After ruling out diet differences and changes in body size as possible causes, researchers have concluded that reductions in physical activity are the root cause of degradation in human bone strength across millennia."

cam.ac.uk/research/news/hunter-gatherer-past-shows-our-fragile-bones-result-from-physical-inactivity-since-invention-of

This is not at the top as it is actively flagged by people that can't psychologically cope with the advances of AI. Hacker News is no longer a web site of an elite.

I don't think that's it. Multiple or my friends from the target audience (academic mathematicians) admitted to scrolling past because the title made it sound like a review of last month's contributions, instead of 10 new ones.

What about AI research itself? Is OpenAI close to automating its human staff out of a job?

It’s also very very divided (x companies, oss vs not and other interests)

This is one of the most impactful mathematical publications in history by all accounts

I think we've now hit a point where 99.9% of the population gloss over these types of AI advancements because of human competence being insufficient

No human could have published this because it requires paradigm shifts (e. g. Section 5) in multiple mathematical domains. Mastering one of them to this degree is rare, mastering 3+ pretty much non existent for humans.

Honestly just a bit burnt out on posts like this

Its visibility was diminished due to the flamewar detector and most of its front page time being during overnight hours on Friday night/Saturday morning USA time. I've created a new copy to give it some primetime exposure, because it seems like an important enough announcement to warrant it.

tomhow explains they gave the story another shot here: https://news.ycombinator.com/item?id=49158443

It's simple economics. Building a robot to do your chores is expensive and only a small minority of people value their time enough to buy one. Meanwhile, SWEs are expensive and GPUs are (comparatively) cheap.

You shouldn't be getting downvoted for something that a majority of pure math and art enthusiasts believe to be true. The truth is many of these entrepreneurs and VCs are obsessed with AI not for money or human progress, but because it makes them feel closer to being a "god" rather than a mere mortal. Much of it (especially AI art) is out of spite for human creativity, which is done by mortals with limitations.

Very marketable nerd snipes indeed.

Sorry, OpenAI's take is correct here. If you're not convinced, here is how they prompted LLM: https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98... [0]

A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt.

[0]: Not one of the proofs in the linked article, but from OpenAI too.

Your brain also is physical. Electrochemical gradients flow between physical molecular constructs. Isn't it just chemistry? Do you attribute it to physics or some whole-is-greater-than-the-parts idea?

> Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down.

When the tool is a 3D printer, or any CNC system really, you bet I attribute a build to it.

I could also attribute the operator; there is no contradiction, it's a free choice, just like saying "I am in Berlin" does not contradict "I am in Germany".

> AI has no self-awareness

What is your mechanistic model of self awareness that yields this conclusion?

> It's a tool

Does your model suggest that tools can't have self awareness?

A better analogy would be a manufactured object, say 3d printed for simplicity. The 3d printer is given an input, and an object manifests itself after some time. We say that the creator of the object is the person turning on the machine, sending the data, and collecting the object. Not the machine itself.

It's worth reading Marcus' first line, for the naysayers and flaggers on this post:

> Astra, a new model that OpenAI is testing internally, is amazing. No denying that.

I find Marcus on this, something approaching sophistry and rhetorical showmanship in service of maintaining an ideological position, for reasons unrelated to the nominal intellectual clarity.

To sharpen that, I think he's (obviously) interested in maintaining his own brand as "thought leader" and this necessitates de rigeur defense of particular postures.

Sometimes this is easy because the facts warrant it; other times, a bit of rhetorical license is required to preserve nominal coherence and (at least, for the moment) hold certain lines.

This is one of the latter cases, and it's not subtle.

One of the celebrated properties of many intellectual advances or inventions in whatever domain is precisely that it appears obvious in hindsight. It is quite cynical to leverage consensus distrust of large AI players, warranted but also a popular social construction, to insinuate that these are not "real" advances or "real" hard problems, on the grounds they were in some sense cherry-picked.

Identifying the problems amenable to strategies on the table and intuitions (sic) about where bridges might be, is exactly the discerning work that is the core driver of almost all prior progress, but for celebrated accidents and flashes of insight. Anyone working in any challenging discipline knows that those are celebrated and told around campfires precisely because meaningful durable results arising like that is so uncommon.

These two articles make me think of nothing so much as my own durable reaction to the creeping goalposts of AI critics generally: that they often seem to me not unlike a water color cohort scoffing and jeering at the horse, because it got a D on its tensor calculus exam.

Marcus should be on guard against his own cynicism and take care that his assumptions do not prevent clear sight.

"Gary Marcus' has a good take "

I think that is an oxymoron.

Zero. These were open problems.

This wouldn't be a problem so long as they properly attribute.