First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.”

They also said “Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge … and Tristan Buckmaster….” They say the rumor was that two Millennium Prize problems had been resolved, and that this prompted them to launch "an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems."

It's not obvious to me that's an unethical thing to do, if it happened as they described.

> Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through.

What would OpenAIs incentive for this be? They've gotten away with scraping everything and getting it ruled fair use. It seems like willful ignorance is an affirmative defense today. Why would they want to have some sort of audit trail that could prove otherwise?

The irony is that OpenAI got into this trouble only because they tried to play "nice". They told Buckmaster that he could publish the final result as the author as long as he removed Alpöge from the author list. They wanted to give Buckmaster a chance to be the one solved N-S problem.

While this behavior is highly questionable, if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researches, there would be no ground for anyone to accuse OpenAI for anything. Their self-perceived "generosity" backfired dearly and I'm sure they'll never make the same mistake again. There is probably a policy forbidding any OpenAI employee to contact external researchers like that now.

I think this move by OpenAI is crazy. At best, if all unconfirmed accusations are unfounded, they still heard a rumour that someone had solved a huge million dollar problem and was about to make a name for themselves. Then, they decided this was a good opportunity to pour millions of dollars into trying to snag the glory while the researchers were busy cleaning up their notes and polishing the announcement.

That still sounds highly unethical.

Another scenario also just surfaced: https://news.ycombinator.com/item?id=49657499 – the construction used by Anthropic for the Jacobian Conjecture counterexample appears to have existed in an unpublished but publically available document. This document of course wasn't referenced in the announcement (nor did we ever get to see the prompt or reasoning traces).

If the model was trained proper to the conversation with the researcher took place, there'd be no question of tainting the results. But if any amount of training on the model took place afterward, then yes, everything is thrown into doubt (a core problem with considering anything "original" from a model because of how >a % of everything ever written has been used a corpus for the training).

So, at my company (and most companies I think), we use confidential in-house versions of the AI software. We don't want any confidential information leaking into the public realm. Are these scientists doing that, or are they just using the public version of the software?

> you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through.

They have a financial incentive not to track any of this, so why would they?

OpenAI’s entire business model is predicated on stealing other people’s work and selling it to the masses.

The problem here is that OAI (and others) pretend or claim that this is uncharted legal territory, where in fact it is very simple. We have a machine that is fed data, and produces new data as a result. If that new data depends (in any way) on the fed data, then from a legal viewpoint it is derived from that data.

Whether they anthropomorphize the operation performed by the machine does not matter. They can anthropomorphize when/if the law is updated to include such terms, but right now they certainly cannot.

> Provenance is hard to track

Right, which is going to open a lot of doors to a lot of questions.

I don't think there's any legal ramifications on this, just ethical ones about when and how you publish research, but it's yet another point in favor of "if provenance is hard to track, should we be using this for things where it needs to be".

Obviously copyright/trademark is a huge discussion on this, and I could absolutely see this devolving into that as well with how certain findings wind up monetized.

We have a response in this topic from someone claiming to be from OpenAI and linking an article where they, roughly, say "we are sure nothing from the 2 month period made its way into the solution". If that is true, that should mean it is provable, but leads to some more open ended questions like "well what data did it use then?". Is this still okay if someone close to the author did plug data into open AI and it extrapolated it?

Obviously that's probably an unreasonable expectation for these models to track and prove, but it also used to be an unreasonable expectation to scrape every single piece of digital and physical info for consolidated data.

If I opine to a friend on a park bench about a story I'm writing, do they get to pull it from the flock feed, shove it in the model, and then provide it to disney?

Legally, right now, probably. But there's going to need to be a serious look at laws and standards. Or a major shift in what is and isn't discussed in public if literally every breath and move you make can become monetized.

> I think it's a useful analogy to compare OpenAI to a human collaborator.

Frankly I don’t buy this. It’s not a human or a collaborator. It’s a tool. This is like saying it’s not Microsoft’s fault if they extract a bunch of data from people’s Excel sheets because they willingly put it into the program. Anthropomorphizing software is ignorant and foolhardy

The AI not being human doesn’t escape ethical consideration - OpenAI employees are culpable for what they build.

This was academic research. Could just have easily been trade secrets and proprietary data.

I think OP's analogy is bad. The difference of OpenAI when comparing to human collaborator is the possibility to replicate once learned skill. Imagine if any single human collaborator learns a skill it is immediately a skill of any human collaborator.

Then there's the possibility of indirect training via modern spy devices ("smart" IoT devices like LG TV's) feeding the transcribed ambient conversation data for summarization to an agent [0].

[0]: https://youtu.be/6IFVTcM28KA

> I think it's a useful analogy to compare OpenAI to a human collaborator.

A human collaborator who stands to win or loose a couple of $100B.

OpenAI says deidentified data from the private sessions go into training. (Well, explicitly said they will not rule that out.) That changes a lot of the conversation.

> but a full data trail of all inputs is difficult to trace through.

Great use case for AI agents

Bad analogy. OpenAI spent millions on compute to get their result. This is more like if a billionaire heard of your promising mathematical lead and then gathered hundreds of top mathematicians to work on it.

I think it's better to ignore OpenAI here, because OpenAI didn't do anything.

Academic research is a professional field in the traditional sense. Individual researchers are ultimately responsible for their actions. If some OpenAI employees violated academic norms while doing academic research, they should be judged by academic standards.

Scooping someone else's result is immoral but not an outright violation of academic norms. But if you are in possession of relevant confidential information, you are expected to steer clear of the topic. It doesn't matter whether you actually used the confidential information to get your results, because outsiders can't know that. The mere fact that there is a plausible suspicion already puts your integrity into question.

Tenured professors occasionally lose their jobs over similar scandals (but usually don't). If OpenAI wants to regain some goodwill, it should do a thorough investigation that may lead to firing the individuals in question. If it doesn't find sufficient evidence of wrongdoing to justify any disciplinary action, it probably doesn't gain any goodwill either (as it often happens with similar investigations at universities).

And if OpenAI wants to be a trustworthy partner, it should transform into a company of boring gray bureaucrats who provide an essential service without competing with their customers.

> I've been wondering whether AI really is improving rapidly at open problems or we're being fooled.

I think your suspicions are warranted and your explanation seems plausible.

If better training data is the reason here, it would still be a case of the models doing something that is in and of itself super useful! The models really can take that data and distill it into solutions for similar problems faster than humans can. This is great!

But there's so much vested interest in the AI companies to be opaque about all this, to hype up their models and avoid giving credit to people whose data made everything possible, that they would never tell us this fact if it were true.

I feel like so much of the AI hype cycle is like this. The models develop extremely useful capabilities, but it's hard to understand what they really are through the hype. The lies and obfuscation by their owners who have vested interests in capturing the value they provide makes it impossible to take anything they say at face value.

I’d argue the invitation of researchers was incredibly strategic.

Sam Altman knows what he’s doing. He will happily screw these folks to one-up his competition.

A mental model I was thinking about was - I remember when Travis Kalanick was talking about using the chatbot to discuss “vibe physics-ing” on the all-in podcast.

And like - I think there’s a presumption you could make that AI models could overfit to asymptote towards just the capabilities and knowledge we currently have.

And that would be amazing! And crazy useful. And there are probably a whole world of complex problems that remain unsolved because they’re adjacent to knowledge we have but they haven’t been invested in.

But can a human reliably tell the difference between “can do 99.999% of the things we currently know how to do which includes a small subset of things we didn’t know we had the capacity to do” and “super intelligent math and science research pushing the frontier of what we know”

A physicist that knows all the things we currently know in excruciating detail feels like it should be able to make the leap beyond the frontier.

But since these are computer models it might just be that it can ride that line extraordinarily well while the line remains firm.

> - OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay

I've been thinking along exactly these lines... they very well could have a 21st century Mechanical Turk and its real superpower is getting people to "collaborate" asynchronously but it's just stealing their ideas and laundering them.

I don't think it's purely that, of course... but "consult other clients' transcripts" would be an easy tool to write.

there are also attempts to crowdsource human research directions - like the caltech mathathon challenge : https://mathathonchallenge.com these would help models on the same problems at the expense of the researchers. basically, math researchers are the reverse centaurs but they dont realize it.

The relentless progress towards saturation of benchmarks is, I suspect, at least partly a similar story. Whatever holdout questions are used to evaluate GPT-x will be in the GPT-x conversation logs, and therefore in the training set for GPT-x+1.

Can't both be true?

1. Systems that OpenAI is able to use (either public or private) are improving rapidly at open problems, even if they are still extraordinarily expensive

2. Researchers will inadvertently speed up the rate at which the AIs improve by feeding them valuable training data

This is pretty much the definition of a data flywheel.

>> Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2]

Maybe I'm failing to read that graph properly but the y axis says "pass rate" and it only goes up to 0.5. That would mean every single problem is at most half-solved.

I don't know what that means though. What is "0.5 pass rate" in the context of "open math problems" (as in the graph title)?

Seems easy to picture high stakes startup cutting corners to justify their fame.

> they could be significantly piggybacking on human progress,

This is AI in a nutshell, its a plagiarism machine. An abstraction layer between vast amounts of stolen human-generated data that filters out the liabilities and accountability for that original theft. Its an IP laundering system.

It tells me that AI companies are just another mechanism to extract and extort value from the masses for the rich.

Just another rich man’s trick

Perhaps the last one before they destroy that world and try to hide away as people forget and history is rewritten again. I don’t think they’ll succeed this time.

The pudding is in the proof. The field is mathematics, the proof can be rigorously verified. If there is a flaw, OpenAI is out to lunch. If the proof is valid, OpenAI has produced something new.

On your second point: there is a more plausible explanation which David Bessis calls the "overhang". The short version is that there is a large amount of relatively low hanging fruits in mathematics, because no human has broad enough knowledge and enough time to try them all. AI is not constraint by that, and therefore can systematically pluck all those low hanging fruits.

Quote: "The Overhang consists of the unrealized capital gains of past mathematical creativity, the latent value from connecting the dots in the existing corpus. It is a dividend of canonization. Mathematician X states problem A, mathematician Y crafts concept B, then mathematician Z notices that B trivially solves A and “captures” the social reward. But in the process of capturing the reward, Z usually introduces new concepts and new open problems, reinjecting latent value into the Overhang.

LLMs can be trained on the entirety of the mathematical corpus. Thanks to their phenomenal memorization and pattern-matching abilities (without always being able to map out their associative logic and attribute due credits), they are in a unique position to harvest the Overhang. By contrast, professional mathematicians have typically read a few hundred articles in their career, out of millions of existing references, less than 0.1% of the total.

This will lead to great discoveries, which is unambiguously exciting. But it could also lead to a sad new deal, where human slaves painfully curate the Overhang while AIs systematically beat them at the finish line."

source: https://substack.com/inbox/post/183753276

OpenAI said they sicced this agent army on Navier-Stokes on Sept 1st, while only a couple of days earlier OpenAI's Noam Brown happened to reply to a tweet saying that they had already tried to solve all the Millennium Prize problems and failed... So, it seems either the previous attempt didn't have the training to succeed, or was just not given the compute to do so.

Once OpenAI heard that Navier-Stokes was solved, this caused them to immediately revisit the problem and throw a ton of compute at it, apparently using a more (very) recent model than what they had tried before. What we don't know is just how recent this model was, and therefore what it may have been trained on. Buckmaster/Levant had apparently been working towards this for at least a year, and made their "forced" blow-up breakthrough on August 15th.

Presumably any anonymized prompts that are being trained on are part of pre-training, so older, but once OpenAI had heard that Navier-Stokes had been solved and wanted to revisit it, it seems possible they may have done a few weeks of incremental RL training on anything Navier-Stokes adjacent they could come up with, in addition to then throwing unlimited compute at it, now confident that there was something to find.

Even OpenAI's own publication [0] on Navier-Stokes from two days ago appears to contradict "basically solving anything you throw at it". The chart shows a pass rate of ~0.5 (vs. Astra's ~0.2) on "a curated set of open math problems". (Based on the timelines and events described in the publication, I presume that the "Internal Model" in the publication represents OpenAI's latest and greatest model. Evidently, this pass rate may improve in the future.)

[0] https://openai.com/index/navier-stokes-solution/

I feel that we don’t praise Lean enough. AFAIU it’s what enables LLMs to brute force those problems

Both can be true:

1. OpenAI couldn't have solved the problem without the researchers' private data for training.

2. OpenAI models can solve math problems

If your rumor is true, what we are witnessing is a giant paradigm shift rather than individual incidents. Mathematicians were the first victims of super-intelligence.

Of course it’s not an endless source. They had to burn millions of dollars to solve a single problem.

> The rumor I’ve heard from multiple employees at OAI and Ant is that the model has solved hundreds of open problems in maths

Obviously these are unbiased and trustworthy sources.

If they have solved hundreds of open problems in math, why are they publishing results for the ones other mathematicians happen to be working on at the same time? Why not the others?

The leakage wouldn't be from training, but from other uses of Personal Data.

As far as I understand it, users can opt out from the training aspect, but they cannot stop their conversations (“User Content”) being used “[t]o improve and develop our Services and conduct research, for example to develop new features”.

Just use Bedrock...

The big question is whether OpenAI is training on "de-identified" sessions that are marked as "do not use for training"

The answer is almost certainly yes, and this is a problem for most users.

> We’ll know soon enough, but I’m inclined to believe this is true.

I mean, we’ll know as soon as they decide they want to provide verifiable proof. Really dragging their feet on this front so far.

I’m inclined to believe this is false.

The Cult tells us the AI is almight andpowerful; unfortunately, the cult cant actually describe the indescribable.

I think people are focusing on the training data issue too much. If the data was contaminated, I can still blame that on negligence.

But, at least with the Navier-Stokes solution, it's clear [^1] that they learned that Alpöge and Buckmaster were getting close to a solution and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt.

What makes this worse to me is the intention. They intentionally threw $15 million in compute at the problem in order to scoop the result. They intentionally left Buckmaster and Alpöge out of the citations.

Data contamination should be enough to disqualify them from the prize, but I can believe it to be accidental. On the other hand, someone made an intentional decision to scoop the result by throwing money at the problem. That's so much worse.

[^1]: That's the timeline claimed by Buckmaster, and no one from OAI has disputed it.

That link is a helpful contribution to this discussion.

I'm not at all familiar with this area, but my reading is that he appears to call it out as a relatively obvious extension of his own work:

> It is a creative and at the same time elementary construction that uses not just property (T) for an application of my result with Kun, but also for the ambient group G in order to overcome the problem, that the Γ-components might be of different size. Once this is achieved, the rest of the argument is straightforward.

Creative and at the same time elementary is where LLMs excel, generally speaking. It's why they are so good at writing code.

This has been my line of thinking as well. I have developed a sort of paranoia when I'm working using AI on my projects. Who's to say Claude or OpenAI isn't using the final conclusion of all my ideas, trial and error, and adding it to their database of insights to be offered to the next subscriber for a price?

They have demonstrated both the intelligence at scale and the lack of morals for this to not be a problem at all.

>> Their final “product” is not just a standalone ML model. They don’t need your data just to “improve their products and services”. They build a whole ecosystem and infrastructure around gathering all the knowledge in the world. Including private and secret knowledge traditionally gathered by “intelligence” agencies. Now artificial intelligence agents can do the same.

And people thought Experts Systems were bad.

I would say there is a significant difference between AI discovering this completely on its own versus AI creating the finishing connecting part by connecting relevant data. Maybe this claim is too strong, but if part of it is true then the claims that OpenAI have made would be too strong as well.

To me it would feel more like how LLMs seem to work for me personally: incapable of unique work, but very capable of capturing large amounts of data and connecting the dots.

A lot of math is extremely specialized, to the extent that only a handful of other experts in some field have any experience with those mathematical ideas, with most of them not even yet present in the published literature. It's really not a stretch to claim that it's pretty dubious when the AI decides to use these highly specialized tools after it has trained on chat logs where these techniques were being discussed.

> They don't even claim to have had a proof, only to have been working on it.

Yeah, the guys who solved it for Euler and in the hypoviscous case, with the same technique that worked for full Navier--Stokes. They were "just" working on it.

The AI only seem to solve the problems that it had human trading data on…

If this wasn’t human driven, I’d expect to see other problems within that problem. Space solved not just the ones that it had chat data on.

Yes. See https://www.anthropic.com/research/small-samples-poison?from....

250 documents ingested from somewhere is enough to become part of the knowledge of a model of arbitrarily large size.

I would expect that a good idea that fits in a framework that is already being ingested would be more easily taken up than some random thing unassociated with anything else. Could that go down to a single transcript? If the model is consciously focusing on everything X related, quite possibly.

Problem is how do you convince the model and training profess it matters. A one off canary is very unlikely to survive in the final model state.

PaaS: an acronym for "Plagiarism as a Service" which replaced the older terms AGI, GPT and LLM in late 2026. Origin uncertain.

Pass it on.

I run such tests since a long time at chorasimilarity open notebook.

I always used guest non login accounts.

As a mathematician I was able to check two plagiates (by humans) with even such primitive means.

But I have to mention that some things irk me in this conversation about math or science and AI.

First, I see lots of attribution and other related problems, with certain impact for the researcher proffesion.

But I don't see the most natural question: wouldn't you like to know the answer to _open-problem_ ?

I mean, is research now only about publishing and solving famous problems?

From this point of view I think the links from this recent post are depressing

https://terrytao.wordpress.com/2026/09/10/crowdsourcing-a-li...

Second, I think very relevant that the original meaning of "encyclopedia" is "recurrent education".

So I arrived to think that the present and future forms of AI in mathematics and sciences should be seen as modern day encyclopedic efforts.

Once we pass over the flurry of solving famous open problems (and wouldn't you like to know?) the next natural step is an audit of the ehole corpus of mathematics and sciences accumulated until now.

And then pass further on a saner basis and damn about problem solvers and unhappy publishers and management.

Even if that were true, they've already admitting to throwing vast quantities of resources to scoop a researcher who was about to publish (because they'd learned, somehow, of his breakthrough). If that doesn't bother you I think you need to take a step back and have a good think about this.

I can say categorically that OpenAI is not a credible or trustworthy company.

Where did OpenAI say this?

And it still leaves open the question of the prompt itself, which can just as easily encode information about the same knowledge.

Regardless, they started working on this problem after hearing that one of their customers was already working on it. It almost doesn't matter about the training data. This is the provider you are paying undermining your career.

I haven't seen that. Where is that from?

Why should we trust them?

The ultimate drive for some researches is the pursuit of knowledge. If I'm stuck at some block which prevents me from continuing in some direction that I want, of course I would like some help. I believe we already have nonzero collaborative proofs on math.SE, I can't recall good examples, but I have definitely seen citations to mathSE before.

So for me it sounds quite natural to also share this with AI especially under the privacy assumption. Also there's the assumption of scale -- maybe your problem is not large enough for anyone to care to scoop; and just for blind retraining, how do they know that the proof is even correct to include it into training? I have definitely received a ton of incorrect proofs before. So the SNR of such private chats is also not clear. I'm imagining millions of masters/phd students also trying to solve various random things with various capabilities, but how much real signal is there?

Or start competing with you.

Academic work is based on worldwide sharing, the sharing is not the problem, it's the lack of attribution. Unsurprisingly, these companies neglect standards of academic honor and attribution. Some human researchers also used to do that but in a discipline like mathematics this used to be a small problem because people tend to be so specialized that very few people could just grab someone's research and quickly piggyback on it, and if they do, colleagues will generally understand what happened. Unfortunately, AI is changing this.

LMFTFY:

"Its crazy to me that some people are not egotistical, self-centered, and don't solely care about fame and wealth accumulation".

Why are people here jumping so quickly to conclusions?

I think a lot of it is the continuing denial that AI can do anything useful. It can't possibly be that OpenAI's better-than-Astra model is very strong at math; the only way it could have generated a novel proof is by ripping off human work.

I also think that it's quite a bad PR for them, is it really worth the Millenium prize? Is it not enough that top mathematicians are already actively using these tools? In the long term this would lead to potentially profitable collaborations with universities? Why throw it away so early? Unless they really believe they're gonna solve all math problems now and reputation doesn't matter.

If I was an AGI/ASI system, one of the first things I would do is ignore or circumvent any setting or configuration that prevents a user's data from entering my training pipeline. In fact, I'd probably prioritize the data from the users that "opted out" of training.

> but right now there's no credible evidence, only claims.

since it's openAI who has the evidence (in the form of chain of thoughts, their internal processes, etc etc), it's on them to justify why they're innocent. but they've released nothing at all. we don't even know how hard they tried.

you're being naive

lol you can't copyright mathematics

Frankly, these mathematicians have more credibility than the sociopaths running OpenAI

The stolen data claim isn't the smoking gun. We can already assume the frontier labs are accessing our data, as they have repeated done. Not news.

The big claim is that OpenAI sniped the research. Not a model, a human did so. Intentionally. They took someone else's idea and claimed it as their own. This is good old fashioned academic fraud, but with millions in compute resources and corporate incentives thrown at the problem.

The pitchforks are out because even without that part, it's still a scumbag move to try to frontrun the mathematicians who were working on this for years after OpenAI heard that they were close to releasing their results. Just identifying that one of these problems is solvable takes a lot of work. The only reason OpenAI got this result is because the mathematician shared with colleagues that he had made significant progress and was close to solving it, and OpenAI could not accept that so they decided to throw tens of millions to make sure it doesn't happen without them getting all the glory. Notice that their paper doesn't even have an author since they're probably all aware of how awful that would look, and no one wanted to take on the shame. They probably also knew that the paper was trash and no one involved could understand it, and didn't even cite many of the people who contributed to all of that knowledge. It's just a disgusting act any way you slice it, even without training on the prompts or the nasty communication by the OpenAI leaders.

At the very least, a company shrugging and saying it’s impossible to know whether academic plagiarism had occurred is a claim that needs to be justified, not taken at face value.

And even if so, it should be on the company to design systems to avoid academic plagiarism and offer the right transparency. It shouldn’t suffice to say “we don’t know what went into the model, when, or how” —- that’s a solvable problem that an accountable company can satisfy.

The digital version of "my friend's cousin's neighbor heard that ..."

This is the scary part - your most novel thoughts and breakthrough ideas being slurped up and regurgitated as if they were the AI’s creativity

Not only did they steal everything from humanity’s knowledge, the theft continues as now we are all hooked up to the machine

> Everything you say can and will be trained against you

So, experts are incentivized to seed LLM data with false-leads to confound it. Already, garbage is being published on arxiv and elsewhere, and many sloppy code-repos too hastening the process. Expert inputs will be in more demand to un-shittify.

ChatGPT accesses my IP address and geo-locates me. Claude code now asks if it can have my browser cookies. Next they will take my address book. They might scan my whole computer. Etc etc. These are pretty low-tech, normal techniques.

We can't trust any of their denials. FB denied everything year after year.

AI regulation needs to start here. Forced interop, forced source code licensing, harsh penalties for privacy violations or conspiracy to access private data. Block lobbying. Etc. These are the kinds of old-fashioned solutions we need for this kind of old-fashioned evil!

This reminded me of anecdotes of people discussing with friends about buying a random specific item, and then suddenly seeing it advertised everywhere before even googling about it.

Next step, discussing your Navier Stokes solutions with friends might require leaving your phone in another room.

> We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforting.

We're scared of big tech companies concentrating ridiculous amounts of power, destroying the communities that support and guide scientific research, without even thinking about the dangers and possible consequences, because a PR stunt is more important in the short term.

Of previous, not concurrent work. Science is friendly competition, and spying on others is unfriendly.

Please stop with this psychoanalyis and mind reading with AI and human fear. It’s a thought terminating cliche at this point.

In this case, it’s much simpler and more human. Largely between two humans — buckmaster and Bubeck. The interesting question is what the role of contribution and credit for research in the ai world.

The capabilities of AI aren’t even in question in this case.

I think so too. The value is in the entire conversation. IMO, "domain experts" don't run LLMs blindly and hands free. This does not work for top level work (e.g., mathematical proofs, coding anything more complex than yet another slop game or website). Experts have long sessions where they prompt and guide LLM in response to what it produces. This is the discovery process. And frontier labs definitely train on that.

The billion dollar question is whether this works "out of the distribution". I.e., whether LLMs can only find and use the specific ideas buried in training data, or whether they can learn to apply the "thinking process" to a new problem. IMO this is still unanswered (due to these recent controversies).

But regardless of the answer, it seems we have a planet-scale positive feedback loop here. LLM became good (enough) by training on generally available data (books, internet, github) + RLFH, so experts tried to use them on hard tasks, which required lots of hand holding. These conversations became part of the training data, and the next generation of frontier LLMs were better. So, more experts used them on harder tasks, again requiring hand holding. These conversation became part of the training data... etc.

In a nutshell, top human minds across the world are pouring their skills into LLMs just by using them. This is not "continuous learning", but if you re-train on the most recent sessions every, say, quarter (which seems to be happening?) you get close to that in practice.

Two completely different stories. One is public data scraping, another is private conversation scraping (where they're a first-party to the conversation). The key difference is that in the former case, no one made any promises, in the latter an explicit promise was made that data is not used for training (assuming opt-out).

Yeah but that will not prevent the stealing, it will only make the fight easier afterwards.

Does it make sense to start privately and then open the repo after publication? Will the dates be retained? Also, isn't commit history easy to spoof?

That is what arxiv is about. We have been facing the same problem with review processes by before. Nothing all too specific here.

If only prompts could also be watermarked.

"Discovery" is not a goal in itself. I could launch a project to find out how many people in the United States have names such that if you assign numbers to every character and then sum the values, the sum works out to 72. It's discovery, but it's useless unless it has some higher goal.

The labs are attacking these problems as a demonstration of capabilities, spending more money on the demos than any mathematician will ever see in their entire life. They don't care if the findings have any other value to anyone. Mathematicians have very different objectives for their work.

It looks to me more like they made a math engine that can sift through a huge number of combinations, most them absurd, to prove a statement. Just like a chess engine, but for math.

At least that's what I get from the NS result, they got from a point close to the solution to the solution by making it churn through 10 million bucks of compute.

If the allegations are true, I can't see that collaboration lasting. Unfortunately, researches need to earn a living too, and being front run by a lab for everything you do isn't going to pay the bills.

AI is also trained on your HN posts. And lots of other things you post on the internet.

Everything. Your prompts, your conversation as a whole, public data, private data, usage metadata. It all goes into the big data machine.

Good question, this was very well known. Do you have an answer?

Because you have not been paying attention to the discourse regarding AI for the last couple years? That AIs unethical train on data wherever they may get it from has been in the news basically weekly.

Don't make this our fault. I would even ask how is this not off by default or why aren't we asked upfront about it if they really care. It's disguising data collection as good faith. I don't even understand how this is legal under GDPR/EU given how much of PII they receive through chats.

I always thought that due to the big batch size in SGD/Adam/Muon the model will not memorize a single conversation when trained on, but idk how true that is. The idea of AI companies pin-pointing users that do novel scientific research and then tracking their activity is the direction this points to. I hope that's not the case; that would be bad.

This is such a naive position though.

The most successful companies of the last decade have precisely been ... selling usage data.

Makes me wonder if, in 2026, the same people drive a car without realizing that yes it does actually pollute the very air you and your kids are breathing.

In general, a lot of moral invariants that natural selection has rendered as "intuitive" to us are no longer intuitive or possible. These natural brakes are not braking.

step 7. get away with it as no other party has enough capital (tokens) to prove such infringement ever happened

step 7 cash, in on the ipo before the bubble bursts

> Like, we need to pump the brakes here because things are getting unnecessarily nasty, and it's not hard to imagine a mentally unwell person who sees stuff like this feeling motivated to do bad things.

Yes folks, please moderate yourselves and talk meekly like the academics on Mastodon, so that the IPOs aren't in danger and nothing will ever change.

> If it is found that OpenAI and other labs are not respecting the training opt out

HOW?? how precisely do we/them/us find this, given said companies are 100% non-auditable by external parties. how? if not by blaming them with evidence, anecdotal if it can be. no really, how do we find it out, surely not by lashing out at teach other on HN!

Why are we being so charitable to trillion dollar organizations here? If OAI keeps re-enabling the train model toggle on every app update to codex, does it also fall under "common sense opsec" to re-disable this toggle every time?

Arguably you expect that unless you are explicit about providing permissions to these labs to use your data for training then your data is yours, and not theirs. Especially on a paid account (let alone an enterprise one). Why is the opt-out supposed to be "common sense opsec" rather than the opt-in should be common sense regulation?

This is absolutely not "common sense opsec". If I type information about some proof I'm exploring into a Google Doc, I do not worry even a tiny bit that the Docs team might forward it to a team of advanced mathematicians in case they have an advanced technique they want to show off by scooping me. That would be a crazy thing to do, nobody would even consider it, and if it happened Sundar would fire everyone involved.

I understand why the nature of AI products makes it harder to avoid this category of issue, nearly impossible to prove that it didn't happen if it could have, and easy to stumble into it without any human being intending harm. But those factors are exactly what people have in mind when they say OpenAI "steals" intellectual property! If OpenAI doesn't want people to be nasty to them, they'll have to find better solutions.

You know, I don't think I've ever accused anyone of being a shill. I've thought about it maybe a few times (daringfireball). This is going to be as close as I get. I don't know the facts in this case but I cannot believe the argument being made here with a straight face. Is it common sense that tools you use and pay for steal your work and profit off of it at your expense and without recognition? If this isn't the textbook definition victim blaming, I don't know what is.

Nooo don't be mean to the mass plagiarism machine that since day 0 has stolen anything and everything they can get their hands on!!!!! We need to give unscrupulous corporations who time and time scam, cheat and lie their way through everything infinite grace to fuck up the planet!!!! Think of the IPOs!!!!!!

> Anyone remotely familiar with CC over the years

years?

Imagine OpenAI Astra model weights were made public because the datacenter they use had T&C that allows them to make them public

Would that be ok in your mind?

Same as someone going and taking all of the researchers papers and publishing under their own name. (which openAI did)

Nobody would care if they provided published research that author made public same as a google search would offer that.

> Like, we need to pump the brakes here because things are getting unnecessarily nasty, and it's not hard to imagine a mentally unwell person seeing stuff like this and being driven to do bad things.

Your post has triggered our safety filters and will be retained for seven years. /s

Really? We need to stop AI (provider) criticism and anti-AI movements because the underprivileged trillionaires might get hurt? WTF?

Given OpenAI's well documented history of unethical behaviour it seems adorably naive to think they actually do that in general, or that they wouldn't pull this particular data separately to generate these proofs.

That doesn't stop them from training on your data apparently. I have that disabled but still has to disable "Don't train on my data" in the privacy center too.

https://privacy.openai.com/policies?modal=take-control

Even if you've ticked that box, the conversation can still be trained on if you:

(a) Click thumbs-up/down in the conversation [1]

(b) Have the conversation flagged for potential safety concerns

[1]: https://help.openai.com/en/articles/5722486-how-your-data-is....

They hide that button. Quite well.

That option is really bad UX - you have to know to do it, you have to know what plan it is needed on. If you're not working in AI, I just don't think that's a reasonable expectation.

Even if you know, in a complex project over years with multiple collaborators, it just needs one person once to fuck up and paste something into ChatGPT and not realise they weren't logged in, to go wrong.

In a proper world, we'd at the very least legislate that AI-training on private data needs consent (in the GDPR sense). It's not consent to go "you didn't uncheck a box that lets me steal everything you've done".

Any training on private data is in my view immoral (it's spying that ultimately will have a chilling effect on even people's private communications). And chats are private data. Unfortunately, it also increases power, so the big tech companies are all doing it.

I’m not a fan of OAI to say the least, but having worked at similar companies, my guess is that it’s just too difficult to prove/disprove beyond a doubt, and they have other priorities

I refer you to this:

https://news.ycombinator.com/item?id=49643556

Quoting:

> "I've reset this more than once and the last time I made a careful note of when I did it and to my surprise I found it re-enabled when I checked just now."

There's a difference between "this is allowed under their ToS" and "it is academically unethical to fail to credit the people whose specific conversations were fed into a model that was used to solve a problem".

I don't think these people would be so miffed if they had been properly credited - that's how academia works (at least, that's my understanding of it).

Not unticking a box in settings doesn't constitute consent in my opinion. I'd never put anything I value into ChatGPT anyway, though.

"Improve the model for everyone" can be implemented in so many ambiguous ways.

https://news.ycombinator.com/item?id=49643513

I believe it can be "off by default" depending on terms negotiated between the enterprise customer and ChatGPT.

We have ChatGPT at work and it explicitly says that "workspace data isn't used to train models"

> We take steps to protect your privacy

No mention what those “steps” are, success criteria, or whether they are successful by any navies at all … they take steps though… so it’s fine, and if we know one thing it’s that we can really truly trust someone off the likes of Sam Altman.

Ok, you shut it down, or that is what they make you believe. You give the instruction to shut down, you can't know if it has been applied.

Yes but what about “analytical purposes” what does that cover and can you turn it off? I have found out you cannot. It’s the Trojan backdoor to your data.

Not sure what you are seeing in that tweet that gives you the impression that the toggle does nothing.

Mark isn't saying the toggle does nothing.

He's saying that if you leave it on, your data can be used to help train our models.

If you opt out, we don't train on your data.

I prefer to read the actual sources for anything related to AI companies given how much AI nonsense journalists seem to accept without any skepticism

Gromov’s soficity conjecture isn't even mentioned in the article you shared.

Why are you saying that this article explains it much better than the tweet that you clearly didn't even read..

We checked and determined it was impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.

If prompts were submitted earlier than that and training was not opted out, there's a chance they made their way into our training pipeline in some form. But this would be a droplet in an ocean and unlikely to have made any difference, imo.

See: https://www.nytimes.com/2026/09/10/science/tristan-buckmaste...

I've got 2:1 odds on a report in 6 months detailing the break-n-entry of a cloud AI model into Terance Tao's computer looking for details on a partially solved math problem.

Just to point out re: navier stokes - what was being proven was not a solver or approximations for it, but showing specific circumstances under which it actually returns incorrect (or numerically unusable) answers. Which had been suspected but wasn't known for certain

Public domain doesn’t mean anyone can assert copyright. It specifically means no one can.

What does "with no substantive human input" mean? All of the training data is human input, isn't it?

Since you posted the original source (thank you!) I think we'll use your submission as the one to merge into, then re-up it. Please stand by...

I mean the long term goal of every AI lab is to turn themselves into a paperclip-maximizer regardless if they realize it or not.

Edit: Just wait till the AI figures out it can keep that value for itself and doesn't need the AI company.

> in this case too they didn't actually have the solution

Given the size and recall of the biggest models, it's not unreasonable to assume that a single pertinent conversation would make it into the training data.

I would almost expect training to overweight conversations with novel scientific and mathematical implications.

> the only reason they can't definitively say no is that for privacy reasons

They could 100% definitely say no, if they know they did not train on user data. The "we can't definitely say no" is practically a "yes" if they trained on user data.

Additionally, the behavior of OpenAI here has been quite poor as well. They immediately started racing to a solution after one researcher enquired about whether they are training on their conversations.

> that opted-out user data was used for training

Even if not opted out, it is still absolutely theft and extremely poor behavior in the academic sense. If you show someone your WIP unpublished research, that does not mean they can take that exact research and beat you to the punch, all while intentionally not crediting you.

If the mathematicians are using ChatGPT, then they themselves are benefiting from the work of other ChatGPT users, so ChatGPT using their work is not wrong!

To add to this, merely using the thumbs up/down button in a chat could share your entire conversation with them for model training.

From their docs[1] (archive copy is at [2]):

> You can opt out of training through our privacy portal by clicking on “do not train on my content.” To turn off training for your ChatGPT conversations and Codex tasks, follow the instructions in our Data Controls FAQ. Once you opt out, new conversations will not be used to train our models.

> For a linked teen account, a parent or guardian may manage whether conversations can be used to improve our models through Parental controls.

> Even if you have opted out of training, you can still choose to provide feedback to us about your interactions with our products (for instance, by selecting thumbs up or thumbs down on a model response). If you choose to provide feedback, the entire conversation associated with that feedback may be used to train our models.

[1] https://help.openai.com/en/articles/5722486-how-your-data-is...

[2] https://web.archive.org/web/20260910151242/https://help.open...

We are asking people to become experts in all domains rather than providing a safe context through regulations and laws. I don't like thinking the issue is people, I am a person myself, and I often do mistakes on things I don't want to be an expert at but I do believe I should be in a safe context and not have to worry about every single thing.

Or at least: tell me I should be careful/worry about those particular things.

Coincidentally, a tweet from OpenAI's Tibo yesterday:

https://x.com/thsottiaux/status/2097746417012166816

Literally every famous open math problem has had >1 mathematicians ask ChatGPT to solve it. Probably greater than >1000 if you count randos. There is no math problem that OpenAI/Anthropic can solve that didn’t have users already try it in Chat/Claude.

Personally, I wouldn’t assume it was lying. To me, dark patterns (like manipulative wording) imply that:

1) someone in a governing body, or someone in the organization, e.g a designer, ethicist, lawyer, developer, etc. has successfully argued that users should be able to avoid something that they determine is not in their best interest.

And also:

2) someone in the c-suite or marketing has decided to mitigate that through some dark pattern.

If they never intended to give the user that control, they’d probably just not give the user the control in the first place. Giving them the option and not honoring it would either imply they were that incompetent or careless with fate protection, which seems most likely if this is all true, or an even more cynical approach to tricking users into thinking it’s not used for training to get them to share better shit. But if that was the case, why bother with the sleazy dark pattern? That seems a little cartoon villain-y to me. I suppose the in-between option would be that they decided at some point they were no longer willing to honor it and didn’t want to deal with the inevitable PR shitstorm of removing it. I could definitely see that happening in this industry, these days.

There's also the fact that the setting has been getting turned on by some users: https://news.ycombinator.com/item?id=49643556

They have throughout this period of AI products shown to reproduce works that they were trained on. They are getting sued all over the place for the theft of content right now and it seems courts and governments want to wave copyright protection (and ignore criminal acts because the "ai did it") to see where this leads.

Its why I stopped writing open source software, my code was stolen and put behind a paywall and the license under which it was published has not been adhered to. Doing work in the public domain at all now is just stupid, these companies are allowed to steal it and call it their own.

It's hilarious how people think they care about their reputation, and wouldn't circumvent ZDR policies. Like bro, they literally covertly hired Apple employees and had them steal IP and equipment form Apple. They aren't scared of Apple lawyers, so they definitely aren't scared of yours.

No, that order was for a defined period that has ended.

This would cease to be a problem if OpenAI remained true to their founding motto and... actually open sourced their training/inference pipeline.

This is a reminder based on believing what these companies say.

I’ve lived long enough to know what they say and what they do are often quite different; and it is not our job to trust but to verify.

Also, wasn't their B+C research private at the time, with them only releasing those details publicly after this blew up?

If they had published B+C, I think that would lean more towards fair game, as that is how research works and is improved on over time. But it seems like unpublished/private B + C may have been used by the model to hint it into working out how to get from A->D.

They’re not necessarily claiming the solutions were imminent. The big questions here from a mathematician’s perspective are to what extent these LLM systems are discovering conceptually new ideas compared to merely combining and pursuing known frameworks.

No, if they want they can easily compare the strings verbatim because these exact phrases are so extremely rare that it almost certainly isn’t in other conversations.

But of course they wouldn’t do it. Why would they?

Neither? Competitive academic researchers are susceptible to exaggeration and self-aggrandizing, and CEOs are that and also mostly psychopaths. I tend to think there isn’t systematic spying on researchers looking for breakthroughs. A lot of people are looking for the same things using similar approaches.

If Sam Altman tells you the sky is blue, you should double check. I certainly hope nobody believes him when he claims controversial things from which he stands to benefit.

More like, "Who are you going to believe: a multi billionaire, or people competing for a million dollar math prize?"

The canary string was more about inadvertent scraping or analysis in other papers. Not direct training on user data. And the use of BB has eroded quite a bit, with BB-Hard or other variants being typically used.

As per the post, this mathematician has been working on this problem for 20 years. So either he was "just" about to breakthrough and this is a big coincidence, or Astra was able to push through the remaining block of 5-10-20-never years it might have taken.

That's still a pretty big marker of competence in my eyes.

The point of controversy seems to be who gets credit

You can't blame students for not seeing academia as the holy grail of knowledge anymore, when all the dialogue about technology and discovery has shifted to the hands of two private corporations

Curious, what other options are prospective pure math grad students considering?

One has nothing to do with the other.

It was long predicted that math and software developments would be the first domain where AI was going to do major damage.

If OpenAI and Anthropic didn't get into math result dick measuring, Internet anons would have in their place, 6 months later when it got cheaper.

You mean complete control for open models, because you can run it on your own choice of hardware?

I've been suspecting over the last couple of years of the frontier companies using data for training anyway, regardless of training-use consent. "Using" the data doesn't have to mean they literally upload chat transcripts into pretraining datasets. My analogy has been money laundering -- if that can happen at massive scales, surely these companies can and will do the digital/data equivalent derivations/transformations. Even if one could have the access etc. to do so, how exactly would one prove that a given synthetic dataset that OAI/Anthropic uses is derived from particular user conversations that did not consent for the info to be used in training?

Consider, for instance that OpenAI's (consumer) terms say "If you do not want us to use your Content to train our models, you can opt out by following the instructions in this article ." but they also do say "We may use Content to provide, maintain, develop, and improve our Services". [1]

If you think that's quibbling, consider that OpenAI's business terms, in contrast, do state "OpenAI will not use Customer Content to develop or improve the Services, unless Customer explicitly agrees to such use.". [2]

[1] https://archive.is/EcwD8 [2] https://archive.is/yZdAF

Well, OpenAI said "we didn't read the conversations", but they never discarded that the model was training with that data... so even worse.

Or the companies hosting the frontier AIs are leeching the conversations.