This is also true for almost every other field, even within computer science. The only difference is that a lot of people operate at a very surface level without realizing just how much background knowledge they have accumulated. Think about the number of keywords your average SWE is expected to know. It is rather insane.

Cache, stack, heap, process, thread, socket, file, tcp, http, tls, websocks, socks, soc2???, deadlock, stack, queue, race, atomic, event loop, coroutine, async, database, transaction, index, replication, sharding, consistency, serialization, DNS, load balancer, container, namespace, and so on.

Every sub fields (web/kernel/backend/etc.) has a million/bazillion weird words used in a dozen different contexts and if you read a paragraph of even semi technical software text you will feel like an over stuffed turkey.

Even cache could mean the CPU caches, the page cache, a browser cache, a CDN cache, a Redis cache, or imagine the flurry of words we have that have real world meaning. Session, handle, pool, buffer, stream, channel, event, task, worker, or queue. Generally there is some overlapping meaning but often there isn't.

I had a few moments of this in the past. For example, in my quantum class the teacher wrote "H Psi = E Psi" on the board, we all laughed, "just cancel the psi" but it turns out one was a multiplcation and the other was a matrix multiplication (operator) and so we had to learn all new nomenclature.

Similarly, at some point somebody pointed out to me "the reason you're confused is that the bold on that variable means it's a matrix"

A term that gets tossed around in math is "mathematical maturity." It's similar to what you see in other fields - e.g. learning how to program, learning how to make music, learning how to cook - that involves many "aha" moments and reshapes your perspective. Math is full of such steps, moreso than most other endeavors, probably because the main limit is the abstract reasoning itself.

The abstraction is by necessity. Our puny brains have only a very small working memory. The only way we can reason about many problems is by creating multiple levels of hierarchy. That is actually the essence of what mathematics is.

Is there something that translates math formula into code? There are many (comparatively) "simple" algorithms I simply cannot make heads or tails when they are described via math prose or math formula, but when it's code I basically instantly know how to rewrite it into any other language I know, at least, and sometimes that's a starting point for poking at it to understand it a bit better, if not the same way would if you understood all the underlying math.

For poor old me, too many wikipedia articles on algorithms useful mostly or only for programming are described in formulas rather than simply code with detailed comments. Scrap the whole page and just gimme the code :( Not even to copy and paste, because that's a language I can understand, and enjoy learning.

I agree. I think the distance in capabilities between great mathematicians can be so much more vast than other fields as well. Some of them need every step to be derived, while others can skip ten in their head.

Math strives to minimize ambiguity, which other fields don't do as much. Non-math fields tend to reuse regular words as jargon (i.e. with specificity of meaning that may fly over the laymen's heads). Social sciences and humanities are most notorious for this, often resulting in non-practitioners not realizing they are out of their depth because they are not looking at symbols from non-Roman alphabets.

Yeah it is a lot of simple ideas stacked one on top of the other, but the edifice is so large from some vantages that the building blocks aren't visible, or tractable to think about independently. And sometimes the ideas are very subtle, so you can only develop fluency partly by spending lots of time playing with those blocks by building your own little structures. You also develop fluency by talking to other mathematicians

I like to emphasize that the ideas are usually very simple at their core. Sometimes they map to kinds of objects or reasoning that non-mathematicians use implicitly all the time in their daily lives, mathematicians just have words for them and so are able to use them explicitly.

And I suspect the density of the language/terminology may give the wrong impression about how mathematicians think about the math they are working on. I mean, different people think / experience / practice math differently of course but IME the underlying thought about a particular problem tends to be much looser and concrete than formal math writing would imply.

That more formal language is needed of course because at the end of the day, it is how we communicate our thoughts in the way that other mathematicians can understand them, not to mention how we can check our own thinking

I once didn't understand the task given in an exercise sheet for a CS logic lecture, so I googled the topic, and all Google did was send me back to that exact exercise sheet.

I agree, the nomenclature is impenetrable, it's like reading software that is not well commented. Perhaps LLMs are very good at "challenging" mathematics because what we perceive as challenging is primarily the language component and not the conceptualization.

Yes, the nomenclature in math is atrocious. It isn’t much better in physics or biology however. As a species, we suck at naming and classification, and keep starting new trends atop old ones. There is certainly need for the many abstractions of math to be as complex and well specified as they are. There’s no reason for their nomenclature to be so bad. The end result is a substantial portion of the population, which can certainly hold and manipulate abstractions, fails to even contend with pure math. I do think visualization tools will help in the future, to demystify some of this. But as with all sciences, the need for personal glory/mentor deification often conflicts with broader explainability.

Isn't it just what you studied in depth? I am not in this area but can understand what's going on "at a high level" here. But I studied no other science formally since the age of 15 (this is possible in the UK school system). So physics and biology just go over my head unless they are sufficiently mathematical.

But somehow the conversation is still enthralling. Just seeing the first few words of each response gives you a feel for the level they’re on.

Yes, math is hard.

People in their second year of graduate school only get to about the early 20th century in terms of understanding. Third year is getting to about the mid-century. Fourth and fifth years get kind of to modern times but with increasingly smaller breadth.

It can't be one language, and that's the big problem. It's inescapably a bunch of tiny DSLs. Once you see both the inconsistency and the necessity for inconsistency, it becomes much easier to just roll with it.

IMO, it's just the notation. Something I've actually found ChatGPT useful for is to create mathematics lessons for me in the form of computer programs. When broken down into a series of readable almost-plain-English steps, it's so much easier to understand. And it's easy to tinker with programs and get a hands-on feel for things quickly.

I'm sure having a compact notation is absolutely invaluable for people who dedicate their lives to maths, but for someone with just a passing interest, I find it more obscuring than helpful. I feel the same way about music notation.

This is a very notorious area for dense definitions and concepts that interrelate closely and have to be memorized. Mathematicians from other areas are going to have difficulty but may have some idea of what the concepts try to capture.

Some areas are hard in different ways. I could never quite wrap my head around the way logicians have to think. A clever combinatorial bijection is a work of art you probably can explain to a undergrad class easily but good luck coming up with it. And number theorists will throw the kitchen sink at their problems: no area of math is safe from getting used by them.

People who do this have spent years of their life thinking in this language and studying it, so it is going to be hard. We're also not good at communicating the intuition which for algebraic geometry often comes from other fields.

That is one of the things that fascinates me most about mathematics compared with other fields, and it led me to discuss the subject with professional mathematicians. The funny thing is that they admitted it is the same for them...stray even slightly outside their own specialized area, and within two or three lemmas, they also feel completely lost.

Mathematicians emphasize definitions not labels/names.

It doesn't matter if natural numbers include 0 or not, what matters is how you define them, not how you call them.

This makes them also bad at naming things because...there's a definition anyway.

Most other fields do not have or can't have the same luxury, so naming might be more thoughtful.

You develop a set of heuristics for skimming it in the same way you do with code.

Big wrapping operations like sums, integrals, and matrices, then what's nearby them, give you a very good idea of where things are going context wise.

Progress in scientific fields is limited by how fast you can perform experiments. Most areas of math are limited only by the number of practitioners.

It's just language. Mathematicians don't invent notation for fun, they do it because they naturally start thinking at a higher level of abstraction. If you're not thinking at that level then, well, it will be all Greek to you.

Yes exactly

Math isn’t necessarily hard, but it’s incredibly dense

A simple statement like let f(x) be a continuous function can carry a lot of definitions

In that statement, if you missed the day in class where they covered continuous functions it might not even register that it’s a well defined term

And that’s the most over simplistic example I could think of

As a math major, I remember that being one of the first lessons I learned, that every single word could be carrying a lot of weight so to look things up in detail if I was ever struggling on a problem. One of the oldest entries in my memory.md file

> just repeatedly saying "keep going" to ChatGPT

For posterity, this indeed works for most problems where an agent might give up. LLMs don't inherently know something is impossible.

The phrase I tend to use in my harder prompts to automate this with a sane loop breaker:

> **REPEAT THIS PROCESS UNTIL CONVERGENCE AND YOU ARE OUT OF OPTIMIZATION IDEAS.** You have permission to keep iterating.

Without any more context, "keep going" seems to be doing a lot of work. The user is placing a lot of faith in the LLM to not make subtle logic mistakes and to take good approaches to each problem. In my experience, even frontier models (such as Fable) are quite capable of getting confused during even simple technical work I've done in the dev ops world. For example:

LLM: This package hasn't made it to production.

ME: are you sure? i see it right here!

LLM: You're right to push back. I inferred that based on weak data. I see now that the package has been deployed!

If the above conversation is typical for me, how could one expect to achieve a sound result by repeatedly prompting an LLM to simply "keep going" in dense mathematical proofs? Perhaps the user in this case had actually checked the LLM's work before issuing the prompt, but I think you see my point anyway.

“Keep going” is exactly how many mathematicians achieved success in the past. :)

Is this the same as Dinitz Theorem[1] which seems to have been proved in 1994? This is the only result I keep stumbling upon when trying to understand the problem formulation

[1]: https://en.wikipedia.org/wiki/Dinitz_theorem

"it's enough of partial results. let's finish with a complete unconditional counterexample"

"Worked for 88m 24s... >"

"<h1>Complete finite counterexample</h1>"

...

Direct link: https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...

crosses fingers "Low hanging fruit, low hanging fruit, low hanging fruit..." hyper-ventilates

From the prompt:

> You should do a breakthrough

This is just as funny and ridiculous as those "make no mistake" prompts.

"whats next" is another good one

"Gew on, lad!"

this sounds like like an open parenthesis (

without someone independently verifying it, it just dangles there

...

It reinforces how to "learn AI" is to first master the problem domain.

I can use AI for coding after decades of coding. I can't use it for theoretical physics because I can't evaluate the responses.

  > I’m also surprised to see that even Terrence Tao seems to use it in a way that resembles, in progression, how I use llms in my area of expertise

I've mostly seen people trying to oneshot a result, while I'll quickly experienced that going through steps/discovery was more effective and more satisfying, since you can always steer it back in the right direction; while oneshotting is hit (and it kind feel like magic) or miss (and you'll have to rework your prompt).

I encountered something fairly similar working with Claude a few days ago. For a current project I've been fairly hand-wavy with requirements since I was getting good results, but it seemed to be failing hard on some key points, so I started to be more strict with it. Even after the fails were resolved, I've noticed that Claude now behaves differently within that project, carefully checking and rechecking things up front and also looking to me for guidance more often. Mildly irritating, but if it works...

It is ~a meme on subreddits that developers struggling to get good results out of any given model is a "skills issue."

But I think your comment drives at some authentic take on this. Skill with AI is not only crafting iterative prompts the agent will understand, but also very high domain-specific knowledge of what the prompts explore.

One without the other can result in frustration or worse.

I think we have a couple of years of "being good at talking to the LLM about your field of expertise" being a useful human skill, until that too gets washed away

A Minimizer addicted to upvotes.

A community of those who distract themselves from the perfectly fixable problems in their lives corruptly self-evaluates. They validate each other's stagnation and unwillingness to move by finding flaws in each day that will enable shutting off the flow of any new data while condemning the world and any actions in it. The lay-z-boy they collectively protect appears as corroboration with a broad population but is in reality a repetition whose independence is meaningless since they are all copies of one system, one kind of person in the same kind of trap.

Immobile. Clogging the Suez with their sandbagging ways. Nothing to add except reasons to stay put. No aspiration. Only cynicism. They deserve nothing but all of our contempt.

> you can use AI to understand something and map it to your own mental map

This "symbiosis" (for lack of better word) of human with AI seems to be an emergent value proposition of AI. In the process of doing stuff with AI, producing artefacts like code diffs, we are continuously able to decide how strong the mental map is of the current stage of the production process.

I could probably have worded this better but I'm sure it's something others have noticed... this choice we are able to make of how high fidelity our own understanding needs to be of the current working problem, and how that choice never really existed prior to AI.

Hold on isn't this how math professors used grad students for hundreds of years?

The "yes, with caveats" thing is boilerplate for both Codex and Claude since this current generation.

It's actually a bit annoying because it primes you to think that the caveats are real, but most of the time it's just something terribly obvious and not a real caveat, but the model probably has some system prompt that tells it to always consider caveats or something like that.

Same as the model starting every reply with a commitment to be "honest". LLMism are fun but I tend to just suppress them via AGENTS.md because they distract me

>Maybe a year - or two model releases - from now, the AI assistant will be undeniably stronger than Tao, and not an equal anymore.

we're kind of well past that (in my opinion), if you consider that this is the same ai assistant that can help you with a recipe, diagnose a weird sound in your car, help with biology homework, translate languages, and so on.

even in math alone, i think its indisputably already stronger than Tao, considering it has approximately this much depth in ~all of the math subfields.

But what about the later "Repeat previous question" prompts???

I just do very laconic questions about advanced topics, this seems to prompt it a bit more towards reducing fluff in the answers. But that + the activated pro could be an improvement

Came here to flag the same beat. It's wild to me Terrance Tao has to pay to talk to chatgpt, you would think it would be the other way around!

I do the same kind of thing. "You're smarter now, time to try to cut down dumbo ChatGPT".

High IQ bros.

> Where will we be in another 4 years?

Possibly somewhere amazing, but see also: https://x.com/pronounced_kyle/status/1768852493092680036

I'm from the UK. What's in the image?

LLMs can't reason, ok? They just repeat basic text, LLMs will be never smart in 100 years!

/s

I think the "But this is not intelligence because it is known math" is not a correct argument. It is unknown how the overall higher intelligence of humans works.

What I do notice however is that LLMs are becoming capable of doing an increasing part of the intellectual work I can do, and usually a lot faster.

Just today I presented an agent framework that can take an informal incident statement and propose infrastructure changes to fix it, all evidence backed. This did nothing I could not to, but it did all 5 test cases in 6 - 12 minutes each. I would have found all of the monitoring indications it did, but it would have taken me a day per test case. The LLM also included sass to silly tickets. ("This is not even worth spending monitoring resources on. It's obviously a configuration problem.")

That's how this is reading to me as well. It's just fast at slogging through a certain level of "simple" transformations.

That argument says very little, emergent behavior is a thing in complex systems with billions of parts. Humans can also be reduced to voltage potentials propagating along of tubes of fat and synapses getting rewired.

What does "predicting the next token" mean? I ask this every time people say "LLMs are just predicting the next token" and it's maddening that nobody can give a straight answer. Predicting it according to what probability distribution? Every process that produces a sequence of actions (including e.g. a human writing) can be modeled by some probability distribution and therefore their actions are indistinguishable from "predicting the next token" emitted by that distribution.

> there’s no “intelligence” there BUT

There is clearly intelligence there. We have no way to recognise intelligence other than the appearance of intelligence and this very clearly displays that.

It's also quite clearly different to human intelligence in some notable ways, but not in any that preclude describing it as intelligent. At least for normal non-pedantic definitions of the word.

But a property of intelligence is to know when to stop, if we treat intelligence as some sort of search and not some a priori intuition of the entire space. Seems kind of hard, if not impossible, to train for specifically that.

I wonder whether when the fact that AIs have started solving conjectures will enter the training data, they will become more confident in their abilities.

GPT 5.6 already is an explainer agent.

Fork Tao’s convo and prompt this (with your own math level described).

GPT did a great job of translating Tao’s questions and concepts (e.g. “pre image”) into a progression I could understand.

“Ok I have a PhD in financial math and undergrad in engineering math. I have almost zero knowledge of polynomial algebra / geometry, I know what a polynomial is and what roots are but not much beyond that. Could you try and explain to my level what questions the user I the conversation has asked and what the agent has responded with, we can probably go user query by user query to build up”

I'm not sure an AI will speed things up much. You would probably still need years of layers of foundational understanding to get the advanced material. We don't go through years of school to learn math just because teachers are bad - it's because complex subtle ideas are built on countless other ideas, and aren't necessarily compressible to something every layman can understand.

probably like a minute or two? im pretty sure someone unironically said stochastic parrot on the HN post with the tweet announcing the counterexample, and ive read several comments with similar sentiments today (including in this thread)

People are insecure about their leetcode black belts and react slop not giving them cushy jobs anymore so keep missing the forest for the tree.

I was a chemistry researcher this is true in all science.

I'm not sure LLMs can transform this, the incentive is to get more results in your nich, jumping topics don't help unless you have genuine interest and reason to.

You are badly informed the counter-example was found shortly before. Terrence just tried to replicate how it was found.

Maybe true, I'm not sure, but this isn't the conversation where the counterexample was found.

> I don't think chatgpt could have come to this on its own without the amount of steering he did, which just validates the idea that AI is not a replacement for human expertise but an amplifier.

The problem is that, what happens to human expertise as people start to use AI earlier and earlier in their careers, so that in 50 years? The problem is that Terry Tao spent decades as a mathematician before ever encoutering AI. Of course he and people his age will be able to drive AI somewhat sanely and use it to their advantage.

But as more people grow up with AI, they will likely not reach levels like Terry Tao because their exposure to AI and the temptation to use it will certainly dull raw human intellect over time.

This isn't the chat that made the actual discovery, this is just a chat explain it to Tao

Main discussion: https://news.ycombinator.com/item?id=48998362 Probably you shoud read the fist few paragraphs of that post, but it escalates quickly.

I'll cherry pick a few comment I like. I think they are worth reading but I'll quote a highlight of each one.

From kingstnap https://news.ycombinator.com/item?id=49000867

> If the Jacobian is a nonzero constant everywhere this means that nowhere does the the function flatten out. [...] What was conjectured is that this local invertibility property everywhere would mean global invertibility.

From mswphd https://news.ycombinator.com/item?id=48999959

> it doesn't overturn much. [...] the resolution of this is a "surprise" in that it is a very long open with many failed proof attempts. But the direction it resolved was not surprising.

Can I quote myself? https://news.ycombinator.com/item?id=49007165

> From a comment by j2kun https://news.ycombinator.com/item?id=49000833 , someone asked Fable and there was an almost counterexample in 2d but it uses division too. [Instead of f=x^2+7xy they have something like f=x^2+7x/y so it's not a polynomial.] It looks like the new trick was to use a third variable to avoid the division.

Presumably this entire conversation is at least somewhat typical of the way high level mathematicians talk to each other in ego-less fashion? Although, it would be interesting to what extent the model was trained on math conversation as opposed to just analysis and proofs.

The flow of the whole conversation, with Tao guiding and the model calculating, gave me the feel of Tao perhaps talking to himself - just that each of those model responses would have taken him much longer to calculate by hand.

It would be fascinating to hear Tao talk about what he may have learnt from this, and if it suggests approaches to other problems he might not have considered, as well as an analysis of the original Fable counter-example construction.

Yes.

Obsidian Web Clipper has a nice reader mode that works for ChatGPT transcripts (disclaimer: I made it)

It’s crazy how much these companies invest in their models but when it comes to UX they do fuckall

To me, this shows that extremely talented and qualified mathematicians (can) use frontier-level LLMs to automate their personal grind-y workloads that would otherwise (probably) take more time to accomplish with natural intelligence.

By itself, no consequence. But over time, provided we keep pumping out talented and qualified mathematicians and keep subsidizing costs, we could maybe hit a breakthrough... somewhere... that has real impact.

It's an indicator of AI progress. The solutions aren't especially revolutionary, but no person had been able to solve them after decades of collective attempts.

Practically, from this specific one? Nothing, it's very much a math thing. It's like art or music at this level. Are there consequences to a van Gogh?

"Hey Fable, please generate me the next 1000 undiscovered bitcoin hashes"

Or, "Oh just give it here, let me do it."

I expect Centaur Field Medals to be awarded. It's not clear how much steering Levent Alpöge did to get the result. In math it's very common to hide the search pattern and show only a simplified path to reach the final result.

I think that's partly because it's a fairly long conversation and partly because of all the LaTeX. There's enough formulas there to fill a small text book, and I doubt their renderer is all that efficient.

I've found it's had unacceptable scroll performance in long contexts for a while now.

In this case, it takes me 12 seconds to see content when first opening the link, and about 18 to re-render content when scrolling.

"You are an expert software engineer with ten years of experience. How do I center a div?"

Antirez spends weeks planning with multiple models till he has a clear architecture and understood the constraints before he writes a line of code.

Ask your LLM to hack it.

https://archive.is/z1vmx