Sadly I don’t see how our current social paradigm works for this. There is no history of any sort of long planning like this or long term loyalty (either direction) with employees and employers for this sort of journeyman guild style training. AI execs are basically racing, hoping we won’t need a Schwartz before they are all gone. But what incentives are in place to high a college grad, have them work without llms for a decade and then give them the tools to accelerate their work?

I've been using ChatGPT to re-bootstrap my coding hobby. After the initial honeymoon wore off, I realized I was staring down the barrel of a dilemma. If I use AI to "just handle" the parts of the system I don't want to understand, I invariably end up in a situation where I gotta throw a whole bunch of work out. But I can't supervise without an understanding of what it's supposed to be doing, and if I knew what it was supposed to be doing, I could just do it myself.

So I settled on very incremental work. It's annoying cutting and pasting code blocks into the web interface while I'm working on my interface to Neovim, spent a whole day realizing I can't trust it to instrument neovim and don't want to learn enough lua to manage it. (I moved onto neovim from Emacs because I don't like elisp and gpt is even worse at working on my emacs setup than neovim, the end goal is my own editor in ruby but gpt damn sure can't understand that atm) But at least I'm pushing a real flywheel and not the brooms from Fantasia.

> Which means we need people like Alice! We have to make space for people like Alice, and find a way to promote her over Bob

The solution is relatively simple though - not sure the article suggests this as I only skimmed through:

Being good in your field doesn't only mean pushing articles but also being able to talk about them. I think academia should drift away from written form toward more spoken form, i.e. conferences.

What if, say, you can only publish something after presenting your work in person, answer questions, etc? The audience can be big or small, doesn't matter.

It would make publishing anything at all more expensive but maybe that's exactly what academia needs even irrespective of this AI craze?

I think we already know what we need to do: encourage people to do the work themselves, discourage beginners from immediately asking an LLM for help and re-introducing some kind of oral exam. As the article mentions, banning LLMs is impractical and what we really need are people who can tell when the LLM is confidently wrong; not people who don't know how to work with an LLM.

I hope it will encourage people to think more about what they get out of the work, what doing the work does for them; I think that's a good thing.

> the paradox is, the LLMs are only useful† if you're Schwartz, and you can't become Schwartz by using LLMs.

That you can't "become Schwartz" by using LLMs is an unproven assumption. Actually, it's a contradiction in the logic of the essay: if Bob managed to produce a valid output by using an LLM at all, then it means that he must have acquired precisely that supervision ability that the essay claims to be necessary.

Btw, note that in the thought experiment Bob isn't just delegating all the work to the LLM. He makes it summarise articles, extract important knowledge and clarify concepts. This is part of a process of learning, not being a passive consumer.

The article is a thought experiment. The author hypothesizes that Bob isn't getting the same benefit that Alice is getting. That hypothesis could be wrong. I don't know and the author doesn't know. It could be that Bob is going to have a very successful career and will deeply know the field because he is able to traverse a wider set of problems more quickly. At this point, it's just hypothesis. I don't think that we can say we need more Alices any more than we can say we need more Bobs. Unfortunately we will have to wait and see. It will be upon the academic community to do the work to enforce quality controls. That is probably the weakness to worry about.

AI is an accelerant, not a replacement for skill. At least, not yet.

I built a full stack app in Python+typescript where AI agents process 10k+ near-real-time decisions and executions per day.

I have never done full stack development and I would not have been able to do it without GitHub Copilot, but I have worked in IT (data) for 15 years including 6 in leadership. I have built many systems and teams from scratch, set up processes to ensure accuracy and minimize mistakes, and so on.

I have learned a ton about full stack development by asking the coding agent questions about the app, bouncing ideas off of it, planning together, and so on.

So yes, you need to have an idea of what you're doing if you want to build anything bigger than a cheap one shot throwaway project that sort of works, but brings no value and nobody is actually gonna use.

This is how it is right now, but at the same time AI coding agents have come an incredibly long way since 2022! I do think they will improve but it can't exactly know what you want to build. It's making an educated guess. An approximation of what you're asking it to do. You ask the same thing twice and it will have two slightly different results (assuming it's a big one shot).

This is the fundamental reality of LLMs, sort of like having a human walking (where we were before AI), a human using a car to get to places (where we are now) and FSD (this is future, look how long this took compared to the first cars).

> And so the paradox is, the LLMs are only useful† if you're Schwartz, and you can't become Schwartz by using LLMs.

I have gained a lot of benefit using LLMs in conjunction with textbooks for studying. So, I think LLMs could help you become Schwartz.

Profession (1957) by Isaac Asimov is relevant: https://news.ycombinator.com/item?id=46664195

>And so the paradox is, the LLMs are only useful† if you're Schwartz

For so many workers, their companies just want them to produce bullshit. Their managers wouldn't frame it this way, but if their subordinates start producing work with strict intellectual rigor it's going to be an issue and the subordinates will hear about it.

So, you're not wrong. But the majority of LLM customers don't care and they just want to report success internally, and the product needs to be "just good enough." An LLM might produce a shitty webpage. So long as the page loads no on will ever notice or care that it's wrong in the way that a physics paper could be wrong.

Why use a tool that generates plausible garbage?

I totally agree - the article misses this point in a very conspicuous way. It suggests that Alice and Bob will both graduate at the same level.

What may well happen instead is that Bob publishes two papers. He then outcompetes Alice based on the insistence that others have on "publish or perish". Alice becomes unemployed and struggles, having been pushed out.

The person who puts the time and effort in doesn't just sit at the same level and they don't both just find decent employment. Competition happens and the authentic learning is considered a waste of time, which leads to real and often life threatening consequences (like being homeless after being unable to find employment).

> And so the paradox is, the LLMs are only useful† if you're Schwartz

Was the LLM even useful for Schwartz, if it produced false output?

> So if Bob can do things with agents, he can do things.

The problem arrises when Bob encounters a problem too complex or unique for agents to solve.

To me, it seems a bit like the difference between learning how to cook versus buying microwave dinners. Sure, a good microwave dinner can taste really good, and it will be a lot better than what a beginning cook will make. But imagine aspiring cooks just buying premade meals because "those aren't going anywhere". Over the span of years, eventually a real cook will be able to make way better meals than anything you can buy at a grocery store.

The market will always value the exact things LLMs can not do, because if an LLM can do something, there is no reason to hire a person for that.

I'm glad you've posted this comment because I strongly feel more people need to see sentiment, and push back against what many above want to become the new norm. I see capitulation and compliance in advance, and it makes me sad. I also see two very valid, antipodal responses to this phenomenon: Exit from the industry, and malicious compliance through accelerationism.

To the reader and the casual passerby, I ask: Do you have to work at this pace, in this manner? I understand completely that mandates and pressure from above may instill a primal fear to comply, but would you be willing to summon enough courage to talk to maybe one other person you think would be sympathetic to these feelings? If you have ever cared about quality outcomes, if for no other reason than the sake of personal fulfillment, would it not be worth it to firmly but politely refuse purely metrics-focused mandates?

> The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things.

"Being able to deliver using AI" wasn't the point of the article. If it was the point, your comment would make sense.

The point of the program referred to in the article is not to deliver results, but to deliver an Alice. Delivering a Bob is a failure of the program.

Whether you think that a Bob+AI delivers the same results is not relevant to the point of the article, because the goal is not to deliver the results, it's to deliver an Alice.

They aren't going away but for some they may become prohibitively expensive after all the subsidies end.

I do think coding with local agents will keep improving to a good level but if deep thinking cloud tokens become too expensive you'll reach the limits of what your local, limited agent can do much more quickly (i.e. be even less able to do more complex work as other replies mention).

> I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading.

I dread the flip side of this which is dealing with obtuse bullshit like trying to understand why Oracle ADF won’t render forms properly, or how to optimize some codebase with a lot of N+1 calls when there’s looming deadlines and the original devs never made it scalable, or needing to dig into undercommented legacy codebases or needing to work on 3-5 projects in parallel.

Agents iterating until those start working (at least cases that are testable) and taking some of the misery and dread away makes it so that I want to theatrically defenestrate myself less.

Not everyone has the circumstance to enjoy pleasant and mentally stimulating work that’s not a frustrating slog all the time - the projects that I actually like working on are the ones I pick for weekends, I can’t guarantee the same for the 9-5.

It's the next level of abstraction. Bob is still learning, he's just learning a different set of skills than Alice.

Also, the premise that it took each of them a year to do the project means Bob was slacking because he probably could've done it in less than a month.

> So if Bob can do things with agents, he can do things.

Yes, but how does he know if it worked? If you have instant feedback, you can use LLMs and correct when things blow up. In fact, you can often try all options and see which works, which makes it ”easy” in terms of knowledge work. If you have delayed feedback, costly iterations, or multiple variables changing underneath you at all times, understanding is the only way.

That’s why building features and fixing bugs is easy, and system level technical decision making is hard. One has instant feedback, the other can take years. You could make the ”soon” argument, but even with better models, they’re still subject to training data, which is minimal for year+ delayed feedback and multivariate problems.

>The thing is, agents aren’t going away...

Aren't they currently propped up by investor money?

What happens when the investors realize the scam that it is and stop investing or start investing less...

There is still a lot of engineering to be done with LLMs. Maybe not exactly writing code but I think a lot of optimization problems will be there no matter what.

Some people treat toilet as magic hole where they throw stuff in flush and think it is fine.

If you throw garbage in you will at some point have problems.

We are in stage where people think it is fine to drop everything into LLM but then they will see the bill for usage and might be surprised that they burned money and the result was not exactly what they expected.

> if Bob can do things with agents, he can do things.

This point is directly addressed in the paper: Bob will ultimately not be able to do the things Alice can, with or without agents, because he didn't build the necessary internal deep structure and understanding of the problem space.

And if Alice later on ends up being a better scientist (using agents!) than Bob will ever be, would you not say there was something lost to the world?

Learning needs a hill to climb, and somebody to actually climb it. Bob only learned how to press an elevator button.

Many things have come and gone in this fashion oriented industry. Everyone is already bored to hell by AI output.

AI in software engineering is kept afloat by the bullshitters who jump on any new bandwagon because they are incompetent and need to distract from that. Managers like bullshit, so these people thrive for a couple of years until the next wave of bullshit is fashionable.

> I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue.

I am in the same boat, but close enough to retirement that I'm less "scared" about it. For me I'm moving up the chain; not people management, but devoting a lot more of my time up the abstraction continuum. Looking a lot more at overall designs and code quality and managing specs and inputs and requirements.

I wrote some design docs past few days for a big project the team is embarking on. We never had that before, at least not in the level of detail (per time quantum) that I was able to produce. Used 2 models from 2 companies - one to write, one to review, and bounce between them until the 3 of us agree.

Honestly it didn't take any less time than I would have done it alone, but the level of detail was better, and covered more edge cases. Calling it a "win" right now. I still enjoy it, as most of the code I/we was/are writing is mostly fancy CRUD anyway, and doesn't have huge scaling problems to solve (and too few devs I feel are being honest about their work, here).

> if Bob can do things with agents, he can do things

I’ve been reminded lately of a conversation I had with a guy at hacker space cafe around ten years ago in Berlin.

He had been working as a programmer for a significantly longer time than me. Long enough that for many years of his career, he had been programming in assembly.

He was lamenting that these days, software was written in higher level languages, and that more and more programmers no longer had the same level of knowledge about the lower level workings of computers. He had a valid point and I enjoyed talking to him.

I think about this now when I think about agentic coding. Perhaps over time most software development will be done without the knowledge of the higher level programming languages that we know today. There will still be people around that work in the higher level programming languages in the future, and are intimately familiar with the higher level languages just like today there are still people who work in assembly even if the percentage of people has gotten lower over time relative to those that don’t.

And just like there are areas where assembly is still required knowledge, I think there will be areas where knowledge of the programming languages we use today will remain necessary and vibe coding alone wont cut it. But the percentage of people working in high level languages will go down, relative to the number of people vibe coding and never even looking at the code that the LLM is writing.

Can you run an industry level LLM at home?

If not, you're changing learning to cook for Uber only meals.

And since the alternative is starving, Uber will boil the pot.

Don't give up your self sufficiency.

> The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things.

Following the model of how startups have worked for the last 20 years or so, I expect agents to eventually be locked-down/nerfed/ad-infested for higher payments. We are enjoying the fruits of VC money at the moment and they are getting everyone addicted to agents. Eventually they need to turn a profit.

Not sure how this plays out, but I would hang on to any competencies you have for anyone (or business) that wants to stick around in software. Use agents strategically, but don't give up your ability to code/reason/document, etc. The only way I can see this working differently is that there are huge advances in efficiency and open-source models.

I think a good analogy is people not being able to work on modern cars because they are too complex or require specialised tools. True I can still go places with my car, but when it goes wrong I'm less likely to be able to resolve the problem without (paid for) specialised help.

I understand your point, but this is a purely utilitarian view and it doesn’t account for the fact that, even if agents may do everything, it doesn’t mean they should, both in a normative and positive sense.

There is a vast range of scenarios in which being more or less independent from agents to perform cognitive tasks will be both desirable and necessary, at the individual, societal and economic level.

The question of how much territory we should give up to AI really is both philosophical and political. It isn’t going to be settled in mere one-sided arguments.

Some people probably enjoyed writing assembly (I am not one of those people, especially when I had to do it on paper in university exams) and code agents probably can do it well - but for the hard tasks, the tasks that are net new, code agents will produce bad results and you still need those people who enjoy writing that to show the path forward.

Code agents are great template generators and modifiers but for net new (innovative! work it‘s often barely usable without a ton of handholding or „non code generation coding“

> I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading.

You're still working on intellectually stimulating programming problems. AI doesn't go all the way with any reliability, it just provides some assistance. You're still ultimately responsible for getting things right, even with key AI help.

Programmers have ultimate (or they had) skill to solve anything if you have enough resources.

Now, you don't do thing and do other things when LLMs get stuck. There is no "given enough time I can do it".

I can't see how somebody would go solving slop bugs (slugs :)) in heavy AI generated codebase.

Hope, I'm wrong but that's somehing I personally encountered. Stay sharp.

I don't like it either. But what is really guaranteeing other markets from flunking similarly later on? What's to say other jobs are going to be any better? Back in college, most of my peers would say "I'm not cut out for anything else. This is it". They were, sure enough, computer and/or math people at heart from an early age.

More importantly, what's gonna be the next stable category of remote-first jobs that a person with a tech-adjacent or tech-minded skillset can tack onto? That's all I care about, to be honest.

I may hate tech with a passion at times and be overly bullish on its future, but there's no replacing my past jobs which have graced me and many others with quality time around family, friends, nature and sports while off work.

Agents may not go away, but they are going to fall off significantly once people wake up to how bad they are at making software. It's like in the early 00s when business execs were stoked about the idea that they could cut costs by hiring bottom rate Indian contractors: it turned out to be a disaster for quality, and eventually there was a shift back towards having staff in the US. The same thing is going to happen with LLMs.

Bob can't do things, Bob's AI can do things that Bob asks it to do. And the AI can only do things that have been done before, and only up to a certain level of complexity. Once that level is reached, the AI can't do things anymore, and Bob certainly isn't going to do anything about that, because Bob doesn't know how to do anything himself. One has to question what value Bob himself even brings to the table.

But let's assume Bob continues to have an active role, because the people above him bought in to the hype and are convinced that "prompt engineer" is the job of the future. When things inevitably start falling apart because the Bobs of the world hit a wall and can't solve the problems that need to be solved (spoiler: this is already happening), what do we do? We need Alices to come in and fix it, but the market actively discourages the existence of Alice, so what happens when there are no more Alices left? Do we just give up and collectively forget how to do things beyond a basic level?

I have a feeling that, yes, we as a species are just going to forget how to do things beyond a certain level. We are going to forget how to write an innovative science paper. We are going to forget how to create websites that aren't giant, buggy piles of React spaghetti that make your browser tab eat 2GB of RAM. We've always been forgetting, really - there are many things that humans in the past knew how to do, but nobody knows how to do today, because that's what happens when the incentive goes missing for too long. Price and convenience often win over quality, to the point that quality stops being an option. This is a form of evolutionary regression, though, and negatively affects our quality of life in many ways. AI is massively accelerating this regression, and if we don't find some way to stop it, I believe our current way of life will be entirely unrecognizable in a few decades.

Being able to deliver junior-level work isn't the goal of training juniors.

> So if Bob can do things with agents, he can do things.

But he does things wrong.

>The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things.

He'll get things (papers, code, etc) which he can't evaluate. And the next round of agents will be trained on the slop produced by the previous ones. Both successive Bob's and successive agents will have less understanding.

No - you need to understand the details in order to do the “high level” work.

The thing is Bob can use HammerAsAService™ to put in a nail. It is so cheap! Way cheaper than buying an actual hammer.

The problem with unlearning generic tools and relying on ones you rent by big corporations is that it is unreliable in the long term. The prices will be rising. The conditions will worsen. Oh nice that Bob made a thing using HammerAsAService™, but the terms of conditions (changing once a week) he accepted last week clearly say it belongs to the company now. Bob should be happy they are not suing him yet, but Bob isn't sure whether the thing that came out a month after was independently developed by that company or not just a clone of his work. Bob wishes he knew how to use a hammer.

> So if Bob can do things with agents, he can do things.

I think the key issue is whether Bob develops the ability to choose valuable things to do with agents and to judge whether the output is actually right.

That’s the open question to me: how people develop the judgment needed to direct and evaluate that output.

> The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things.

Can he? If he outsources all his thinking and understanding to agents, can he then fix things he doesn't know how to fix without agents?

Any skill is practice first and foremost. If Bob has had no practice, what then?

> I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue.

It’s not for me. Being a middle manager, with all of the liability and none of the agency, is not what I want to do for a living. Telling a robot to generate mediocre web apps and SVGs of penguins on bicycles is a lousy job.

> agents aren’t going away

Why not? Once the true cost of token generation is passed on to the end user and costs go up by 10 or 100 times, and once the honeymoon delusion of "oh wow I can just prompt the AI to write code" fades, there's a big question as to if what's left is worth it. If it isn't, agents will most certainly go away and all of this will be consigned to the "failed hype" bin along with cryptocurrency and "metaverse".

> The thing is, agents aren’t going away.

Let’s wait until they a business model that creates profit.

Most of them won’t go away, but many will become outdated or slow or enshittificated.

Imagine building your career based on the quality of google‘s search

[dead]

The whole premise is bad. If the supervisor can do it in 2 months, then they can do it in 2 weeks with AI.

Didn't PhD projects used to be about advancing the state of art?

Maybe we'll get back to that.

Then you're using it more towards vibe coding than AI-assisted coding: I use AI to write the stuff the way I want it to be written. I give it information about how to structure files, coding style and the logic flow.

Then I spend time to read each file change and give feedback on things I'd do differently. Vastly saves me time and it's very close or even better than what I would have written.

If the result is something you can't explain than slow down and follow the steps it takes as they are taken.

For me it seems more or less similar to reviewing others' changes to a codebase. In any large organization codebase, most of the changes won't be our own.

This is my primary personal concern. I think it could be an silent psychological landmine going off way too late (sic).

In a living codebase you spent long stretches to learn how it works. It's like reading a book that doesn't match your taste, but you eventually need to understand and edit it, so you push through. That process is extremely valuable, you will get familiar with the codebase, you map it out in your head, you imagine big red alerts on the problematic stuff. Over time you become more and more efficient editing and refactoring the code.

The short term state of AI is pretty much outlined by you. You get a high level bug or task, you rephrase it into proper technical instructions and let a coding agent fill in the code. Yell a few times. Fix problems by hand.

But you are already "detached" from the codebase, you have to learn it the hard way. Each time your agent is too stupid. You are less efficient, at least in this phase. But your overall understanding of the codebase will degrade over time. Once the serious data corruption hits the company, it will take weeks to figure it out.

I think this psychological detachment can potentially play out really bad for the whole industry. If we get stuck for too long in this weird phase, the whole tech talent pool might implode. (Is anyone working on plumbing LLMs?)

[dead]

Ask Claude to explain the code in depth for you. It's a language model, it's great at taking in obscure code and writing up explanations of how it works in plain English.

You can do this during the previous change phase of course. Just ask "How would one plan this change to the codebase? Could you explain in depth why?" If you're expected to be thoroughly familiar with that code, it makes no sense to skip that step.

The problem is in most environments Bob won’t spend the rest of the year figuring out what the LLM did, because bob will be busy promoting the LLM for the next deliverable, and the problem is that if all bob has time for us to prompt LLMs, and not understand, there will be a ceiling to Bob’s potential.

This won’t affect everyone equally. Some Bob’s will nerd out and spend their free time learning, but other Bob’s won’t.

Bob will never figure out there is an error in his paper. If someone tells him, the LLM will have trouble to figure it out as well, remember the LLM inserted the error to make it "look right".

Your perspective is cut off. In the real world Bob is supposed to produce outcomes that work. If he moves on into the industry and keeps producing hallucinated, skewed, manipulated nonsense, then he will fall flat instantly. If he manages to survive unnoticed, he will become CEO. The latter rather unlikely.

"LLMs won't take anything away if you are still willing to take the time to understand what it's actually building"

But do you actually understand it? The article argues exactly against this point - that you cannot understand the problems in the same way when letting agents do the initial work as you would when doing it without agents.

from the article: "you cannot learn physics by watching someone else do it. You have to pick up the pencil. You have to attempt the problem. You have to get it wrong, sit with the wrongness, and figure out where your reasoning broke. Reading the solution manual and nodding along feels like understanding. It is not understanding. Every student who has tried to coast through a problem set by reading the solutions and then bombed the exam knows this in their bones. We have centuries of accumulated pedagogical wisdom telling us that the attempt, including the failed attempt, is where the learning lives. And yet, somehow, when it comes to AI agents, we've collectively decided that maybe this time it's different. That maybe nodding at Claude's output is a substitute for doing the calculation yourself. It isn't. We knew that before LLMs existed. We seem to have forgotten it the moment they became convenient."

The problem is that LLMs stop working after a certain point of complexity or specificity, which is very obvious once you try to use it in a field you have deep understanding of. At this point, your own skills should be able to carry you forward, but if you've been using an LLM to do things for you since the start, you won't have the necessary skills.

Once they have to solve a novel problem that was not already solved for all intentes and purposes, Alice will be able to apply her skillset to that, whereas Bob will just run into a wall when the LLM starts producing garbage.

It seems to me that "high-skill human" > "LLM" > "low-skill human", the trap is that people with low levels of skills will see a fast improvement of their output, at the hidden cost of that slow build-up of skills that has a way higher ceiling.

> why are we bothering with it at all?

Because we largely want people who have committed to tens of thousands of dollars of debt to feel sufficiently warm and fuzzy enough to promote the experience so that the business model doesn’t collapse.

It’s difficult to think anyone would end up truly regretting doing a course in astrophysics, or any of the liberal arts and sciences if they have a modicum of passion, but it’s very believable that a majority of them won’t go on to have a career in it, whatever it is, directly.

They’re probably more likely to gain employment on their data science skills, or whether core competencies they honed, or just the fact that they’ve proven they can learn highly abstract concepts, or whatever their field generalises to.

Most of the jobs are in not-highly-specific academic-outcome.

> Take away the agent, and Bob is still a first-year student who hasn't started yet. The year happened around him but not inside him. He shipped a product, but he didn't learn a trade.

We're minting an entire generation of people completely dependent on VC funding. What happens if/when the AI companies fail to find a path to profitability and the VC funding dries up?

I was reading in the article that what matters is the process that leads to the (typically useless) result, what the people get out of it.

Once I realized that this white on black contrast was hurting my eyes, I decided to stop as I didn't want to see stripes for too long when looking away.

Some activity has outcomes that aren't strictly in the results.

The arguments of the LLM psychosis afflicted get more and more desperate. Astrophysics is about understanding and thinking, this comment paints it as result oriented (whatever that means).

The industrialization of academia hasn't even produced more results, it has produced more meaningless papers. Just like LLMs produce the 10.000th note taking app, which for the LLM psychosis afflicted is apparently enough.

Why should we only do things that produce some sort of value? Do we really want to reduce all of human existence to increasing profits?

Hard sciences play this crucial and often unseen role in our society : they help train humans to develop critical thinking. Not everyone with PhD in Astrophysics ends up doing Astrophysics in life; it's a discipline, or a training regime for our minds. After that PhD; the result is a human being who can tackle hard problems. We have many other such disciplines (basically any PhD in hard sciences) which produces this outcome.

Until the LLM is wrong and Bob passes the erroneous result off as accurate, reliable and vetted by a knowledgeable person. At that point Bob is not producing a useful result. Then it becomes a trap other people might get caught in, wasting valuable time and energy.

The goal of academic research is to create understanding, not papers. If we outsource all research to LLMs, then we are only producing the latter.

You missed the argument. When we are talking about faculty, yes their result is the only thing that matters, so if it was produced quicker with a LLM, that's great. But when we are talking about the student, there is a drastic difference in the student in the with LLM vs without LLM cases. In the latter they have much better understanding. And that matters in the system when we are educating future physicists.

Is that what "academia" wants? Last I checked "academia" is not a dude I can call and ask for an opinion or definition of what it was interested in.

I will make an explicit, plausible, counterpoint: academia wants to produce understanding. This is, more or less, by definition, not possible with an AI directly (obviously AIs can be useful in the process).

Take GR as an example. The vast majority of the dynamical character of the theory is inaccessible to human beings. We study it because we wanted to understand it, and only secondarily because we had a concrete "result" we were trying to "achieve."

A person who cares only about results and not about understanding is barely a person, in my opinion.

Completely missed the point of the blog post which was that the point was producing the scientist not the result

We aren't talking pocket calculators here (I see the irony of phone app in pocket), LLMs are hugely expensive things made and controlled behind costly commercial subscriptions. And likely in the middle of a huge investment bubble and stability is uncertain. So we all need to be careful about "gee we don't need that skill or person anymore", etc.

Marketing is the moat llms haven't been able to overcome. Being able to create a Word clone is easier but the difficulty of selling it is as hard or harder than ever.

Show me an llm that can sell my product and find market fit.

In reality llms are taking away profitable tools and keeping the revenue themselves.

I'm waiting for Anthropic to realise they can just set a few thousand agents loose to do just that, and monopolize the entire software market overnight. I'm not sure why they haven't done this yet.

Going 100x faster is the problem. As they say, “slow is smooth, and smooth is fast,” when going for product market fit this is very important (and understated, in my opinion). It doesn’t help that your thread is spinning out at a hundred yards a second when what you’re doing is trying to thread the needle.

Could be possible that the 10x devs working at 100x are just starting down the homestretch…

The 10x dev doesn’t just set out to build a hello world app, ya know.

Yes, the overwrought "It's not X it's Y" are signals of LLM involvement. No human uses them like that all the frickin' time. AI loves this construct way too much and cannot really tell whether the contrast is relevant/actually makes sense.

So we already have this problem and things are "fine"?

In my personal experience, the rate at which Claude Code produces suboptimal Rust is way higher than 1%.

> LLMs will get _so_ good that I don't feel the need to do this, in the same way that I don't think about the transistors my code is ultimately running on.

The problem is, they're nothing like transistors, and never will be. Those are simple. Work or don't, consistently, in an obvious, or easily testable, way.

LLM are more akin to biological things. Complex. Not well understood. Unpredictable behavior. To be safely useful, they need something like a lion tamer, except every individual LLM is its own unique species.

I like working on computers because it minimizes the amount of biological-like things I have to work with.

This is exactly the problem I see today.

And it's not just a volume problem.

Mediocre devs previously couldn't complete a project by themselves and were forced to solicit help and receive feedback along the way.

When all managers care about is "shipping", development becomes a race to the bottom. Devs who used to collaborate are now competing. Whoever gets the slop into the codebase fastest, wins.

I think it just really depends. There is no fixed rule to how PhD programs are supposed to work. Sometimes your advisor will suggest projects he finds interesting and wants to see done, he just doesn't have time to do it himself. That's pretty common. Sometimes advisors don't have that and/or want students to come up with their own projects proposals, etc.

It depends on the program, and even more so, the student and the mentor. It can also vary over time, with more direction early on in a graduate program, and less direction later. Some mentors are very directive, and basically treat students as labor executing tasks they don't have time or want to do. Other times, the student is coming up with all the ideas and the mentor is facilitating it with resources or even nothing but uncertain advice or permissions now and then.

This can lead to a lot of problems as I think in some fields, by some academics, the default assumption is the former, when it's really the latter. This leads to a kind of overattribution of contribution by senior faculty, or conversely, an underappreciation of less senior individuals. The tendency for senior faculty be listed last on papers, and therefore, for the first and last authors to accumulate credit, is a good example of how twisted this logic has become.

It's one tiny example of enormous problems with credit in academics (but also maybe far afield from your question).

It is a spectrum. My advisor was very hands off. He didn't, ultimately, even really understand my PhD. He knew the problem, but he had no path in mind to solve it, that was up to me. I'm now working (as a software engineer) with a person who is very hands on with his students (and even postdocs) to the point of giving them specific tasks to do and then discussing the result every week. He defines the problems and structure of the solution, the students at least partially are an extension of himself, they are doing stuff he merely doesn't have time to do himself.

And there is everything in between.

Often at the start yes. So the students gets a bit of recognition, a bit of experience and a bit of knowledge.

From the cases I've observed directly in the area I work in, yes.

[dead]

In the future the llms can answer those questions for you by listening and feeding you answers into your headset.

The process you describe is a gate keeping exercise which will change to include llm judges at somepoint.

Faster doesn't always mean better. I've "learned" things from LLM really fast, but I don't retain the information the same way as if I had taken my time to really work through it

Every AI/agentic thread on HN follows the same tension: builders want to build and solve problems. Code or task completion are implementation details to be done on the path to the actual prize: solving the problem. And then there are the coders, that have honed their mechanical skill of implementation and derive their intellectual fulfillment from that. The latter crowd has a rough time because much of it can be automated now, the former camp is happy because look at all the stuff that can now be built!

> You need both.

yea, there are multiple parts to education. 1) teach skills useful to the economy 2) teach the theories of the subject, and finally 3) tweak existing theories and create new ones. An electrician can fix problems without understanding theory of electromagnetism. These are the trades folks. A EE college graduate has presumably understood some theory, and can apply it in different useful ways. These are the engineers. Finally, there are folks who not only understand the theory of the craft, but can tweak it creatively for the future. These are the researchers.

Bob better fits as a trades-person or engineer whereas Alice fits better as a researcher.

I have to disagree that Bob will be a better producer, although I do agree that Bob will produce more. In this scenario, Bob isn't clear on which LLM output is valid and important and which is erroneous and misleading; I think that's a pretty critical distinction. It's the kind of thing that might go undetected for a long time, until a particular paper turns out to be important and it's discovered that it's also entirely wrong, wasting a lot of time and energy.

If results are what matters, why is popular software so buggy and lacking in features?

I think I disagree in what I see around me it’s less like going from fire to lighter and more like going from hand tools to power tools. If your skill was in understanding how the hand tools work, it’s harder to get a level of abstraction up and have a vision for building a house. If we’re not able to learn than less people are going to be able to get that vision, especially if you’re in technical domains where engineering and architecture matter. It’s going to be a weird future. I’m pretty effective with these tools, but I’m fifteen years of hacking on them manually. Some folks who are not as far into their careers don’t seem to know where to start.

There’s a reason most people aren’t promoted to manager until they have years of experience under their belt. And now we’re expecting folks to be managers on day 1.

> The questions were similar. The progress was similar. The trajectory, from the outside, was identical.

LLM speak. But the rest of that quote doesn't look LLM-generated, it's too fiddly and complex of an argument. I think this was edited with AI, but the underlying argument at least is human.

Or maybe the author is just a competent writer.

> why would you write in perfect prose

If you could, why wouldn't you? LLM witch-hunts over every halfway competent writer are becoming quite tiresome

It sure is

> "no conceivable word where the agent will be taken away"

LLM access is a paid service. HN concerns itself with inequality constantly and it's not inconceivable that some individuals get ahead because they can afford to pay for more tokens and better models than those who are poorer.

AI is extremely good at producing well formatted bullshit. You need to be constantly on guard against stuff that sounds and looks right but ultimately is just noise. You can also waste a ton of time on this. Especially OpenAI's offering shows poorly in this respect: it will keep circling back to its own comfort zone to show off some piece of code or some concept that it knows a lot about whilst avoiding the actual question. It's really good at jumping to the wrong conclusions (and making it sound like some kind of profound insight). But the few times that it is on the money make up for all of that noise. Even so, I could do without the wasted time and endless back and forths correcting the same stuff over and over again, it is extremely tedious.

> as if debugging python code is the point of it all.

You have a good point, but I would argue that debugging itself is a foundational skill. Like imagine Sherlock Holmes being able to use any modern crime-fighting technology, and using it extensively. If Sherlock is not using his deductive reasoning, then he's not a 'detective'. He's just some schmuck who has a cool device to find the right/wrong person to arrest.

Debugging is "problem-solving" in a specific domain. Sure, if the problem is solved, then I guess that's the point of it all and you don't have to solve the problem. But we're all looking towards a world in which people have to solve problems, but their only problem-solving skill is trying to get an AI to find someone to arrest. We need more Sherlocks to use their minds to get to the bottom of things, not more idiot cops who arrest the wrong person because the AI told them to.

Are you working on compilers? If not it seems you did not understand what is being talked about here.

Do you lack fundamental understand of those apps you built that are still in use? Did you lack understanding of their workings when you built them?

You don’t need to understand compilers because the code it compiles, when valid according to the language specification, is supposed to work as written, and virtually always does. There is no language specification and no “as written” with LLMs.

No problem with that.

However, at one point in my career, I was frustrated with limitations in a language (Fortran II) and my curiosity got the better of me and I studied compilers thoroughly.

This led to a new job and the understanding of many new useful programming concepts. Very rewarding.

But if you are curious, studying compilers, maybe even writing a new one, will give you tools to do other things.

While working with LLMs, much of my experience gives me new ideas to push the LLM to explore.

Have you written a compiler, though?

AI does not give you knowledge. It magnifies both intelligence and stupidity with zero bias towards either. If you were above average intelligent then you may be able to do a little bit more than before assuming you were trained before AI came along. And if you were not so smart then you will be able to make larger messes.

The problem, and I think the article indirectly points at that, is that the next generation to come along won't learn to think for themselves first. So they will on average end up on the 'B' track rather than that they will be able to develop their intelligence. I see this happening with the kids my kids hang out with. They don't want to understand anything because the AI can do that for them, or so they believe. They don't see that if you don't learn to think about smaller problems that the larger ones will be completely out of reach.

A related quote from A. N. Whitehead:

> It is a profoundly erroneous truism ... that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them.

Current civilization is very complex. And it’s also fragile in some parts. When you build systems around instant communication and the availability of stuff built in the other side of the world on a fixed schedule, it’s very easy to disrupt.

> 4. People will eventually get the hang of using AI to do the optimum amount of delegation such that they still retain what is necessary and delegate what is not necessary. People who don't do this optimally will get outcompeted

Then they’ll be at the mercy of the online service availability and the company themselves. Also there’s the non deterministic result. I can delegate my understanding of some problems to a library, a software, a framework, because their operation are deterministic. Not so with LLMs.

I think you missed the point. The entire article is about specialists: astrophysicists. The problem with AI is that specialists are delegating their thinking about their specialty! The fear here is that society will stop producing specialists, and thus society will no longer progress.

The danger is also for Alice, if she gets moved to Bob's project when things start falling apart in production.

Do you have any more platitudes to add so I can fill my dismissive HN comment bingo card?

What if AI in the long run makes us slower and less effective. As someone who is one of the folks supercharged by these tools, I could see it.

I think people are underestimating the level of experience and knowledge that’s required to prompt LLM’s. Not in the micro sense but in the macro. It seems so easy because it feels easy. But if you don’t have deep understanding of the domain, it will just feel impossible. The person next to you with domain experience will say “it’s so easy, look at This simple sentence I typed in”. And be like “it’s just a skill issue, why is everyone struggling so much” and not understand the years of accumulated wisdom or innate talent it took to type that simple sentence.

AI makes the easy things easy and the hard things harder, and more omnipresent.