I used to have dozens of phone numbers memorized. Once I got a cell phone I forgot everyone's number. I don't even know the phone number of my own mother.

I don't want to lose my ability to think. I don't want to become intellectually dependent on AI in the slightest.

I've been programming for over a decade without AI and I don't suddenly need it now.

"But there is something to be said about atrophy. If you don't use it, you lose it. I wonder if my coding skill will deteriorate in the years to come..."

"You don't want to over-rely on AI because a certain degree of "productive struggle" is essential for learning something deeply."

These two ideas are closely related and really just different aspects of the same basic frailty of the human intellect. Understanding that I think can really inform you about how you might use these tools in work (or life) and where the lines need to be drawn for your own personal circumstance.

I can't say I disagree with anything you said and think you've made an insightful observation.

> a certain degree of "productive struggle" is essential

Honestly, I'm not sure this would account for most of the difficulty in learning. In my experience most of the difficulty involved in learning something involved a few missing pieces of insight. It often took longer to understand the few missing pieces than the rest of the topic. If they are accurate enough, LLMs are great for getting yourself unstuck and keep yourself moving. Although it has always been a part of the learning experience, I'm not sure frantically looking through hundreds of explanations for a missing detail is a better use of one's time than to dig deeper in the time you save.

A current 3rd year college student here. I really want LLMs to help me in learning but the success rate is 0.

They often can not generate relatively trivial code When they do, they can not explain that code. For example, I was trying to learn socket programing in C. Claude generated the code, but when I stared asking about stuff, it regressed hard. Also, often the code is more complex than it needs to be. When learning a topic, I want that topic, not the most common relevant code with all the spagheti used on github.

For other subjects, like dbms, computer network, when asking about concepts, you better double check, because they still make stuff up. I asked ChatGPT to solve prev year question for dbms, and it gave a long, answer which looked good on surface. But when I actually read through because I need to understand what it is doing, there were glaring flaws. When I point them out, it makes other mistakes.

So, LLMs struggle to generate concise to the point code. They can not explain that code. They regularly make stuff up. This is after trying Claude, ChatGPT and Gemini with their paid versions in various capacities.

My bottom line is, I should NEVER use a LLM to learn. There is no fine line here. I have tried again and again because tech bros keep preaching about sparks of AGI, making startup with 0 coding skills. They are either fools or genius.

LLMs are useful strictly if you already know what you are doing. That's when your productivity gains are achieved.

> Are you producing? Or are you learning?

> AI makes me at least 2x more efficient at my job. It seems irrational not to use it

Fair, but there is a corollary here -- the purpose of learning at least in part is to prepare you for the workforce. If that is the case, then one of the things students need to get good at is conversing with LLMs, because they will need to do so to be competitive in the workplace. I find it somewhat analogous to the advent of being able to do research on the internet, which I experienced as an early 90s kid, where everyone was saying "now they won't know how to do research anymore, they won't know the Dewey decimal system, oh no!". Now the last vestiges of physical libraries being a place where you even can conduct up-to-date research on most topics are crumbling, and research _just is_ largely done online in some form or another.

Same thing will likely happen with LLMs, especially as they improve in quality and accuracy over the next decade, and whether we like it or not.

Our internal metrics show a decrease in productivity when more inexperienced developers use AI, and an increase when experienced developers with 10+ years use it. We see a decrease in code quality across experience levels which needs to be rectified across all experience levels but even with the time spent refactoring it’s still an increase in productivity. I think I should note that we don’t use these metrics for employee review in any way. The reason we have them is because they come with the DORA (eu regulation) compliance tool we use to monitor code-quality. They won’t be used for employee measurement while I work here. I don’t manage people, but I was brought in to help IT transition from startup to enterprise so set the direction with management confidence.

I’m a little worried about developers turning to LLMs instead of official documentation as the first thing they do. I still view LLMs as mostly being fancy auto-complete with some automation capabilities. I don’t think they are very good at teaching you things. Maybe they are better than Google programming, but the disadvantage LLMs have seem to be that our employees tend to trust the LLMs more than they would trust what they found on Google. I don’t see an issue with people using LLMs on fields they aren’t too experienced with yet however. We’ve already seen people start using different models to refine their answers, we’ve also seen an increase in internal libraries and automation in place of external tools. Which is what we want, again because we’re under some heavy EU regulations where even “safe” external dependencies are a bureaucratic nightmare.

I really do wonder what it’ll do to general education though. Seeing how terrible and great these tools can be from a field I’m an expert in.

How long are your progress reports? Mine are a one sentence message like "Hey, we've got a fix for the user profile bug, but we can't deploy it for an hour minimum because we've found an edge case to do with where the account signed up from" and I'm not sure where the AI comes in.

> But there is something to be said about atrophy. If you don't use it, you lose it.

YMMV, but I didn't ride a bike for 10ish years, and then got back on and I was happily riding quickly after. I also use zsh and ctrl+r for every Linux command, but I can still come up with the command of I need to, just, slowly. Ive overall found that if I learn a thing, it's learnt. Stuff I didn't learn in university, but passed anyways, like Jacobians, I still don't know, but I've got the gyst of it. I do keep getting better and better at the banjo the less I play it, and getting back to the drumming plateau is quick.

Maybe the drumming plateau is the thing? You can quickly get back to similar skill levels after not doing the thing in a while, but it's very hard to move that plateau upwards

> I wonder if there will be an analog for intellectual work. Will people be going to "mental" gyms in the future?

Already do — that's what "brain training" apps; consumer EdTech like Duolingo and Brilliant; and educational YouTube and podcasts like 3blue1brown, Easy German, ElectroBOOM, Overly Sarcastic Productions all are.

It is trivially easy to spot AI writing if you are familiar with it, but if it requires failing most of the class for turning in LLM generated material, I think we are going to find that abolishing graded homework is the only tenable solution.

The student's job is not to do everything the teacher says, it is to get through schooling somewhat intact and ready for their future. The sad fact is that many things we were forced to do in school were not helpful at all, and only existed because the teachers thought it was, or for no real reason at all.

Pretending that pedagogy has established and verified methodology that will result in a completely developed student, if only the student did the work as prescribed, is quite silly.

Teaching evolves with technology like every other part of society and it may come out worse or it may come out better, but I don't want to go back fountain pens and slide rules and I think in 20 years this generation won't look back on their education thinking they got a worse one than we did because they could cheat easier.

> I think often AI sceptics go too far in assuming users blindly use the AI to do everything

Users are not a monolithic group. Some users/students absolutely use AI blindly.

There are also many, many ways to use AI counterproductively. One of the most pernicious I have noticed is users who turn to AI for the initial idea without reflecting about the problem first. This removes a critical step from the creative process, and prevents practice of critical and analytical thinking. Struggling to come up with a solution first before seeing one (either from AI or another human) is essential for learning a skill.

The effect is that people end up lacking self confidence in their ability to solve problems on their own. They give up much too easily if they don't have a tool doing it for them.

This is also a great PR press release for openai. They are telling users NOT to generate content with lazy prompts, but to use as an aid.

Sometimes I tried to be transparent in having used chatgpt like for minor stuff, and I got pooled into being a lazy fuck who submitted slob.

We are probably going to see an end to the bearish wave of ai and a correction towards reasonable AI use.

No, it cannot solve new math problems and is not as smart as Alakazam, but it CAN format your citations and make you a cup of coffee.

Well, I got chatgpt (gpt4o) to write me a very basic json parser once (and a gltf parser). Although it was very basic and lacking any error checking, it did what i asked it (although not in one go, i had to refine my questions multiple times).

This iterative process hasn’t led to better results than my best effort, but it has led to 90% of my best in a fraction of the time. That’s especially true if I have curated a list of quotes, key phrases, and research literature I know I want to use directly or pull from.

I teach basic statistics to computer scientists (in the context of quantitative research methods) and this year every single one of my group of 30+ students used ChatGPT to generate their final report (other than the obvious wording style, the visualizations all had the same visual language, so it was obvious). There were glaring, laughable errors in the analyses, graphs, conclusions, etc.

I remember when I was a student that my teachers would complain that we did “compilation-based programming” meaning we hit “compile” before we thought about the code we wrote, and let the compiler find the faults. ChatGPT is the new compiler: it creates results so fast that it’s literally more worth it to just turn them in and wait for the response than bothering to think about it. I’m sure a large amount of these students are passing their courses due to simple statistics (I.e. teachers being unable to catch every problematic submission).

Agreed! Using AI as a collaborative tool rather than a replacement is the best approach

I imagine maths teachers had a similar dilemma when pocket calculators became widely available.

Now, in the UK students sit 2 different exams: one where calculators are forbidden and one where calculators are permitted (and encouraged). The problems for the calculator exam are chosen so that the candidate must do a lot of problem solving that isn't just computation. Furthermore, putting a problem into a calculator and then double checking the answer is a skill in itself that is taught.

I think the same sort of solution will be needed across the board now - where students learn to think for themselves without the technology but also learn to correctly use the technology to solve the right kinds of challenges and have the skills to check the answers.

People on HN often talk about ai detection or putting invisible text in the instructions to detect copy and pasting. I think this is a fundamentally wrong approach. We need to work with, not against the technology - the genie is out of the bottle now.

As an example of a non-chatgpt way to evaluate students, teachers can choose topics chatgpt fails at. I do a lot of writing on niche topics and there are plenty of topics out there where chatgpt has no clue and spits out pure fabrications. Teachers can play around to find a topic where chatgpt performs poorly.

My takeaway: a chrome plugin that writes LLM generated text into a Google doc over the course of a couple of days is a great product idea!

> Of course a student who would cheat may also lie about cheating, but in a few cases they were able to prove authorship using the history feature built into Google docs.

It's scary to see the reversal of the burden of proof becoming more accepted.

:sigh:

With all the concern over AI, it's being used _against recommendations_ to detect AI usage? [0][1]

So while the concern for using AI is founded, teachers are so mistaken at understanding what it is and the tech around is that they are using AI in areas it's publicly acknowleded it doesn't work. That detracts from any credibility the teachers have about AI usage!

[0] https://openai.com/index/new-ai-classifier-for-indicating-ai... openai pulled their AI classifier [1] https://mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-wo...

If there's something more unethical than AI plagiarism, that's going to be using AI to condemn people for it. I'm afraid that would further devalue actually writing your own stuff, as supposed to iterating with ChatGPT to produce the least AI-sounding writing out of fear of false accusations.

Nice! You should check out a free chrome plugin that I wrote for this called revision history. It’s organically grown to 140k users, so the problem obviously resonates (revisionhistory.com).

I've seen my 15 year old use ChatGPT for her homework and I'm ok with most of what she does.

For example she tends to ask it for outlines instead of the whole thing, mostly to beat "white page paralysis" and also because it often provides some aspect she might have overlooked.

She tends to avoid asking for big paragraphs because she doesn't trust it with facts and also dislikes editing out the "annoying" AI style or prompting for style rewrites. But she will feed it phrases from her own writing that get too tangled for simplification.

Also she will vary the balance of AI/own effort according to the nature of the task, her respect for the teacher or subject: Interesting work from a engaging lecturer? Light LLM touch or none. Malicious make-work or readable Lorem Ipsum when the point is the format of the thing instead of the content? AI pap by the tons for you. I find it healthy and mature.

> Most of them seem overconfident that they could write papers of similar quality if they ever tried. But so far, according to all of them, they have not.

Ah, the illusion of knowledge..

Coming from an education system where writing lengthy prose and essays is expected for every topic from literature to mathematics, I can confidently say that, after not having actively practiced that form of writing for over a decade, I wouldn't be able to produce a paper of what was considered average-quality back then. It would take time, effort, and a few tries, despite years and years of previous practice. Even more so if the only medium in front of me would be a blank sheet of paper and a pen.

So to confidently claim you can produce something of high quality when you've never really done it before is.. ..misguided.

But in the end, perhaps not really different to the illusion knowledge one gets with google at its fingertips. Pull the plug, and you are left with nothing.

1st year college student here, and an alumni of a certain high school programme I'd rather not mention ever again.

I've used LLMs MULTIPLE times during "academic" work, often for original idea generation. Never for full-on, actual writing.

Think of my usage as treating it as a tool that gives you a stem of an idea that you develop further on your own. It helps me persevere with the worst part of work: having to actually come up with an entire idea on my own.

And AI detection tools are still complete garbage as far as I can tell, a paper abstract I've written in front of one of my professors got flagged as 100% AI generated (while having no access to outside sources).

Also anecdotally, I’m a college student and do not use LLMs to generate my papers.

I have however asked ChatGPT to cite sources for specific things, to varying success. Surprisingly, it returns sources that actually exist most of the time now. They often aren’t super helpful though because they either aren’t in my school’s library or are books rather than articles.

The real hack is you give it an example of previous work you have done and ask it to write in that style.

> When I was young, I refused to learn geography because we had map applications. I could just look it up. I did the same for anything I could, offload the cognitive overhead to something better -- I think this is something we all do consciously or not.

This is certainly useful to a point, and I don't recommend memorizing a lot of trivia, but it's easy to go too far with it. Having a basic mental model about many aspects of the world is extremely important to thinking deeply about complex topics. Many subjects worth thinking about involve interactions between multiple domains and being able to quickly work though various ideas in your head without having to stop umpteen times can make a world of difference.

To stick with the maps example, if you're reading an article about conflict in the Middle East it's helpful to know off the top of your head whether or not Iran borders Canada. There are plenty of jobs in software or finance that don't require one to be good at mental math, but you're going to run into trouble if you don't at least grok the concept of exponential growth or have a sense for orders of magnitude.

I think at a certain point, you either value having your own skills and knowledge, or you don't. You may as well ask why anyone bothers learning to throw a baseball when they could just offload to a pitching machine.

And I get it. Pitchers who go pro get paid a lot and aren't allowed to use machines, so that's a hell of an incentive, but the vast majority of kids who ever pick up a baseball are never going to go pro, are never even going to try to go pro, and just enjoy playing the game.

It's fair to say many, if not most, students don't enjoy writing the way kids enjoy playing games, but at the same time, the point was mostly never mastering the five paragraph thesis format anyway. The point was learning to learn, about arbitrary topics, well enough to the point that you could write a reasonably well-argued paper about it. Even if a machine can do the writing for you, it can't do the learning for you. There's either value in having knowledge in your own brain or there isn't. If there isn't, then there never was, and AI didn't change that. You always could have paid or bullied the smarter kids into doing the work for you.

> We talked about "critical thinking", "creative problem solving", and "comprehension of complex systems" as the next step, but even when discussing this, how long will it be until more models or workflows catch up?

Either these things are important to learn for their own sake or they aren’t. If the former, then nothing about these objectives needs changing, and if the latter then education itself will be a waste of time.

> I refused to learn geography because we had map applications

Which is ironic, because geography isn’t about memorizing maps

> more education around basic models and how they work

yes, I think this is critical. There's a slate star codex article "Janus Simulators" that explains this very well, that I rewrote to make more accessible to people like my mom. It's not hard to explain this to people, you just need to let them interact with a base model, and explore its quirks. It's a game, people are good at learning systems that they can get immediate feedback from.

> How can a human be a facilitator and shepherd of the workflows in such a way that can complement the model and grow the human?

Humans must use what the AI doesn't have - physicality. We have hands and feet, we can do things in the world. AI just responds to our prompts from the cloud. So the human will have to test ideas in reality, to validate, do experiments. AI can ideate, we need to use our superior access and life-long context to help it keep on the right track.

We also have another unique quality - we can be punished, we are accountable. AI cannot be meaningfully punished for wrongdoing, what can you do to an algorithm? But a human can assume responsibility for an AI in critical scenarios. When there is a lot of value at stake we need someone who can be accountable for the outcome.

Actually, it shows the real problem about education... and what education is for!

Education is not a way to memorize a lot of knowledge, but a way to train your brain to recognize patterns and to learn. Obviously you need some knowledges too, but you generally dont need to be an expert, only to have "basic" knowledges.

Studying different domains allow to learn some different knowledges but also to learn new way of thinking.

For example : geography allows you to understand geopolitic and often sociology and history. And urban city design. And war strategy. And architecture...

So, when students are using LLM (and it's worst for children), they're missing on training their brain (yes... they get dumber) and learning basic human knowledge (so more prone to any fake news, even the most obvious)

I think there is a bit of a 3rd category as well:

1. What can tools do better now that no human could hope to compete with?

2. Which other tasks are likely to remain human-led in the near term?

3. For the areas where tools excel, what is the optimum amount of background understanding to have?

E.g. you mention memorizing maps. Memorizing all of the countries and their main cities is probably not very optimal for 99.999%+ of people vs referencing a map app. At the same time needing to pull up a map for any mention of a location outside of "home" is not necessarily optimal just because the map will have it. And of course the other things about maps in general (types, features, limitations, ways to use them, ways they change) outside of a particular app implementation that would go along with general geography.

I'm not sure I understand the geography point - maps and indexes have been around for hundreds of years - what did the app add to make it not worthwhile learning geography?

The correct answer, and you'd see it if folks paid attention to the constant linkedin "AI researcher/ML Engineer job postings are up 10% week over week" banners, is to aggressively reorient education in society to education about how to use AI systems.

This rustles a TON of feathers to even broach as a topic, but it's the only correct one. The AI engineer will eat everything, including your educational system, in 5-10 years. You can either swim against the current and be ate by the sharks or swim with it and survive longer. I'll make sure my kids are learning about AI related concepts from the very beginning.

This was also the correct way to handle it circa the calculator era. We should have made most people get very good at using calculators, and doing "computational math" since that's the vast majority of real world math that most people have to do. Imagine a world where Statistics was primarily taught with Excel/R instead of with paper. It'd be better, I promise you!

But instead, we have to live in a world of luddites and authoritarians, who invent wonderful miracle tools and then tell you not to use them because you must struggle. The tyrant in their mind must be inflicted upon those under them!

It is far better to spend one class period, teaching the rote long multiplication technique, and then focus on word problems and applications of using it (via calculator), than to literally steal the time of children and make them hate math by forcing them to do times tables, again and again. Luddites are time thieves.

> geography

In Germany the subject is called "Erdkunde" which would translate to geology. And this term is, I assume, more appropriate as it isn't just about what is where but also about geological history and science and how volcanoes work and how to read maps and such.

How did the people who wrote the LLM and associated software do it when they had no such thing to "just look it up"?

A lot of them are from Russia/Europe/China, at least the most successful implementers. My guess is that at least Russia/China will continue with the traditional education while watching with glee that the West is dumbing itself down even further.

> It's kind of sad that the people who invented all this stuff really loved school

Really? I didn't invent ChatGPT or anything like that, but I work in tech, I love science, maths, and learning in general, but I hated school. I found school to make the most interesting things boring. I felt it was all about following a line and writing too many pages of stuff. Maybe I am wrong, but it is certainly how I felt back then, and I am sure many people at OpenAI felt this way.

The school system is not great for atypical profiles, and most of the geniuses who are able to come up with revolutionary ideas are atypical. Note that I don't mean that if you are atypical and/or hate school then you are a genius, or that geniuses in general hate school, but I am convinced that among well educated people, geniuses are more likely to hate school.

Falling an exam is many things but "catastrophic".

Why do they get homework then? I don’t expect the professors are willing to go over and correct autogenerated LLM homework. The purpose of homework is to apply and cement knowledge. In some cases homework is so excessive that students find ways to cheat. If homework is reasonable, students can just do it and bypass LLMs altogether (at least for the purpose of the homework).

The point of the article is to highlight how students should be using ChatGPT.

Now it's up to you to share it with your kid and convince them they shouldn't cheat themselves out of an education by offloading the learning part to an LLM.

This doesn't change the value provided by the institution they're enrolled in unless the teachers are offloading their jobs to LLMs in a way that's detrimental to the students.

Cheating has been and will always be a thing.

I’ve seen a lot of places that require students to reference their ChatGPT use — and I think it is wrong headed. Because it is not a source to cite!

But, sharing links for helping teachers understand your prompting is great

> Would it be a viable solution for teachers to ask everyone to do this? Like a mandatory part of the homework? And grade it? Just a random thought...

To ask everyone to use ChatGPT, or to ask everyone to document their use of ChatGPT? I don't think the former is reasonable unless it's specifically the point of the class, and I believe that the latter is already done (by requirements to cite sources), though, as often happens, rapid technological developments mean that people don't think of ChatGPT as a source that they are required to cite like any other.

As an Information Tech Instructor I have my students use ChatGPT all the time - but it never occurred to me to make them share the link. Will do it now.

I don't like the idea of requiring it in school. It is tantamount to the government (of which school is a manifestation) forcing you (a minor) to enter into a rather unfavorable contract (data collection? arbitration? prove you are a human?) with "Open"AI. This type of thing is already too normalized.

or submit a screen recording of their writing process.

seems hard to fake that. and you could randomly quiz them on it .

I teach at the university level, and I just expect more from my students. Instead of implementing data structures like we did when I was in school, something ChatGPT is very good at; my students are building systems, something ChatGPT has more trouble with.

Instead of paper exams asking students "find the bug" or "implement a short function", they get a takehome exam where they have to write tests, integrate their project into a CI pipeline, use version control, and implement a dropbox-like system in Rust, which we expect to have a good deal of functionality and accompanying documentation.

I tell them go ahead and use whatever they want. It's easier than policing their tools. If they can put it together, and it works, and they can explain it back to me, then I'm satisfied. Even if they use ChatGPT it'll take a great deal of work and knowledge to get running.

If ChatGPT suddenly is able to put a project like that together, then I'll ask for even more.

> I imagine the bar will just keep getting higher and more will be able to taught in less time.

But more won’t be able to be _learned_ in less time

> I'm really curious to see where higher education will go now that we have LLM's. I imagine the bar will just keep getting higher and more will be able to taught in less time

On the other hand, 54% of US adults read and write at a 6th grade level or below. They will get absolutely left in the dust by all this.

https://www.snopes.com/news/2022/08/02/us-literacy-rate/

They're still largely abysmal for any other discipline that's not StackOverflow related so apart from ripoff bootcamps (that are dead anyway) higher education is safe for the time being.

After calculators were invented basically no one can can do math in their head.

I’d argue the bar will be lower and lower. Yeah those who want can learn more in less time. But those who don’t - will learn much less.

Creative writing does not seem to have a raised bar.

Prior knowledge part becomes important as you realise, verifying the output of an LLM to be right requires the same too.

In fact you can only ask the smallest possible increment so that the answer can be verified to be true with least possible effort, and you can build from there.

The same issue happens with code. Its not like total beginners will be able to write replacement for the Linux kernel in the first 5 mins of use. Or that a product manager will just write a product spec and a billion dollar product will be produced magically.

You will still do most of the code work, AI will just do the smart typing work for you.

Perhaps it all comes down the fact that, you have to verify the output of the process. And you need to be aware of what you are doing at a very fundamental level to make that happen.

> presumably not be hired

Definitely "no hire"

Is this even realistic? The font wouldn't maintain its size in the ChatGPT box... It would take a large prompt or a careless person to not notice the extra instructions.

You can use a screenshot to get around it. It's a case of the "analogue hole".

Conversely there's a non-zero chance someone uses the same language or patterns an LLM might. I've fed my own writings into several AI detectors and regularly get scored 60% or higher that my stuff was written by an LLM. A few things getting into the 80s. F me for being a well read, precise (or maybe not?), writer I guess. Maybe this explains my difficulty in finding new employment? The wife does occasionally accuse me of being a robot.

Yeah, but that's the only actually creative part of writing. It's the bit that, once you reach a certain level of skill, becomes enjoyable.

I mean, I know what you've described is what everyone will do, but I feel sad for the students who'll learn like that. They won't develop as writers to the point of having style, and then - once everything written is a style-less mush of LLM-generated phrases - what's the point of reading it? We might as well feed everything back through the LLM to extract the "main ideas" for us.

I guess we're "saving labor" that way, but what an anti-human process we'll have made.

Sure if your goal is to generate a race of Eloi who live exclusively as passive consumers, casually disregarding anything a machine can hide from their feeble minds.

>I know this is heretical within the tech bro monoculture of HN.

Because it's objectively a false statement.

The LLM output is only "garbage" if your prompt and fact-checking also are garbage.

It's like calling a Ti-84 calculator "garbage" because the user has a hole in their head and doesn't know how to use it, hence they can't produce anything useful using it.

They need more users to increase the valuations and get more money. The interesting part is that more users are equal to more expenses and, as far as anyone can tell, they aren't breaking even.