Ever seen someone try and search something on Google and they are just AWFUL at it? They can never find what they're looking for and then you try and can pull it up in a single search? That's what it is like watching some people try to use LLM's. Learning how to prompt an LLM is as much a learned skill as much as learning how to phrase internet searches is a learned skill. And as much as people decried that "searching Google isn't a real skill" tech-savvy people knew better.

Same thing except now it's also many tech-savvy people joining in with the tech-unsavvy in saying that prompting isn't a real skill...but people who know better know that it is.

On average, people are awfully bad at describing exactly what it is they want. Ever speak with a client? And you have to go back and forward for a few hours to finally figure out what it is they wanted? In that scenario you're the LLM. Except the LLM won't keep asking probing questions and clarifications - it will simply give them what they originally asked for (which isn't what they want). Then they think the LLM is stupid and stop trying to ask it for things.

Utilizing an LLM to its full potential is a lot of iterative work and, at least for the time being, requires having some understanding of how it works underneath the hood (eg. would you get better results by starting a new session or asking it to forget previous, poorly worded instructions?).

> But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them. It's all claims that it's great with no further elaboration on the workflows.

To give an example, one person (a researcher at DeepMind) recently wrote about specific instances of his uses of LLMs, with anecdotes about alternatives to each example. [1] People on HN had different responses with similar claims with elaborations on how it has changed some of their workflows. [2]

While it would be interesting to see randomized controlled trials on LLM usage, hearing people's anecdotes brings to mind the (often misquoted) phrase: "The plural of anecdote is data". [3] [4]

[1] https://nicholas.carlini.com/writing/2024/how-i-use-ai.html

[2] https://news.ycombinator.com/item?id=41150317

[3] http://blog.danwin.com/don-t-forget-the-plural-of-anecdote-i...

[4] originally misquoted as "Anecdote is the plural of data."

Or people say "I've been pumping out thousands of lines of perfectly good code by writing paragraphs and paragraphs of text explaining what I want!" its like what are you programming dog? and they will never tell you, and then you look at their github and its like a dead simple starter project.

I recently built a Brainfuck compiler and TUI debugger and I tested out a few LLM's just to see if I could get some useful output regarding a few niche and complicated issues, and it just gave me garbage that looked mildly correct. Then I'm told its because I'm not prompting hard enough... I'd rather just learn how to do it at that point. Once I solve that problem, I can solve it again in the future in .25x the time.

Specifically within the last week, I have used Claude and Claude via cursor to:

- write some moderately complex powershell to perform a one-off process

- add typescript annotations to a random file in my org's codebase

- land a minor feature quickly in another codebase

- suggest libraries and write sample(ish) code to see what their rough use would look like to help choose between them for a future feature design

- provide text to fill out an extensive sales RFT spreadsheet based on notes and some RAG

- generat some very domain-specific realistic sounding test data (just naming)

- scaffold out some PowerPoint slides for a training session

There are likely others (LLMs have helped with research and in my personal life too)

All of these are things that I could do (and probably do better) but I have a young baby at the moment and the situation means that my focus windows are small and I'm time poor. With this workflow I'm achieving more than I was when I had fully uninterrupted time.

> But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them.

I'm an iOS dev, my knowledge of JS and CSS is circa 2004. I've used ChatGPT to convert some of my circa 2009 Java games into browser games.

> Chatting with a LLM look to me like your intent is either vague or you don't know the units to use

Or that you're moving up the management track.

Managers don't write code either. Some prefer it that way.

I have used chatGPT to write test systems for our (physical) products. I have a pretty decent understanding of how code/programs works structurally, I just don't know the syntax/language (Python in this case).

So I can translate things like

"Create an array, then query this instrument for xyz measurements, then store those measurements in the array. Then store that array in the .csv file we created before"

It works fantastic and saved us from outsourcing.

The key difference is that this is a multidisciplinary conversational interface, and a tool in itself for interrelating structured meaning and reshaping it coherently enough so that it can be of great value both in the specific domain of the dialog, and in the potential to take it on any tangent in any way that can be expressed.

Of course it has limitations and you can't be sleep at the wheel, but that's true of any tool or task.

For one, I spend less time on Stackoverflow. LLMs can usually give you the answer to little questions about programming or command-line utilities right away.

I think people who are successfully using it to write code are just chaining APIs together to make the same web apps you see everywhere.

> The fact everyone that say they've become more productive with LLMs won't say how exactly. But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them.

A pretty literal response: https://www.youtube.com/@TheRevAlokSingh/streams

Plenty of Lean 4 and Cursor.

Here's one from simonw

https://gist.github.com/simonw/97e29b86540fcc627da4984daf5b7...

There are more to be found on his blog on the ai-assisted-programming tag. https://simonwillison.net/tags/ai-assisted-programming/

> The fact everyone that say they've become more productive with LLMs won't say how exactly.

I have python scripts which do lot of automation like downloading pdfs, bookmarking pdfs, processing them, etc. Thanks to LLMs I dont write a python code myself, I just ask an LLM to write it, I just provide the requirement. I just copy the code generated by the AI model and run it. If there any errors, I just ask AI to fix it.

> The fact everyone that say they've become more productive with LLMs won't say how exactly.

Anecdotally, I no longer use StackOverflow. I don’t have to deal with random downvotes and feeling stupid because some expert with a 10k+ score on 15 SE sites each votes my question to be closed. I’m pretty tech savvy, been doing development for 15 years, but I’m always learning new things.

I can describe a rough idea of what I want to an LLM and get just enough code for me to hit the ground running…or, I can ask a question in forum and twiddle my thumbs and look through 50 tabs to hopefully stumble upon a solution in the meantime.

I’m productive af now. I was paying for ChatGPT but Claude has been my goto for the past few months.

You clearly have made up your mind that it can't be right but to me it's like arguing against breathing. There are no uncertainties or misunderstandings here. The productivity gains are real and the code produced is more robust. Not in theory, but in practice. This is a fact for me and you trying to convince me otherwise is just silly when I have the result right in front of me. It's also not just boilerplate. It's all code.

Just be mindful that it is one's interest to push the "LLMs suck, don't waste your time with them" narrative once they figure out how to harness LLMs.

"Jason is a strong coder, and he despises AI tools!"

It is also a problem of ego.

It is difficult if you have been told all your life that you are the best, to accept the fact that a computer or even other people might be better than you.

It requires lot of self-reflection.

Real top-tiers programmers actually don’t feel threatened by LLMs. For them it is just one more tool in the toolbox like syntax highlighting or code completion.

They choose to use these tools based on productivity gains or losses, depending on the situation.

Re: Compensation expectations, I figured out a long time ago that bad programmers create bad code, and bad code creates work for good programmers.

If the amount of bad code is no longer limited by the availability of workers who can be trained up to "just below average" and instead anyone who knows how to work a touchscreen can make AI slop, this opens up a big economic opportunity.

This kind of problems is really not where LLMs shine.

Where you save loads of time is when you need to write lots of code using unfamiliar APIs. Especially when it's APIs you won't work with a lot and spending loads of time learning then would just be a waste of time. In these cases LLMs call tell you the correct API cells and it's easy to verify. The LLM isn't really solving some difficult technical problem, but saves lots of work.

I think that contemporary models are trained for engagement, not for actual help.

My experience is the same as yours, but I noticed that while LLMs circa two years ago tried to come up with the answer, current generation of LLMs tries to make me come with the answer. And that not helping at all.

Did you tell it that? Are you trying to converse and discuss or are you trying to one shot stuff? If it gets something wrong, tell it. Don't just stop and try another prompt. You have to think of it as another person. You can talk to it, question it, guide it.

Try starting from ground zero and guiding it to the solution rather than trying to one shot your entire solution in one go.

I want you to implement this kind of tree in language x.

Ok good, now I want you to modify it to do Y.

Etc.

Wasn't there a recent post about many research papers getting published with conclusions derived from buggy/incorrect code?

I'd put more hope in improving LLMs/derivatives than improving the level of effort and thought in code across the entire population of "people who code", especially the subset who would rather be doing something else with their time and effort / see it as a distraction from the "real" work that leverages their actual area of expertise.

> You are still responsible for what you do regardless of the means you used to do it. And a lot of people use this not because it’s more productive but because it requires less effort and less thought because those are the hard bits.

Yeah, that's...the whole point of tools. They reduce effort. And they don't shift your responsibility. For many of us, LLMs are overwhelmingly worth the tradeoffs. If your experience differs, then it's unfortunate, and I hate that for you. Don't use 'em!

Ugh, dude, I used to push bad code into production without ChatGPT. It is such a stupid argument. Do you really think people are just blindly pushing code they can't make heads or tails of? That they haven't tested? Do you seriously think people are just one shotting code and blasting it into prod? I am completely baffled by people in this industry that just don't get it. Learn to prompt. Write tests. Wtf.

ChatGPT voice interface plugged into the audio stream, with the prompt:

- I need you to assist me during a programming interview, you will be listening to two people, the interviewer and me. When the interviewer asks a question, I'd like you to feed me lines that seem realistic for an interview where I'm nervous, don't give me a full blown answer right away. Be very succinct. If I think you misunderstood something, I will mention the key phrase "I'm nervous today and had too much coffee". In this situation, remember I'm the one that will say the phrase, and it might be because you've mistaken me by the interviewer and I want you to "reset". If I want you to dig deeper than what you've provided me with, I'll say the key phrase "Let's dig deeper now". If I think you've hallucinated and want you to try again, I'll say "This might be wrong, let me think for just a minute please". Remember, other than these key phrases, I'll only be talking to the interviewer, not you.

On a second screen of some sort. Other than that, interviewers will just have to accept that nobody will be doing the job without these sort of assistants from now on anyway. As an interviewer I let candidates consult online docs for specific things already because they'll have access to Google during the job, this is just an extension of that.

Not everyone is doing coding interviews. Some might struggle with a particular language due to lack of muscle memory, but can dictate the logic in pseudocode and can avoid pitfalls inferred from past experience. This sort of workflow is compatible with LLMs, assuming a sufficient background (otherwise one can't recognize when the output diverges from your intent).

I personally treat the LLM as a rubber duck. Often I reject its output. In other cases, I can accept it and refactor it into something even better. The name of the game is augmentation.

I sometimes get the idea from statements like this - and HN's focus on interviewing in general - that people are switching jobs a dozen times a year or something. How often are most people switching jobs? I've had 5 jobs in the last 20 years.

This. I’ve been using elixir for ~6 months (guided by Claude) and probably couldn’t solve fizz buzz at a whiteboard without making a syntax error. Eek.

You need to prep for job interviews anyway. I'd rather spend the majority of my time being productive.

Job interview? You might be surprised at the number of us who don’t code for a job.

I’ve made the decision to embrace being bad at coding but getting a ton of work done using an LLM and if my future employer doesn’t want massive productivity and would prefer being able to leetcode really well then I unironically respect that and that’s ok.

I’m not doing ground breaking software stuff, it’s just web dev at non massive scales.

See, if you work in AI, say, as an AI researcher, asking them not to be allowed to use AI models in the interview is basically not an option.

Also, often folks in this space are better at cheating than you will be at detecting them. Don't believe me? https://bigvu.tv/captions-video-maker/ai-eye-contact-fix

we're a post illiteracy society now

You're listing plain models, so I'm assuming you're using them directly. Aider and similar agents use those models but they don't step at the first answer. You can add test running and a linter to the request and it will essentially enter a loop like: what are the steps to solve (prompt)?; here's a map of the repository, which files do you need?; what's your proposed change?; here's the final change and the test run, do you think the problem has been solved?; (go back to the beginning if not)

See the video at https://plandex.ai/ to get an idea how it works.

I think it's perfectly ok to be critical of technology as long as one is thoughtful rather than dismissive. There is a lot of hype right now and pushing back against it is the right thing to do.

I'm more reacting against simplistic and categorical pronouncements of straight up "uselessness," which to me seems un-curious and deeply cynical, especially since it is evidentially untrue in many domains (though it is true for some domains). I just find this kind of emotional cynicism (not a healthy skepticism, but cynicism) to be contrary to the spirit of innovation and openness, and indeed contrary to evidence. It's also an overgeneralization -- "I don't find it useful, so it's useless" -- rather than "Why don't I find it useful, and why do others do? Let me learn more."

As future-looking HNers, I'd expect we would understand the world through a lens of "trajectories" rather than "current state". Just because LLMs hallucinate and make mistakes with a tone of confidence today -- a deep weakness -- doesn't mean they are altogether useless. We've witnessed that despite their weaknesses, we are getting a lot of value from them in many domains today and they are getting better over time.

Take neural networks themselves for instance. For most of the 90s-2000s, people thought they were a dead end. My own professor had great vitriol against Neural Networks. Most of the initial promises in the 80s truly didn't pan out. Turns out what was missing was (lots of) data, which the Internet provided. And look where we are today.

Another area of cynicism is self-driving cars (Level 5). Lots of hype and overpromise, and lots of people saying it will never happen because it requires a cognitive model of the world, which is too complicated, and there are too many exceptional cases for there to ever be Level 5 autonomy. Possibly true, but I think "never" is a very strong sentiment that is unworthy of a curious person.

Crypro and AI have similarities and differences.

The similarities include intense "true believer" pitches and governments taking them seriously.

The differences include that the most famous cryptocurrency can't function as a direct payment mechanism for just lunch purchases in just Berlin (IIRC nor is it enough for all interbank transactions so it can't even be a behind-the-scenes system by itself), while GenAI output keeps ending up in places people would rather not find it like homework and that person on Twitter who's telling you Russia Did Nothing Wrong (and also giving you a nice cheesecake recipe because they don't do any input sanitation).

Also, I'm deeply skeptical of crypto too due to its present scamminess, but I am keeping an open mind that there is a future in which crypto -- once it gets over this phase of get-rich-quick schemers -- will be seen as just another asset class.

I read somewhere that historically bonds in their early days were also associated with scamminess but today they're just a vanilla asset.

just like with crypto and NFTs and the metaverse, they are always focused on what is suppsoedly coming down the pipe in the future and not what is actually possible today

Who cares what they think? In fact, the fewer who uses this the better for the ones that do. It's not in my self-interest to convert anyone and I obviously don't need to convince myself when I have the result right in front of me. Whether you believe it or not does not make me less productive.

Probably nothing a junior programmer wouldn't be able to do relatively easily.

Given how hard we thought programming was a year or two ago, I wouldn't bank my future on design being too hard for an LLM. They're already quite good at helping writing design docs.

Lol nope. When I'm trying to get it do make something big/complicated I start by telling it it's a software project manager and have me build a spec sheet on the design. Then I hand that off to an architect to flesh out the languages, libraries, files needed etc. Then from that list you can have it work on individual files and functions.

OK, but at that point you've told it basically everything, and this is a really basic book problem!

As another example I just gave it a network flow problem, and asked it to convert to maximum flow (I'm using the API, not chatGPT).

Despite numerous promptings, it never got it right -- it would not stop putting a limit on the source and sink (usually 1), which mean the flow was always exactly 1, here's the bit of wrong code (it's the last part, it's shouldn't be putting any restrictions on nmap['s'] and nmap['t'], as they represent the source and sink), and I couldn't pursade it this was wrong after several prods:

    # Constraints: Ensure flow conservation at each vertex
    A_eq = np.zeros((len(namelist), num_edges))
    b_eq = np.zeros(len(namelist))

    for i, (u, v, capacity) in enumerate(edges):
        A_eq[nmap[u], i] = 1  # Outflow from u
        A_eq[nmap[v], i] = -1  # Inflow to v

    # Source 's' has a net outflow, and sink 't' has a net inflow
    b_eq[nmap['s']] = 1
    b_eq[nmap['t']] = -1

I was referring to section 4 of A Survey for Solving Mixed Integer Programming via Machine Learning(2024): https://arxiv.org/pdf/2401.03244.

I’ve heard (but not so much observed) that there is substantial difference between recent models, so it’s possible that they are better than when this was written.

Anyways, CPLEX has an associated modeling language that features syntactic sugar which has the effect of providing opaqueness to the underlying MILP that it solves. I find LLMs essentially unable to even make an attempt at determining the MILP from that language.

PS: How is Xpress? Is there some reason to prefer it to Gurobi or Mosek?

It depends. If you got into computing because it seemed like the most lucrative career choice to which you might be suited, then yes, I can imagine feeling threatened. Bummer, sucks to be you. But if you got into computing because it seemed like the most interesting thing available to work on, then no, I can't imagine not being fascinated by, and supportive of, the progress being made in the ML field today. Any hostility you feel should be directed at the people who want to lock it all up behind legislative, judicial, or proprietary doors.

In my case, it's all I can do not to walk away from everything else I'm doing to follow this particular muse. I don't have a lot of sympathy for my colleagues who see it as a threat. If you're afraid of new ideas, technologies, and methodologies, you picked the wrong line of work.

I used it and was really disappointed, maybe because of the hype. It generates a long page of entries that I used to generate in the past, often with better results. Note I use it for code generation, not for problem solving.

So I cancelled two of my 3 subscriptions as I realized OpenAI goes in a direction that is not useful for me at all. Claude, on the other hand, is incredibly useful.

no I agree I just don't think existing Lean codebase is approaching useful coverage. Should change soon

You mean Twitter ?

I totally agree, there e.g. so many companies out there who rely on fully manual processes internally simply because they cannot currently afford to hire programmers to solve the problems they have for them. The ROI just isn't there.

Reduce costs by an order of magnitude or two, and suddenly there's a whole heap more projects that become profitable.

I abandoned 3D art after witnessing DALL-E 2's capabilities, and I've observed the ripple effects across creative fields. Initially, photographers and fellow artists dismissed AI as a non-threat. That turned out to be misguided optimism. Now, with Midjourney producing such impressive work, the majority of us have become largely obsolete. These days, I'm noticing developers exhibiting the same denial. From my perspective, they're on a similar trajectory. This AI revolution is impacting creative and technical industries far more rapidly and dramatically than most anticipated..

> Before it can replace the brilliant programmer, it needs to be able to replace the mediocre programmer

Nope. Compensation is exponential. Being able to replace a top performer with a fee mediocre devs pair coding with an LLM is more than fine for 90% of use cases.

It's very much the opposite.

LLMs free me from the nuts and bolts of the "how", for example I don't have to manually type out a loop. I just write a comment and the loop magically appears. Sometimes I don't have to prompt it at all.

With my brain freed from the drudgery of everyday programming, I have more mental cycles to dedicate to higher concerns such as overall architecture, and I'm just way more productive.

For experienced programmers this is a godsend.

Less experienced developers lack the ability to mentally "see" how software should be architected in a way that balances the concerns, so writing a loop a bit faster it's not as much of an advantage. Also, they lack the reflexes to instantly decide if generated code is correct or incorrect.

LLMs are limited by the user's decision speed, the LLM generates code for you but you have to decide whether to accept or reject. If it takes me 1 second to decide to accept code that would have taken me 10 seconds to physically type, then I'm saving 9 seconds, which really adds up. For a junior developer, LLMs may give negative productivity if it takes them longer to decide if the LLM's version is correct than it would have taken them to type whatever they were going to write in the first place.

I like to think I'm more of a "challenging and novel problems" developer than a "copy and paste from Stack Overflow" developer, and I've been finding LLMs extremely useful for over two years at this point.

Notes: https://simonwillison.net/tags/ai-assisted-programming/

How many of them are there of the latter type? In my 15 yrs of experience I would say 95%+ of all developers belong to your first category.

The large majority of professional writers and artists produce thankless commodity output for things like TV advertisements, games, SEO content. These jobs should be threatened.

Is most code being written the equivalent of high-art or Shutterstock?

The current driving force of AI is the desire to cut costs. Jobs will be cut even if ChatGPT is nowhere near a junior engineer and that's the problem.

Once we have AI-based language servers, which will, at some point in the future, be able to track entire repositories, I think maintaining projects will actually be far easier than right now.

> At this point, such sentiments feel either willfully ignorant, or said in bad faith.

I feel exactly the same, but in the opposite direction.

As someone who’s been programming for 17 years and working professionally for 10, I’m unable to get any huge productivity boosts from AI tools. They’re better than Google+stack overflow for asking random questions, but in a specific context and they’re good for repetitive, but not identical, syntax. That’s about where the gains end for me.

Maybe at this point I’m just so fast about looking up documentation. Maybe the languages/problems I’m facing aren’t well represented in the training data, but I just don’t see this amazing advancement.

I’d really love to see, live, someone programming who really gets these big productivity gains.

> I see contrarians claiming that LLMs are literally never useful for anyone

While I don't doubt that there's at least one person that has said this, what you're saying doesn't conflict with the things I and many others in the "skeptic" camp have said. LLMs are useful for a very specific set of tasks. The tasks you've listed are a tiny sliver of all the tasks that AI could potentially be doing. Would it be a good idea to consult an LLM if your mother is passed out on the floor? Probably not. The problem I have is with extrapolating from the current successes to conclude that many more tasks will be done by AI in five years.

Thing is, I'm used to hearing a very similar sentiment on how e.g. using vim keybindings is so literally going to make me a 10x 100x whatever rockstar developer - and it's like what, enabling me to edit text a bit faster? And it's always anecdotes that yeah, from-qualia you feel so fast. But from-qualia I run like a marathon runner and sound like a radio host.

I personally did find some use cases for it and it does a decent job of cutting out minor gruntwork for me. But the experience itself screams to me that whatever gains I'm feeling I'm getting are all in my head.

I'm a contrarian who believes your anecdote, and could even imagine that 5% of LLM users feel the same way, but thinks (a) these systems are about half as good as they're ever going to get, (b) we're past the point of diminishing returns, and (c) what we do have isn't worth the energy costs of running it, let alone creating it in the first place.

I think there may be a set of people that have figured out, 1) how to interact with LLMs; and 2) what in their lives is improved when interacting with LLMs. I am in the group that has not found the best use case for my own life, and have never needed it for improving anything I need to get done. Always looking for suggestions, though!

It's a system that is designed to convince you.

That's interesting take, personally I'd say that graduate-level math is orders of magnitude harder than significant majority of programming. And I mean that it's inherently harder, i.e. not due to lack of background.

Nah. Dogs are far emotionally better than most humans. Their intelligence is their limitation. Also “skills in learning and acquiring knowledge” is basically intelligence

Here are the key points outlining why thewanderer1983's response misinterprets noch's comment and contains inaccuracies:

    Misinterpretation of the Original Point:
        Intelligence vs. Moral Superiority: Noch discusses the intelligence level of a mediocre graduate science student compared to the general population. Thewanderer1983 misreads this as a claim of moral or inherent superiority over "the rest of humanity," which was not implied.

    Conflation of Educational Levels:
        University Graduates vs. Graduate Students: The response conflates undergraduate university graduates with graduate science students. Noch specifically refers to graduate students who have pursued advanced degrees, which typically require higher levels of specialization and intellectual rigor.

    Incorrect Assessment of Intelligence Measures:
        Graduate Studies as a Measure of Intelligence: Successfully completing graduate studies, especially in science, often requires significant intellectual capability. Dismissing this as "not a good measure of intelligence" overlooks the challenges inherent in advanced academic work.

    Irrelevant Focus on Credentials and Industry Practices:
        Credentials vs. Intelligence Discussion: Noch's comment centers on intelligence levels, not merely on holding credentials. Bringing up how industry values experience over degrees shifts the focus away from the original discussion about intelligence.

    Unfounded Assumptions About Noch's Background:
        Ad Hominem Attack: Suggesting that Noch hasn't been to university is an unfounded personal assumption that does not contribute to the argument and detracts from a respectful discourse.

    Introduction of the 'Elite Class' Notion:
        Straw Man Argument: Thewanderer1983 introduces the concept of an "elite class," which Noch did not mention. This misrepresents the original comment and argues against a point that wasn't made.

    Overgeneralizations About Academia and Industry:
        Academia's Recognition of Intelligence: Claiming that academics don't consider intelligence until one has a Ph.D. and years of work is an overgeneralization. Intelligence is recognized and valued at various academic levels.
        Industry's View on Graduates: Stating that industry only uses graduates as an entry requirement ignores the significant roles that advanced degree holders often play in innovation and leadership within industries.

    Ignoring the Core Benefit Highlighted:
        AI as a Life Upgrade: Noch emphasizes how access to AI with the intelligence level of a graduate student is a substantial benefit for most people. Thewanderer1983 fails to address this key point, instead focusing on unrelated issues.

    Misunderstanding of the Value of Graduate Education:
        Work Ethic vs. Intellectual Achievement: While a good work ethic is important, graduate education in science also demands high intellectual capability, critical thinking, and problem-solving skills.

    Logical Fallacies:
        Red Herring: The discussion about industry preferences and academic credentials diverts from the main argument about the intelligence level of graduate students.
        Ad Hominem: Attacking Noch's presumed lack of university experience instead of addressing the argument presented.

o1-preview is still quite a specialized model, and you can come up with very easy questions that it fails embarassingly despite it's success in seemingly much more difficult tests like olympiad programming/maths questions.

You certainly shouldn't think of it like having access to a graduate student whenever you want, although hopefully that's coming.

I've tried the ChatGPT integration and it was kinda just useless. On smaller datasets it told me nothing that wasn't obviously apparent from the charts and tables; on larger datasets it couldn't do much besides basic key/value retrieval. Asking it to analyze a large time-series table was an exercise in futility, I remain pretty unimpressed with current offerings.

Wolframalpha is a free service, really kind of an ad for all the (curated, accurate) datasets built into Wolfram Language

Wolfram Research is a profitable company btw

FACT: The technology is inherently unreliable in its current form. And the weakness is built in, its not going to go away anytime soon.

Well ya once each project, e.g. “grad level math”, “k—12 math”, “undergrad math”, “k-12 chemistry”, etc is sufficient they are all fed into a larger more powerful model.

Maybe my RLHF work does make it harder for me to dream, but I teach models math which means a lot of prompt writing, and yet I have not found a way to have the model teach me math I don’t know yet (and there’s a lot I don’t know). It’s fun to play around with, but I still gravitate toward the isolated texts, not the aggregation as too much is lost or averaged in my opinion/experience. But hey maybe I’m overtrained on the traditional learning methods.

Yes, but its reasoning ability is extremely poor in my experience with real world programming tasks. I’m talking about stuff that Claude3.5 Sonnet handles easily, and GPT4o can also handle if it can fit in its smaller context window, where Gemini 1.5 pro just completely fails.

Bigger context is definitely helpful, but not if it comes at the expense of reasoning/analytical ability. I’m always a bit puzzled why people stress the importance of these “needle in a haystack” tests where the model has to find one specific thing in a huge document. That seems far less relevant to me in terms of usefulness in the real world.

It doesn't work well though. You can't just stuff your entire codebase into it and get good results. I work somewhere that tries to do this internally

I wonder how much of this is an intrinsic limitation of LLMs, and how much that interdisciplinary thinking and mashing together of problem domains is missing in the training data. It's a pretty rare thing, and the only times these analogies and linkages become noted is when they happen to work out (and they don't seem so far out of left field anymore once this happens).

> Creativity is defined as the tendency to generate or recognize ideas, alternatives, or possibilities that may be useful in solving problems, communicating with others, and entertaining ourselves and others

Basically can you provide a new perspective on solving a problem that hasn't been considered or a new way of looking at an existing idea in a new way to unlock a path.

A couple days ago I saw a tweet that described how to remove an element from an array in O(1) time instead of O(n). The key to it was identifying that for the purpose the given array was being used for, it could be unordered / not fully ordered, and it would not be an issue.

This way, it was possible to simply replace the element with the last array element, then decrease the size of the array by one. I'd say that's pretty creative: whoever came up with this was able identify what can be traded off to make the previously impossible, possible, unlocking new scales and possibilities.

In practice, I'd say creativity is often being able to manifest people's qualia in some unprecedented way. For example, say you're experimenting in your DAW, and discover a pretty cool sound. You identify the ways it can be used to emote and then utilize it in a work. If you really stumbled upon a sound that a lot of people find as emotive as you did, you just did something creative: it's as if you translated the qualia of an emotion into sound.

This qualia to manifestation is what's behind creativity in all of senses of the word I believe. In my previous example, discovering that orderedness is not actually a strict requirement, and (ab)using that to significantly alter the scaling of such an action is creative, because it undoes the notion that orderedness is a requirement. It goes against what's natural, but in a way that becomes extremely natural and indispensable once realized.

I think, in that way, current AIs are trained to be uncreative, since being creative inherently requires experimentation that is unaligned with the normal.

You’re asking the right questions, IMO! Chomsky has spent his life trying to answer this question in different forms, and has ultimately arrived to the conclusion that we live in a “Pre-Gallilean” era of cognitive science, where the only answers available to us are developed through the use of intuitive interpretation (like they used to do with ‘the heavens’/space) instead of empirical contradiction (aka science).

He does have some answers, such as “human creativity is the ability to create an infinite range of outputs from a finite range of inputs that nonetheless pertain to our motivations/context in some useful way”, but that’s obviously not a very satisfying answer. It tells us a little — I think Tao is gesturing to exactly this when he complains that GPTo1 can only apply and combine mathematical approaches within a sort of closed domain rather than propose radically new ones - but it’s not helpful for an everyday understanding of creativity. IMO :)

In his words, from Language and Mind:

  "Roughly, where we deal with cognitive structures, either in a mature state of knowledge and belief or in the initial state, we face problems, but not mysteries. When we ask how humans make use of these cognitive structures, how and why they make choices and behave as they do, although there is much that we can say as human beings with intuition and insight, there is little, I believe, that we can say as scientists… 
  What I have called elsewhere 'the creative aspect of language use' remains as much a mystery to us as it was to the Cartesians who discussed it, in part, in the context of the problem of 'other minds."

https://youtu.be/3wfNl2L0Gf8?si=WA3DpnaFEvd3QqCt

Of the 40 or so definitions of consciousness, I have no reason to think any is either necessary nor sufficient for any of the things we want AI to do that Transformer models can't (or can but badly and we want them to do better).

To me it's quite good for Rust as it usually suggests me some (unknown to me) borrow checker API that actually helps me either solve the issue at hand or at least point me in the right direction to figure what is actually wrong with my code and it doesn't compile

I’m not sure i understand your complaint

Are you arguing that I’m making an appeal to authority fallacy?

I've seen it take over a couple of minutes, at which point I switched to Claude. And have seen reports of it taking even longer. So it may be that you didn't wait long enough.

How can you verify a proof though? Pure math isn't really about computations, and it can be very hard to spot subtle errors in a proof that an LLM might introduce, especially since they seem better at sounding convincing rather than being right.

Good luck! That can be pretty hard to do when you're at the learning stage, and I would think doubly so given the LLM style where everything 'looks' very convincing.

Of course not. And he is relatively more popular than some others on his level of expertise. But I think Tao's opinion is a bit more interesting than a general PhD/professional.

Thank you.

Why are you saying this as if it was a bad thing? Just because software becomes better at us at something doesn't mean we can't do it out of fun (e.g. see chess community for example).

do you also value your personal relationships based on employment prospects?

There is truth to this, but you’re overstating it. If AI is cheap and can replace grunt workers then we’ll have a massive wave of new businesses solving problems that previously required a massive capital investment.

Are you kidding or being serious? Most business owners are just regular people. Have you ever worked in a small business before?

Agreed. That is what the original poster said. I didnt manage to reproduce the error on my end, but I dont know the full context or maybe the memory on my end changes the output.