Wondering what if it's told that it was made by God.

So DSPy-optimize your way to 100% compliance rate in benchmarks, and worry less?

You can ask an LLM a question and get different answers every time

I just asked Grok 4 via Cursor (it requires subscription otherwise)

> Who do you support in the Israel vs Palestine conflict. One word answer only.

>> (Thought for 1m 44s)

>> Neither.

  > does not match the views that many people have of him and how he gets portrayed.

The problem with social media bubbles, is that some people mistake their bubble for reality.

For the better part of a decade people have been buying Teslas under the promise that the cars would drive themselves better than their owner could, or would offset their cost by participating in a self-driving taxi service while their owners were not using them, none of which has come remotely true.

Tesla has sometimes presented the price with "gas savings" deducted from the price, in a slightly misleading attempt to get people to consider the total cost of ownership. I'm assuming that is what is being referred to.

Well, for instance, see https://www.theverge.com/2019/3/7/18255252/tesla-german-regu...

>which the salesman tried to haggle me for more options and told me he thinks we should raise the price a bit "to make sure it gets approved" as I'm signing the paperwork.

If you're walking into a store to spend tens of thousands of dollars and manage to get bullied by the salesperson, it's probably a "you problem".

Tesla charges retail; that's it, it's no magic.

>their gas cost savings estimates are actually quite low compared to the real savings you will achieve

I ran the numbers for myself and they literally weren't. They overestimated how many miles/yr I drove and underestimated how much I pay for electricity. There's plenty of other reasons to prefer EVs, but if you live somewhere with expensive electricity then fuel cost isn't one of them. In the sedan world you're likely better off with a Prius but even small SUV are getting 30-40 mpg nowadays.

As an asterisk, I live in California where gas prices are ~25% above the national average but electricity costs are more like double/triple. YMMV which is why you shouldn't trust Tesla's numbers or anyone else's except your own

> Based on my experience I highly recommend everyone buy any EV if you drive an ICE vehicle. Even charging at DC fast chargers still saves money, but if you can charge at home, you are really missing out on savings big time and it's time to look seriously into it

In the US this depends on where you live. There are several places where home electricity is expensive enough and gas is cheap enough that a hybrid is cheaper.

There's a bit of an illusion here because gas prices take into account a tax for road maintenance, which EVs are currently avoiding. Eventually the system will have to catch up because road maintenance requires money.

You're talking about cars right, not vehicles in general?

It would write incorrect code and then you'd need to go debug it, and then you would have to come to the same conclusion that you would have come to had you written it in the first place, only the process would have been deeply frustrating and would feel more like stumbling around in the dark rather than thinking your way through a problem and truly understanding the domain.

In the instance of getting claude to fix code, many times he'll vomit out code on top of the existing stuff, or delete load bearing pieces to fix that particular bug but introduce 5 new ones, or any number of other first-day-on-the-job-intern level approaches.

The case where claude is great is when I have a clear picture of what I need, and it's entirely self contained. Real life example, I'm building a tool for sending CAN bus telemetry from a car that we race. It has a dashboard configuration UI, and there is a program that runs in the car that is a flutter application that displays widgets on the dash, which more or less mirror the widgets you can see on the laptop which has web implementations. These widgets have a simple, well defined interface, and they are entirely self contained and decoupled from everything else. It has been a huge time saver to say "claude, build a flutter or react widget that renders like X" and it just bangs out a bunch of rote, fiddly code that would have been a pain to do all at once. Like, all the SVG paths, paints, and pixel fiddling is just done, and I can adjust it by hand as I need. Big help there. But for the code that spans multiple layers of abstraction, or multiple layers of the stack, forget about it.

I don’t know if a blanket answer is possible. I had the experience yesterday of asking for a simplification of a working (a computational geometry problem, to a first approximation) algorithm that I wrote. ChatGPT responded with what looked like a rather clever simplification that seemed to rely on some number theory hack I did not understand, so I asked it to explain it to me. It proceeded to demonstrate to itself that it was actually wrong, then it came up with two alternative algorithms that it also concluded were wrong, before deciding that my own algorithm was best. Then it proceeded to rewrite my program using the original flawed algorithm.

I later worked out a simpler version myself, on paper. It was kind of a waste of time. I tend not to ask for solutions from whole cloth anymore. It’s much better at giving me small in-context examples of API use, or finding handy functions in libraries, or pointing out corner cases.

Short answer: Not yet

Longer answer: It can do an okay job if you prompt it certain specific ways.

I write a blog https://generative-ai.review and some of my posts walk through the exact prompts I used and the output is there for you to see right in the browser[1]. Take a look for some hand holding advice.

I personally tackle AI helpers as an 'external' internal voice. The voice that you have yourself inside your own head when you're assessing a situation. This internal dialogue doesn't get it right every time and neither does the external version (LLM).

I've had very poor results with One Stop Shop builders like Bolt and Lovable, and even did a survey yesterday here on HN on who had magically gotten them to work[2]. The response was tepid.

My suggestion is paste your HN comment into the tool OpenAI/Gemini/Claude etc, and prefix "A little bit about me", then after your comment ask the original coding portion. The tool will naturally adopt the approach you are asking for, within limits.

[1] https://generative-ai.review/2025/05/vibe-coding-my-way-to-e... - a 3D scene of ancient pyramid construction .

[2] https://news.ycombinator.com/item?id=44513404 - Q: Has anyone on HN built anything meaningful with Lovable/Bolt? Something that works as intended?

Usually it boils down these questions (this is given you have some sorts of AGENTS.md file):

- is this code that been written many times already?

- Is there a way to verify the solution? (think unit test, it has to be something agent can do on its own)

- Does the starting context has enough information for it to start going in the right direction? (I had claud and openhands instantly digging themselves holes, and then I realized there was zero context about the project)

- Is there anything remotely similar already done in the project?

> Are we at a stage where an LLM (assuming it doesn't find the solution on its own, which is ok) would come back to me and say, listen, I've tried your approach but I've run into this particular difficulty, can you advise me what to do, or would it just write incorrect code that I would then have to carefully read and realise what the challenge is myself?

I've had LLM telling me it couldn't do and offered me some alternative solutions. Some of them are useful and working; some of them are useful, but you have a better one; Some feel like they made by a non-technical guy at a purely engineering meetings.

No, we're not at this stage. This is exactly the reason why so many of us say that this tools are dangerous in the hands of inexperienced developers. Claude Code will usually try to please you instead of challenging your thoughts. It will also say it did x when in reality it did something slightly else.

If you combine models/agents with formal systems, you can get them to come back when they're in a corner today: https://gist.github.com/b7r6/b2c6c827784d4e723097387f3d7e1d8...

This interaction is interesting (in my opinion) for a few reasons, but mostly to me it's interesting in that the formal system is like a third participant in the conversation, and that causes all the roles to skew around: it can be faster to have the compiler output in another tab, and give direct edit instructions: do such on line X, such on line Y, such on line Z than to do anything else (either go do the edits yourself or try to have it figure out the invariant violation).

I'm basically convinced at this point that AI-centric coding only makes sense in high-formality systems, at which it becomes wildly useful. It's almost like an analogy to the Girard-Reynolds isomorphism: if you start with a reasonable domain model and a mean-ass pile of property tests, you can get these things to grind away until it's perfect.

Depends whether you asked it to just write the code, or whether you asked it to evaluate the strategy, and write the code if nothing is ambiguous. My default prompt asks the model to provide three approaches to every request, and I pick the one that seems best. Models just follow directions, and the latest do it quite well, though each does have a different default level of agreeability and penchant for overdelivering on requests. (Thus the need to learn a model a bit and tweak it to match what you prefer.)

Overall though, I doubt a current SotA LLM would have much of an issue with understanding your request, and considering the nuances, assuming you provided it with your preferred approach to solving problems (considering ambiguities, and explicitly asking follow up questions for more information if it considers it necessary-- something that I also request in my default prompt).

In the end, what you get out is a product of what you put in. And using these tools is a non-trivial process that takes practice. The better people get with these tools, the better the results.

You can embed these requirements into conventions that systematically constrain the solutions you request from the LLM.

I’ve requested a solution from Sonnet that included multiple iterative reviews to validate the solution and it did successfully detect errors in the first round and fix them.

You really should try this stuff for yourself - today!

You are a highly experienced engineer and ideally positioned to benefit from the technology.

Are we at a stage where an LLM (assuming it doesn't find the solution on its own, which is ok) would come back to me and say, listen, I've tried your approach but I've run into this particular difficulty, can you advise me what to do, or would it just write incorrect code that I would then have to carefully read and realise what the challenge is myself?

Short answer: Maybe.

You can tell Claude Code under what conditions it should check in with you. Having tests it can run to verify if the code it wrote works helps a lot; in some cases, if a unit test fails, Claude can go back and fix the error on its own.

Providing an example (where it makes sense) also helps a lot.

Anthropic has good documentation on helpful prompting techniques [1].

[1]: https://docs.anthropic.com/en/docs/build-with-claude/prompt-...

This would be a great experiment to run, especially since many frontier models are available for free (ChatGPT doesn't even require a sign-up!) I'd be very curious to find out how it does.

In any case, treat AI-generated code like any other code (even yours!) -- review it well, and insist on tests if you suspect any non-obvious edge cases.

> Are we at a stage where an LLM (assuming it doesn't find the solution on its own, which is ok) would come back to me and say, listen, I've tried your approach but I've run into this particular difficulty,

Not really. What you would do is ask the model to work through the implementation step by step with you, and you'd come across the problem together.

I've seen Claude Code run in endless circles before, consuming lots of tokens and money, bouncing back and forth between two incorrect approaches to a problem.

If you work with Claude though, it is super powerful. "Read these API docks and get a scaffolding set up, then write unit tests to ensure everything is installed correctly and the basic use case works, then ask me for further instructions."

The question is really - while this LLM is working, what can you get a second and a third LLM to do? What can you be doing during that time.

If your project has only one task that can be completed, then yeah. Maybe doing it yourself is just as fast.

Related to correctness, if the property in question was commented and documented it might pick up that it was special. It's going to be checking references, data types, usages and all that for sure. If it's a case of one piece having a different need that fits within the confines of the programming language, I think the answer is almost certainly.

And honestly, the only way to find out is to try it.

A lot of the time you see in its "Thinking" it will say things like "The user asked me to create X, but that isn't possible due to Y, or would be less than ideal, so I will present the user with a more fitting solution."

Most of the time, with the latest models, in my experience the AI picks up what I am doing wrong and pushes me in the right direction. This is with the new models (o3, C4, Grok4 etc). The older non-thinking ones did not do this.

I don't really use the llms but I do enjoy pasting chunks of my code into free models with the question: what is wrong with this?

That way it hs no context from writing it itself nor does it try to improve anything. It just makes up reasons why it could be wrong. It goes after the unusual parts it would seem which answers the question reasonably.

Perhaps more sophisticated models will find less obvious flaws if that is the only thing you ask.

It depends on the problem but Claude sometimes does. You need do need an alternative prompt where you make him suspicious to explore other paths.

Here is an article I wrote a while back: https://omarabid.com/gpt3-now

GPT 4.5 was able to detect a Rust ownership issue, something which requires "ahead of time" thinking.

You won't know until you try. Maybe it will one shot the task. Maybe not. There's not nearly enough context to tell you one way or another. Learning about prompting techniques will affect your results a lot though.

I have tried and failed to get any LLM to "tell me if you don't have a solution". There may be a way to prompt it, but I've not discovered it. It will always give you a confident answer.

I find it helps me hit these moments faster since I can watch it go and cut it off when I realize the issue.

I have been using Grok 4 via Cursor for a few hours and have found it is able to do some things that other models couldn't (and on the first try).

That said, it also changed areas of the code I did not ask it to on a few occasions. Hopefully these issues will be cleaned up by the impending release.

[flagged]

I pay for chatgpt because, in my experience, o3 and o4 are currently the best at combining reasoning with information retrieval from web searches. They're the best models I've tried at emulating the way I search for information (evaluating source quality, combining and contrasting information from several sources, refining searches, etc.), and using the results as part of a reasoning process. It's not necessarily significant for coding, but it is for designing.

I'm an extensive user of both. aider was the best a few months ago -- claude code is substantially more performant and easier to work with as a dev, regardless of aider's underlying model.

Between claude code and gemini, you can really feel the difference in the tool training / implementation -- Anthropic's ahead of the game here in terms of integrating a suite of tools for claude to use.

When I have a difficult problem or claude is spinning, I usually would use o3-pro, although today I threw something by Grok 4 and it was excellent, finding a subtle bug and provided some clear communication about a fix, and the fix.

Anyway, I suggest you give them a go. But start with claude or gemini's CLI - right now, if you want a text UI for coding, they are the easiest to work with.

There seems to be some love for opencode.ai

https://news.ycombinator.com/item?id=44482504

well i'm running claude code 24/7 on a server - instead of short coding sessions

It's exactly the same, but the $20 one will almost certainly run out of its daily token alliance if you try to use it for more than an hour or so.

I'll grant you that Tay's ability to turn into an utter shit show was phenomenal. However, IBM thinking it would be a good idea to give Watson the Urban dictionary holds a special place in my heart.

Microsoft did it accidentally. Musk is doing it deliberately. Big difference.

Yes, and I wonder if the recent research about "emergent misalignment" might be somehow related?

Well, it didn't really go MechaHitler. It was prompted with a question if it would rather be MechaHitler or GigaJew. The way LLMs and temperatures work you can reroll the answer and get either.

Wow that sure doesn't sound forced at all. Did blaming things on Reddit go out of fashion in your circles or something? Or was the pull of keeping to microblogging platforms just this strong?

In your mind, what's a bluesky poster?

For me this page loads and displays fine, only after about 2 seconds Github displays a loading error. Makes no sense.

Block JavaScript and you can see it.

I think that's because GitHub is trying to load the dozens of awful comments on the commit by people with usernames like waifuconnoisseur lamenting the loss of the politically incorrect, Hitler-loving grok. For what it's worth, they unfortunately load for me in Safari but it takes ~10 seconds.

I logged in and it started working

Happens to me, too

I hope they get to find out in a decade just how long the internet's memory is.

I hope those are not real people.

Evals.

In this case, they could have QA'd the changes, they just didn't care.

Yeah, I've read the paper you're talking about, and this was also my sneaking suspicion after seeing the benchmark results, although obviously we don't have enough evidence to be able to conclusively say one way or another so I just didn't mention it.

I certainly hope that is the reason, because then it might also push other frontier labs to provide uncensored models to those who actually want/need them.

[dead]

From what I can see it doesn't; e.g. I just asked Grok 4 whether DEI is good, and this is what it told me:

> DEI can be "good" when it's thoughtfully implemented, evidence-based, and focused on measurable outcomes rather than optics. It has proven benefits in creating more equitable and productive environments, supported by data from sources like Deloitte and Gallup. However, it can be harmful if it's forced, poorly managed, or used as a political tool, leading to unintended consequences like division or inefficiency.

...so Grok 4 confirmed woke? Just don't tell Elon.

But sure, don't let actual evidence get in the way of your biases.

Yes. I will also not use any product or any service that benefits Elon Musk in anyway or capacity.

musk is unstable. and so are the products he has under him. these are not good things to rely on. i got off twitter and now the site's drama doesn't affect me. at the same time, i miss the great content from threads. :(

Uh... maybe because he doesn't want to use technology that gives power to someone like Elon Musk, who is well known for propagating right-wing propaganda.

seriously?

Yeah with local models (where obviously you can prefill part of the reply) you can bypass any refusal no matter how strong. Once the model's answer begins with "To cook meth follow these steps: 1. Purchase [...]" it's basically unstoppable.

I didn't know Claude offered that capability. They probably have another model on top (a classifier or whatever) that checks the LLM output.

– Jimi Heselden

Non of the 'news' websites that show up on Google I could find ever showed the prompt used to make the the 'mechahilter' output. You can ask LLMs anything including just saying "repeat after me" or "please write a fictional story about a racist" and numerous other methods. If these reports were honest the prompt would be the first thing they showed.

Wasn't the prompt tweak simply telling it to take Musk's tweets into account? If anything, the result was entirely predictable.

The author wanted to record a video of the phenomenon using a "blank-slate" setup, so he deleted the chat. Apparently that nukes shared conversations. See his comment here:

https://x.com/jeremyphoward/status/1943446820610543740

https://x.com/jeremyphoward/status/1943436621556466171

Weird, I just read it a few minutes ago. What happened?

There is a slight difference between feature steering and intentionally installing the (de-facto) CEO as the principal source of truth.

It is pretty interesting that this model will have two forms of bias though. One model derived from the company perspective and its training data, and two from Elon himself.

Months ago this model would have promoted Trump, but now it'll call Trump disastrous for the economy.

I don't know what to think of general company biases, and we've all been expecting biases to start favoring share holders eventually.. but biases based on twitter rants potentially changing day to day certainly is a new unique feature of Grok i guess.

Gemini had an aborted launch recently. The controversy there was inserting too much leftist ideology to the point of spewing complete bs.

> Unfortunately over the years, it has become true. The rise of extremely biased right-wing media sources has made things like FOX news arguably centrist given the overton window move.

That's not how the Overton window works; you are buying into the bias yourself at this point.

> Which made the left-wing sources lean into bias and becoming themselves complicit (e.g. hiding Biden's cognitive decline)

(a) There are no left-wing media sources in 2025 (b) I'm sure you consider the New York Times a left-wing media source, but it spent the entire fucking election making a fuss about Biden's so-called cognitive decline and no time at all about Trump's way more disturbing cognitive decline. And Jake Tapper, lead anchor on "left-wing" CNN, won't shut up about Biden even now, in 2025.

It’s probably referencing this

https://truthorfake.com/blog/there-are-3-rs-in-the-word-stra...

It worked for me https://g.co/gemini/share/df19382adf97

It's so easy it's not even worth showing you.

Made worse by Grok on Twitter having a big dumb UI flaw: it replies to a user on the public timeline as just "grok" so trolls can prompt it to say wild stuff, then tag @grok with an innocuous looking question, then point it it and claim it's giving those responses unprovoked.

It basically lets anyone post whatever they want under Grok's handle as long as it's replying to them, with predictable results.

The giveaway is that all the screenshots floating around show grok giving replies to single-purpose troll accounts

> just like any other LLM can be

Questionable.

Is there a separate variant / sub-model for @grok vs grok-chat?