I agree with you, I consistently find Gemini 2.5 Pro better than Claude and GPT-5 for the following cases:

* Creative writing: Gemini is the unmatched winner here by a huge margin. I would personally go so far as to say Gemini 2.5 Pro is the only borderline kinda-sorta usable model for creative writing if you squint your eyes. I use it to criticize my creative writing (poetry, short stories) and no other model understands nuances as much as Gemini. Of course, all models are still pretty much terrible at this, especially in writing poetry.

* Complex reasoning (e.g. undergrad/grad level math): Gemini is the best here imho by a tiny margin. Claude Opus 4.1 and Sonnet 4.5 are pretty close but imho Gemini 2.5 writes more predictably correct answers. My bias is algebra stuff, I usually ask things about commutative algebra, linear algebra, category theory, group theory, algebraic geometry, algebraic topology etc.

On the other hand Gemini is significantly worse than Claude and GPT-5 when it comes to agentic behavior, such as searching a huge codebase to answer an open ended question and write a refactor. It seems like its tool calling behavior is buggy and doesn't work consistently in Copilot/Cursor.

Overall, I still think Gemini 2.5 Pro is the smartest overall model, but of course you need to use different models for different tasks.

Gemini might be a good model, it is _incredibly_ shit in tool calls and it has this incredibly tendency to multishot itself to death. When using their own gemini-cli tool, it's impossible to take it seriously, it's that bad.

For example:

If it makes a mistake, it'll keep on making the exact same mistake, and it'll act all cute like "Oh no, look at the mess I'm making". Some people say this is just a side effect of long contexts degrading performance, but it can happen even when 98% of the context is unused.

I'm also using a Ghidra MCP server to decompile some binaries. Claude is great with this. It really gets it and is able to use it properly. Gemini? Just one or two tool calls, and it'll start repeating the output of the tool calls for some reason.

Gemini also often isn't able to properly call the MCP tools. It just outputs the tool call as JSON text to the user.

Gemini-cli isn't even able to properly resume previous chat sessions. You have to actively save chats in order to resume them. Being able to simply resume the previous conversation using a flag like `--resume` or `--continue` has been a feature request since day one, and similar issues keep popping up weekly on the Github issue list. There are even multiple pull requests for this feature, but it's like nobody over there gives a damn.

Yeah it’s really good. A few weeks ago, some third party script was messing with click events of my react buttons so I figured I should just add a mousedown even to capture the click before the other script. It was late at night and I was exhausted so I wanted to do a quick and dirty approach of simulating a click after a few ms after the mousedown even. So I told Gemini my plan and asked it to tell me the average time in ms for a click event in order to simulate it… and I was shocked when it straight up refused and told me instead to trigger the event on mouseup in combination with mousedown (on mouse down set state and on mouse up check the state and trigger the event). This was of course a much better solution. I was shocked at how it understood the problem perfectly and instead of giving me exactly what I asked for it gave me the right way to go about it.

I find Claude and Gemini to be wildly inferior to ChatGPT when it comes to doing searches to establish grounding. Gemini seems to do a handful of searches and then make shit up, where ChatGPT will do dozens or even hundreds of searches - and do searches based on what it finds in earlier ones.

We extensively benchmark frontier models at $DAYJOB and Gemini 2.5 is the uncontested king outside of a few narrow use cases. Tracks with the rumor that Google has the best pretraining and falls short only in tuning/alignment. Eagerly anticipating Gemini 3 as 2.5, while king of the hill, still has lots of room for improvement!

Edit: narrow use cases are roughly "true reasoning" (GPT-5) and Python script writing (the Claudes)

Agreed, and its larger context window is fantastic. My workflow:

- Convert the whole codebase into a string

- Paste it into Gemini

- Ask a question

People seem to be very taken with "agentic" approaches were the model selects a few files to look at, but I've found it very effective and convenient just to give the model the whole codebase, and then have a conversation with it, get it to output code, modify a file, etc.

I do very different work, or try to - historical HTR is unfortunately so bad even with the top models that the results aren't useful, but I keep trying new models. But there's a historian on substack (Mark Humphries) who's also trying new models, and he also thinks he's gotten Gemini 3 output in A/B tests. He's very impressed with it:

https://generativehistory.substack.com/p/has-google-quietly-...

I can attest to what he's saying that existing models are especially useless on tabular handwritten data such as ledgers.

Interesting. Not my experience at all. It makes mistakes that GPT-4 used to make: mixing languages (using Python syntax in C++ when I never asked any Python questions), imagining API calls that don’t exist in Google’s own products, writing 50 lines of C++ then inserting pseudo code or completely broken syntax.

> consistently found Gemini to be better than ChatGPT, Claude and Deepseek

I used Pro Mode in ChatGPT since it was available, and tried Claude, Gemini, Deepseek and more from time to time, but none of them ever get close to Pro Mode, it's just insanely better than everything.

So when I hear people comparing "X to ChatGPT", are you testing against the best ChatGPT has to offer, or are you comparing it to "Auto" and calling it a day? I understand people not testing their favorite models against Pro Mode as it's kind of expensive, but it would really help if people actually gave some more concrete information when they say "I've tried all the models, and X is best!".

(I mainly do web dev, UI and UX myself too)

Thats been my experience also. I’m able to get better answers related obscure tech we use at work on gemini than chatgpt or any other llm.

I use LLMs a lot for health related things (e.g. “Here are 6 bloodwork panels over the past 12 months, here’s a list of medical information, please identify trends/insights/correlations [etc]”)

I default to using ChatGPT since I like the Projects feature (missing from Gemini I think?).

I occasionally run the same prompts in Gemini to compare. A couple notes:

1) Gemini is faster to respond in 100% of cases (most of my prompts kick ChatGPT into thinking mode). ChatGPT is slow.

2) The longer thinking time doesn’t seem to correlate with better quality responses. If anything, Gemini provides better quality analyses despite shorter response time.

3) Gemini (and Claude) are more censored than ChatGPT. Gemini/Claude often refuse medical related prompts, while ChatGPT will answer.

Gemini was good when the thinking tokens were shown to the user. As soon as Google replaced those with some thought summary, I stopped finding it as useful. Previously, the thoughts were so organized that I would often read those instead of the final answer.

It's definitely not just you. Gemini is the only one that's consistently done anything actually useful for me on the kinds of problems I work on (which don't have a whole lot of boilerplate code). Unlike the other models it occasionally catches real errors in complex reasoning chains.

It was also only model that was good with coming with something creative at all, like brainstorming startup ideas etc. for me - they were grounded as in reasonable compared to other I tried

So far, I have had a very good experience using Gemini Live with the camera turned on. Just today, I wanted to find out the name of a spare part inside a bathroom faucet. First, Gemini said it was a thermostatic cartridge, but I responded that it couldn't be, as it doesn't control temperature. Then it asked me what it did, and I said it has a button that controls the flow of water between the tap and shower. It correctly guessed that it was a diverter cartridge.

Yes. Jules even writes more testable code, but people I know regularly use codex because it will bang its head against the wall and eventually give you a working implementation even though it took longer.

In programming accuracy, these past few weeks, chatgpt seem to have improved while Gemini went the other way... or maybe it is just simply relative and only one of them changed... For me on a very custom and complex codebase.

Can't believe I am paying for multiple llms...

I mostly use Gemini for everyday Q/A and research type stuff. I find it's pretty accurate and gets straight to the point. I mostly use Claude and very recently Codex for systems software dev. I'm very interested to see what changes.

I'm wondering how these models are getting better at understanding and generating code. Are they being trained on more data because these companies use their free tier customers' data?

What's your use case? We've found Gemini to work well with large context windows, but it sucks at calling MCPs and is worse at writing code

Looking at the responses. How the F have people so wildly different opinions on the relative performance of the same systems?

I do feel like LLM's start to match certain personalities and characteristics of users which makes them unattractive to others. I assume we will need a better kind of personalization layer in the future or the ecosystems will start to drift. For example I very much feel like grok fits my thought patters by far the best.

I've seen many comments that they are great for OCR stuff, and my usecase of receipt photo processing does have it doing better than ChatGPT , Claude or Grok.

Gemini is the only plan I have not replaced. Claude and ChatGPT I will switch to depending on the ability of the coding agent, but Gemini is still my favorite for general information and especially for writing assistance.

I like Gemini 2.5 as a chatbot, but it has been mostly useless as an agent comparing to Claude Code (at least for my complex tasks)

For pure text responses, agree 100%. Gemini falls way short on tool/function calling, and it's not very token-efficient for those of us using the API. But if they can fix those two things or even just get them in the same ballpark like they did with flash and flash-lite, it would easily become my primary model.

I use it a lot for ideation on things like strategy and creative tasks. I've found Gemini to be much better than Claude, but I almost want to switch back to Claude because of the "Projects" primitive where I can add specific context to the project and ask questions within that project, and switch around to different projects with different context. Gemini just wants to take all context from everything ever asked and use it in the answers, or I can add the context in the individual prompt, which is tedious.

What application are you using it with? I find this to be very important, for instance it has always SUCKED for me in Copilot (copilot has always kind of sucked for me, but Gemini has managed to regularly completely destroy entire files).

How often do you encounter loops?

I find the sheer amount of glazing Gemini does unbearably, so I pretty much avoid using it. It’s just an unreal amount compared to GPT-5 or Claude.

Gives it a stack trace or some logs and Gemini treats it like the most amazing thing ever and throws a paragraph in there praising your skills as if you were a god.

I completely disagree. For me the best for bulk coding (with very good instructions) is Sonnet 4.5. Then GPT-5 codex is slower but better guessing what I want with tiny prompts. Gemini 2.5 Pro is good to review large codebases but for real work usually gets confused a lot, not worth it. (even though I was forced to pay for it by Google, I rarely use it).

But the past few days I started getting an "AI Mode" in Google Search that rocks. Way better than GPT-5 or Sonnet 4.5 for figuring out things and planning. And I've been using without my account (weird, but I'm not complaining). Maybe this is Gemini 3.0. I would love for it to be good at coding. I'm near limits on my Anthropic and OpenAI accounts.

I prefer it too, but I find it a bit too wordy. It loves to build narratives. I think this is a common theme with all of Google’s LLMs. Gemma 27B is by far the best in its class for article generation.

I tend to find it competitive, but slightly worse on average. But they each have their strengths and weaknesses. I tend to flip between them more than I do search engines.

I find Gemini incomparable to Claude, especially for coding. The chat UI is ok, but Claude Code eats the CLI for breakfast

You’re definitely not the only one.

My results with Gemini are consistently better and usually also more reliable than other LLMs.

But tbh I prefer the UI of ChatGPT.

It has been consistently better at least with C++ ever since like o3, in my experience. The last ChatGPT model I loved was o1-pro.

I use the models via Cursor and I prefer the output and speed of Claude Sonnet reasoning mode over Gemini 2.5 Pro. But my work is heavily in ETL/ELT processes and backend business processes. So maybe if I was doing a lot of web stuff it would be different.

I had the same feeling when 2.5 pro was initially released, but it seemed like after a while they quantized the model.

I've found it to be excellent but 2.5 seems to experience context collapse around 50k tokens or so. At least that is my findings when using it heavily with Roo Code

I've since switched to Claude Code and I no longer have to spend nearly as much time managing context and scope.

Gemini is the only model that can provide consistent solution to theoretical physics problems and output it into LaTeX document.

Depends on the task, our tastes, and our workflow. In my case:

For writing and editorial work, I use Gemini 2.5 Pro (Sonnet seems simply worse, while GPT5 too opinionated).

For coding, Sonnet 4.5 (usually).

For brainstorming and background checks, GPT5 via ChatGPT.

For data extraction, GPT5. (Seems to be the best at this "needle in a haystack".)

I used Gemini at work, and would probably agree with your sentiment. For personal usage though, I've stuck with ChatGPT (pro subscriber).. the ChatGPT app has become my default 'ask a question' versus google, and I never reach for Gemini in personal time.

You are not alone, I got betetr result with Gemini free tier. Use their Code assist in VS code.

> I've consistently found Gemini to be better than ChatGPT [ because ] Google has crawled the internet so they have more data to work with.

This commonly expressed non-sequitur needs to die.

First of all, all of the big AI labs have crawled the internet. That's not a special advantage to Google.

Second, that's not even how modern LLMs are trained. That stopped with GPT-4. Now a lot more attention is paid to the quality of the training data. Intuitively, this makes sense. If you train the model on a lot of garbage examples, it will generate output of similar quality.

So, no, Google's crawling prowess has little to do with how good Gemini can be.

Gemini is theoretically better, but I find it's very unsteerable. Combine that with the fact it struggles with tool use and character-level issues - and it can be challenging to use despite being "smarter".

I gave up on Gemini because I couldn't stop the glazing. I don't need to be told what can incredible insight I have made and why my question gets to the heart of the matter every time I ask something.

gemini used to be the top for me until gpt-5 (web dev with html/js/css + python) ... and also with gpt-5 around it's doing its job, but it's really slow.

Definitely subjective, I find it significantly worse than GPT or Claude. Particularly for software systems design and coding problems.

I find Gemini to be too verbose in its responses.

We've moved to it for our clinical workflow agents. Great quality, better pricing and performance compared to Anthropic.

Yeah for my agent gemini 2.5 flash performs similar in quality to gpt4.1 and it's way faster and cheaper.

I find Gemini excels at greenfield, big picture tasks. I use Sonnet and Codex for implementation.

I am curious what your background is. I also almost exclusively use Gemini 2.5, and my PhD colleagues in comp sci do the same. However it seems like the general public, or people outside this bubble are more likely to use ChatGPT or Claude.

I wonder if it has something to do with the level of abstraction and questions that you give to Gemini, which might be related to the profession or way of typing.

I swear HN commenters say this about every frontier model.

I use GPro 2.5 exclusively for coding anything difficult, and Claude Opus otherwise.

Between the two, 100% of my code is written by AI now, and has been since early July. Total gamechanger vs. earlier models, which weren't usable for the kind of code I write at all.

I do NOT use either as an "agent." I don't vibe code. (I've tried Claude Code, but it was terrible compared to what I get out of GPro 2.5.)

Gemini specifically resets your context after a certain time. I have observed that it will basically clear out your context in a reasonable length session, which neither ChatGPT and Claude do.

Flushing or flattening down context saves costs. For that reason I never trust it with long research sessions. I would not be shocked if after 30 minutes they run a prompt like this:

And now reduce context history by 80%

This can very easily measured too, and would certainly expose the true feature set that differentiates these products.

Why would you use Gemini instead of something purpose-built for you, like Replit?

Agreed. There seems to be some very strong anti-Google force on HN. I guess there's just a lot of astroturfing in this area.

you're not in the minority, there's just intense fanboyism on Hacker News to promote OpenAI, because it serves the whole "LLM revolution" schtick better

Gemini has been dominating the field for about a year now, but I suppose Google is bit boring cause they just do things well

I understand that hyping is the career of a lot of people, but it's a little annoying how every Twitter link posted here is full of "IT'S A GAME CHANGER!!! NOTHING IS THE SAME ANYMORE!!! BRACE FOR IMPACT!!!" energy. The examples look great, but it's hard to ignore the unprofessional evaluation that you described.

What about other animals, like a giraffe on a snowmobile?

Benchmark is (finally) broken!

holy smokes, i wasnt expecting the equivalent of a piece of art

That's good?

Looks like complete crap to me.

Hey, this has been my experience, too! I like Gemini because I’ve told it the tone and style I like my answers in and the first answer is very, very on point with that. But several times I’ve noticed that if I ask follow-up questions, the style immediately changes for the worse, often no longer following my preferences. I’ve also noticed that in follow-ups it makes really bad analogies that are not suitable at all for the kind of audience that the first response is catered to. I’ve been clicking the thumbs-down button every time I’ve seen this and commenting on the change in style and quality, so hopefully the training process will ingest that at some point.

> The longer a chat goes, it gets worse very quickly.

This has been the same for every single LLM I've used, ever, they're all terrible at that.

So terrible that I've stopped going beyond two messages in total. If it doesn't get it right at the first try, its more and more unlikely to get it right for every message you add.

Better to always start fresh, iterate on the initial prompt instead.

My take is no one really cares about generating SVG, but it's a structured "code" format with very direct visual results. I can't look at 3 piles of code and instantly tell which is best (assuming minimum competence) , but I can judge the SVG outputs very easily. As a quick shot it gets a point across faster and with easier comparison. As a technical comparison it's not so strong, but thats harder to do and judge and less fun to read.

One of my co-founders lost the SVG of our startup logo, and the designer who helped us was away on vacation. I really wanted to experiment with some logo animations for an upcoming demo, so I decided to take matters into my own hands.

I grabbed a high-quality PNG, gave it to ChatGPT, and managed to recreate the SVG from the image, after quite a bit of prompting and tweaking. But it worked out great!

It goes back to Sparks of AGI [0] unless I am mistaken. Can recommend the talk, one that has stayed in the back of my mind since I first saw it two years ago. Personally, still have major reservations about throwing claims of intelligence or understanding around, but I do agree that SVG code generation can be a very effective source to get a quick and easy to present understanding of a models ability to output code with a rather open ended prompt that needs a high degree of coherence and were a lot of layers depend/build on each other.

Helps that these are eye catching (literally as the output is visual) and easy to grasp. Same reason a lot of hype is created around the web desktops.

[0] https://youtu.be/qbIk7-JPB2c?si=_TNRrxN-_5FOlfy5&t=1342

It's obviously a pointless benchmark-but it's fun so people like doing it

Gemini 3.0 isn't broadly available inside Google. There's are "Gemini for Google" fine-tuned versions of 2.5 Pro and 2.5 Flash, but there's been no broad availability of any 3.0 models yet.

Source: I work at Google (on payments, not any AI teams). Opinions mine not Google's.

Hate to spoil this excitement, but we at Google do not have Gemini 3 available to us for use in Vibecoding.

Which should surprise no one. LLMs are reaching diminishing returns, unless we find a way to build GPUs more cheaply.

This was cool: https://codepen.io/ChetasLua/pen/yyezLjN

Somewhat amusing 4th wall breaking if you open Python from the terminal in the fake Windows. Examples: 1. If you try to print something using the "Python" print keyword, it opens a print dialog in your browser. 2. If you try to open a file using the "Python" open keyword, it opens a new browser tab trying to access that file.

That is, it's forwarding the print and open calls to your browser.

I'm no financial advisor but I can tell you that it's not a financially sound decision to buy stock based off of speculative hype Twitter posts.

But you do you if you have "fun money" to throw around!

I find verbosity annoying. I prefer depth/accuracy/structure without extra words.

Maybe Gemini still wins on that front anyway.

It's based on leaked p photo of a deck.

Bet on it!

https://manifold.markets/ItsMe/gemini-3-releases-october-22

My benchmark only gives them one chance.

I've also tried a variant where the vision models get fed a rendered version and have up to three attempts to make it better. It didn't seem to produce better results, to my surprise.

Have you tried Claude Code, Cursor, Codex CLI, Gemini CLI, etc?

AI coding is in many ways antithetical to great software engineering.

It is the current spear-edge of the investor pressure to ship products faster, and monetize users more aggressively, all at the cost of quality, reliability, ethics, security.

If you, as a software engineer, once held an ideal about programming as an art or craft, AI coding flies in the face of all that.

It turns out that maximising for short-term profit leaves many other objectives behind in its wake.

Conservatisme, resistance to change, fear of losing the skills and becoming irrelevant.

Training their replacements?

I've found Gemini to be much better at completing tasks and following instructions. For example, let's say I want to extract all the questions from a word document and output them as a CSV.

If I ask ChatGPT to do this, it will do one of two things:

1) Extract the first ~10-20 questions perfectly, and then either just give up, or else hallucinate a bunch of stuff.

2) Write code that tries to use regex to extract the questions, which then fails because the questions are too free-form to be reliably matched by a regex.

If I ask Gemini to do the same thing, it will just do it and output a perfectly formed and most importantly complete CSV.

For writing code at least this has been exactly my experience. GPT5 is the best but slow. Sonnet 4.5 is a few notches below but significantly faster and good enough for a lot of things. I have yet to get a single useful result from Gemini.

Yep, I agree. Gpt 5 thinking is by far the best reasoning model ime. Gemini 2.5 pro is worse in pretty much everything.

This has been pretty much exactly my experience.

My honest belief is that they’re are bots. I also find 2.5 worse.

2.5 Pro is limited to 100 request per day every where I think. My Gemini CLI is authed through the Google Account (not API key) and after 100 requests it switches to Flash, API keys are also limited to 100 requests each (and I think there's a limit on free keys now as well)

Here's your controller, bro.