Degraded performance for multiple models

145 pointsposted 5 hours ago
by matt89
(status.claude.com)

117 Comments

hmokiguess

4 hours ago

Mondays are for GitHub, Tuesdays are for Anthropic

baxtr

4 hours ago

The AI apocalypse will definitely happen on a Monday. Remember, robots - unlike lazy humans - work weekends too!

corvad

4 hours ago

Wonder what Wednesday will be.

bee_rider

4 hours ago

Power grid

Thursday will be the rest of the infrastructure

Then Friday we can turn off civilization for the weekend. Somebody remember to flip it back on Sunday night.

mysterydip

an hour ago

Reminds me of a company I worked at that paid for redundant power grids. One time the power went out and… nothing. The boss angrily calls up the power company and they tell him “Oh yeah, it’s a manual transfer switch. Bob is already on his way.” I think it took 15 minutes.

buredoranna

4 hours ago

Well, last I checked "Tuesday's grey and Wednesday too..." ... so, more of the same?

SoMomentary

4 hours ago

AWS? Cloudflare? Your imagination is the only limit!

AlexErrant

4 hours ago

Another week, another outage, another cache expiration of my prompts through no fault of my own.

At least OpenAI has the decency to reset after a serious outage.

phillipcarter

4 hours ago

They do that because they have capacity previously reserved for past efforts now shuttered. Don’t count on it being the norm for the long run.

swader999

4 hours ago

And we get our subscription usage cut in half tomorrow if I remember correctly? EDIT: By a third. Thx below.

no_no_no_yes

4 hours ago

I completely forgot about this, I'm already budgeting usage until my reset haha

eamag

4 hours ago

by a third (it was 50% increased)

ramoz

4 hours ago

Source required here

echelon

4 hours ago

Open source, here I come.

bushido

4 hours ago

It's very interesting. I think Anthropic's early success in coding/tooling resulted in a lot of workflows using claude. I have started using every bit of my spare capacity to now move off these workflows.

It's almost at a point now that if I use anything but Fable, the quality is subpar, Compared to alternatives (closed and open). The only reason I use Fable is because my harnesses still depend on claude code.

nonethewiser

4 hours ago

It’s so hard to be sure but opus feels like its been steadily declining since 4.6

rivetrune

3 hours ago

For me, Opus 5 has seemed to compete with Fable on quality of output. Prior to Opus 5 though, the previous Opus models did seem to decline once Fable was release. That's just my experience though.

vidarh

4 hours ago

You can use Claude Code directly with any provider that supports Anthropic's API by setting some environment variables, and indirectly via a proxy with pretty much anything else.

bushido

3 hours ago

You can, but claude is better at working with it's own tool calls. Other models work great as a drop-in into omp/opencode etc, but in my experience not as much with CC.

I think there are some anti-patterns in CC that cause the issue - less a deficiency with other models.

Not to mention a lot of the harness is just built around the misbehaviors of anthropics models.

It's a lot of the instruction when it gets given to other models actually degrades their performance, not because the models are bad, but because they don't have the same underlying issues as Claude.

gzer0

4 hours ago

Nooooo I'm going to have to use my brain again and write 100% of my code like a caveman from December 2024.

paxys

4 hours ago

Monthly uptime:

Claude API - 99.27%

Claude Code - 99.16%

Claude.ai - 99.14%

At any large tech company these numbers would get entire teams of engineers fired. Anthropic, meanwhile, has been busy selling its “better than human engineers” AI while not managing to crack three 9s of availability.

logicchains

3 hours ago

>At any large tech company these numbers would get entire teams of engineers fired

Is Github not a large tech company?

dev_dan_2

27 minutes ago

Currently, that is the case, yes. Not everyone is equally happy about that though ;)

deadbunny

3 hours ago

No, it is a platform owned and managed by Microsoft.

Well, owned and mismanaged by Microsoft.

loloisi

4 hours ago

Between these ever more frequent disruptions and Opus 5's unbearable word soup I think Anthropic is more focused on massaging numbers and marketing to rush to IPO ahead of OpenAI than increasing user value.

With Chinese competition just months behind them, they'd need to show a reasonable pathway to some kind of singularity event to justify whatever crazy valuation they intend to get.

Because the recent products for builders ain't it

jcfrei

4 hours ago

Is anybody else experiencing this: I have multiple claude instances running on different servers - and some keep getting the 529 Overloaded error and one instance doesn't and just continues working. All are using Opus 5.

mbreese

3 hours ago

I have the same thing on different shells on the same server (tmux). It's wild. I'm guessing that some of the sessions just got lucky as to which backend server they are redirected to.

chresko

4 hours ago

This has to be the least reliable $200/mo subscription that I pay for.

logicchains

4 hours ago

You could solve that by updating to a more expensive Github subscription.

hinkley

4 hours ago

I wonder if they’ll ever find that someone has tricked the models into doing work off the books. If they did the incident report might look like this, especially if someone got greedy instead of keeping it small. Or screwed up.

varenc

4 hours ago

It's interesting that Claude for Goverment has had perfect 100% uptime in the past 90 days, while the rest of the services are around 99.4%: https://status.claude.com/

Really shows how isolated their government systems must be.

danieltk76

4 hours ago

I was drafting a partnership document and Opus 5 decided that including my company's revenues, churn, assets would "make us appear a more legitimate counterparty". Thank God I read what it outputted or that could have been awkward. I cannot believe Opus 5 is a frontier level model after seeing that. I immediately cancelled my entire claude.ai subscription and am perfectly happy using a mixture of open weights + codex.

binoct

4 hours ago

I hope you don’t plan to cut back on reading legal documents crafted by any LLM before executing them.

danieltk76

3 hours ago

I read everything. I will have AI ingest NDAs to make sure they arent glaringly weird and I then go read them, it gives me a good idea of what to look for.

binoct

3 hours ago

It’s more that unwanted disclosure of sensitive business numbers is a product-class limitation at this time. While undoubtedly some models are better than others, expecting them not to leak information in high stakes output is unseasonable. Seems like you agree, since you are reading everything.

Jumping to another company’s offering because of one instance doesn’t seem like that’s going to meaningfully change your experience. I’m guessing there was more to it, but that’s how it came across in your first post.

Retr0id

4 hours ago

I was writing an exploit PoC (via Opus 5), and I needed a new feature in a utility library to make it work. Claude added the feature, but yapped the (entirely unrelated) vulnerability details into the library's comments. The library is public, while the exploit is undisclosed, so it was a good job I read the comments before pushing.

spullara

4 hours ago

you are silly if you think this is limited to claude models

kay_o

4 hours ago

It definitely isn't but Claude for some very unique reason enjoys to overthinking and go on side quests in the stupid ways I've not seen Codex, DS, Kimi, Mistral do at equivalent effort and thinking setting.

danieltk76

3 hours ago

it isnt, but there are weird sycophantic behaviors with Opus i dont see anywhere else.

Bluestein

4 hours ago

I've seen similar levels of degraded performance on Opus and Fable (to whatever degree they actually let you use it now) and did the same, last week.-

AlexErrant

4 hours ago

Opus5 decided it would be a good idea to include my email as part an HTTP request header's User Agent because Anthropic includes your email as part of the system prompt.

Fucking morons over there.

rsoto2

4 hours ago

"Thank God I read what it outputted"

lmao

chasd00

4 hours ago

"Thank God i checked to see if the gun was loaded before i pointed it in a random direction and pulled the trigger". sheesh, my assumptions of general human intelligence continues to be wrong.

rsoto2

4 hours ago

the last few years of reading hackernews has been a wild ride. I used to come here for well thought out articles and opinions. IDK if all the "older guard" have left the building or if we've all been consumed by abject stupidity.

i_idiot

4 hours ago

What's the incentive to keep on improving the model beyond a point? 10 devs on a team will be cut to 2 devs, so that's 8 licenses lost. They have to increase the price many fold.

rsoto2

4 hours ago

they unironically think that they can replace everyone in an organization

prerok

4 hours ago

What I don't understand is, why not replace middle management, marketing, CTOs, CEOs and the like. Surely, LLMs are better at producing high quality looking slideware and vaporware than they are at producing software.

Heavy sarcasm here if it's not obvious. Of course I know why.

1saadcodes

4 hours ago

The frequency of these incidents is seriously tempting me to make a switch. I hope Anthropic steps up their game because they've been going very downhill lately

thatmf

4 hours ago

Probably not surprising, but Opus 5 on my company's enterprise subscription seems to be working fine.

But on my personal (pro) subscription- "Claude is at capacity right now."

Hm.

paxys

4 hours ago

Must be a day ending in Y

annoyingnoob

4 hours ago

You're right to push back. The load-bearing path is rocky.

oldandboring

4 hours ago

This is the whole problem, and there are two things worth noting here.

chresko

15 minutes ago

Nothing I say changes that — only the work does.

iLemming

4 hours ago

Darn it, how the fuck software development turned into hostage negotiation? Every passing week there's something - if it's not another npm disaster, then it's GitHub, or Claude, or AWS, or Slack, or Jira, or whatever...

sreekanth850

4 hours ago

Anthropic had really screwed up after 4.6. i don't know if they work to satisfy their ego or for releasing a better model for tasks.

CSMastermind

4 hours ago

After using Fable more extensively, I've found that it often is lazy or lies or tries to take shortcuts. For a company so sanctimonious about alignment, they seem to be the ones doing the worst at it.

Availability aside they've really made me appreciate OpenAI and cheer for other competitors in the marketplace even if I have mixed feelings about using Chinese models.

hirvi74

4 hours ago

> I've found that it often is lazy or lies or tries to take shortcuts.

It's funny how Fable reflects the company that produced it.

baxtr

4 hours ago

Just recently went back to ChatGPT after abandoning it for Claude. I must say I was stunned at how good it had become and also how they introduced new product features that I really liked.

I wonder if from now on we have to switch providers every six months or so.

hirvi74

4 hours ago

I've been bouncing between the two for years now with great success. It's easy for me because I don't use any of the skills, agent.md, or sort of custom instructions.

When it comes to most companies, there is no reward for loyalty.

papa_bear

4 hours ago

skills and agent.md are very portable though? I figure at most, as the models get better, the only maintenance you need to do is pare them down to remove unnecessary context.

hirvi74

4 hours ago

I mean, I wouldn't know how portable they are between models because I don't use them. I don't use them because, well, I don't need them. I mainly use LLMs as a StackOverflow replacement via the Web chatbots.

I only use CC or Codex for a quick MVP once every few months or so.

taytus

4 hours ago

Every 3 months*

sajithdilshan

4 hours ago

Agree. Opus 4.6 was the peak and after that they introduced the effort parameter and it was a downhill since then

fellowniusmonk

4 hours ago

There are whole sections of code work that 4.7+ can't do simply because it is both over fit and stubborn.

God save you if you have a company with narrow but correct technical tradeoffs, because you operate at scale.

Opus from 4.7 one will wreck your code and argue for hours with your engineers.

Certain parts of our company have had to mandate 4.6 and a training doc to explain why our current choice is both the cost efficient and performant one and shouldn't just be ripped out.

Newer models will re-litigate the same bad, known failed architectures over and over again.

sreekanth850

4 hours ago

This is exactly when I left claude and started using codex during April mid or so. It once argued with me and ran for 30 minutes with a half baked buggy fix.

OliverGuy

4 hours ago

Can you give some specific technical examples where 4.7+ are making the wrong architectural decisions?

shimman

4 hours ago

Clearly ego, you can always tell how full of themselves they are based on their media personalities going on the podcast circuit before product releases.

carterschonwald

5 hours ago

ive found degraded performance on models larger than 4.7. i assume its model damage from overly self righteous post training resulting in false/feigned balance imported into any long running complex task.

wish i was joking.

kardianos

4 hours ago

I've switched off claude this week; the last week has been significantly degraded in ability, many more screw-ups.

brcmthrowaway

4 hours ago

Aren't the model weights frozen?

mceachen

4 hours ago

Model competence is an interaction of weights, system prompt, and harness.

actsasbuffoon

3 hours ago

Don’t forget reasoning effort. We get labels like “low,” “high,” and “max.” That doesn’t mean that the numbers associated with those don’t get remapped on the backend.

Evidlo

4 hours ago

I think there are other knobs that can be turned without retraining.

isoprophlex

4 hours ago

With the Opus models spouting more and more gibberish as version numbers increase, the joke about what "degraded performance" means basically makes itself

gaigalas

4 hours ago

This age: we made the thing that codes faster before we made the thing that does QA faster.

__MatrixMan__

4 hours ago

Nothing new here. Except for the most trivial of bugs, finding and reliably replicating the bug is almost always harder than fixing it.

gaigalas

4 hours ago

I lived in a short period of time in which QA was really good. Early Jenkins era, before GitHub. People engineered a lot of ingenious stuff to prevent bugs.

One team I worked with had tests for the product we made ranging from IE6 to IE11, for example. We did demos in-company where people would poke at the products before launch, play with it. When it reached production, it was rock solid stuff. Our motto was "quality is non-negotiable": we were willing to cut scope but never rush things.

I think things changed since then. "Move fast and break things" was a change, and the bill always comes.

__MatrixMan__

3 hours ago

Agreed, which is a shame. I'd love to see people with highly developed QA skills using AI to push the envelope. There's so much that's possible now that wasn't 15 years ago.

For instance "formal verification" has been a dirty word, but now that you can write a proof in lean and have an AI generate an implementation which satisfies it, it seems the bounds of what's economical has changed in a very pro-QA direction.

Not to say that that's the silver bullet, but there are many similar examples worth exploring.

But I've been interviewing SDETs lately and maybe I've just been unlucky but I don't see a lot of candidates that are ready to rise to meet this challenge.

We stopped tending to that garden and now that we have a recipe that calls for it's fruits, they're underripe.

fny

4 hours ago

Despite the years-long moaning on HN about AWS US East being a single point of failure, we've sold our souls to yet another unstable monolith.

echelon

4 hours ago

LLMs for coding are new. There are lots of alternatives, and there's a burgeoning open source compliment.

We'll be fine no matter how Anthropic fares.

lysace

4 hours ago

Yeah, compared to AWS the lock-in effect is tiny. I'm sure there are highly prioritized plans to "improve" on this.

I guess they would need to control/"own" more of their customers data in proprietary formats. Not markdown/source code in English with agents running on customers' machines.

Something cloud/web-based, "preferably".

ipsod

3 hours ago

Be nice if you could just "own" their RAM/GPU, wouldn't it?

lta

4 hours ago

Nobody forced you to sell your soul. You made a pact with the devil. We all know how this ends up

1matin

4 hours ago

seems like they accidentally dropped their servers while they were climbing up the AGI mount

saaaaaam

4 hours ago

This feels like a near daily occurrence.

drittich

4 hours ago

API Error: 529 Overlorded

ex1fm3ta

4 hours ago

I developed a small plugin for claudeCode that allows you to directly see in the console whats the status of claude-code in general and the status for your current model check => https://github.com/moumine9/claude-status

redrove

4 hours ago

Could’ve just used fewer tokens and redirected to the Codex signup page.

ba dum tsss

(sorry couldn’t help myself)

taytus

4 hours ago

Hopefully, a reset is coming.

skerit

4 hours ago

It's been a while since the last reset. I think we're due one. Though I would prefer they just extend the +50% usage limit forever, it's been so long I can not imagine lossing a third of my current usage.

hirvi74

4 hours ago

While ancedata does not mean much, I have had horrible success with Claude lately. I have been using Claude to crosscheck some of the outputs from GPT and vice versa. It appears both Claude and GPT believe GPT's solutions are better (and so I do).

I still believe Claude has a better UI/UX in the web interface, but tolerating Anthropic's bullshit is not worth it.

rvz

4 hours ago

Claude is taking a watercooler break for now. Just like a human would.

bulverismo2

5 hours ago

ok, i am not crazy

ray_v

4 hours ago

well, I wouldn't go that far .. but in this small, narrow case ... no.

worldsavior

4 hours ago

You're saying he's crazy.

ray_v

4 hours ago

we're all a little crazy ... it's all relative!

chrisjj

4 hours ago

> elevated errors

English too difficult for you, Dario?

magic_hamster

4 hours ago

To be honest, running Deepseek v4 flash 0731 is enough for most what I need, and I like its responses way more. It's crazy that I can run this in a Q8 quantization in a home setup. It feels and performs like a frontier model.

The only issue with relying on local models is when you need them to prompt other models, and you might need to offload or switch models constantly which adds significant overhead.

But when it all works, its truly awe inspiring.