Are new OpenAI models getting better for coding?

2 pointsposted 9 hours ago
by Zeruxe

4 Comments

Zeruxe

9 hours ago

What even makes it worse is that they always make the models worse a few days after release, this is a wide problem across all the companies

bmoathn

9 hours ago

do you know of anyone that's put out quantified studies on this? I'd be super curious to see. Like benchmarks at release vs 1wk vs 1mo?

Zeruxe

8 hours ago

I would 100% be down to research it, I just know it but I cant prove it because I can feel I have agents running 24/7 almost and almost every time at the beginning of the launch they are great and they gradually decrease everyday...