Baba Is Solved by Fable 5 and GPT-5.6 Sol, but at what cost?

2 pointsposted 11 hours ago
by stared

4 Comments

dash2

10 hours ago

Wouldn’t solutions be in the training data? The game is quite old.

stared

10 hours ago

See the "caveats" section.

It was also our initial assumption, but to our surprise there were no signs of models "knowing" the solutions (we investigated all trajectories). Compare and contrast with SWE-Bench Verified, for which models suddenly generate solutions, or at least, have substantial hints, https://openai.com/index/why-we-no-longer-evaluate-swe-bench....

sscaryterry

11 hours ago

I'd love to see Kimi K3 here :)

stared

11 hours ago

It is on the way! Sadly, this Kimi K3 API rate limits are devastating, at least on the OpenRouter. So far it solved all The Intro levels (as expected) - I am much more curious how it works with The Lake.