Jev beats Opus 5 at a Pokemon Showdown while being ~820x cheaper, ~10x faster

3 pointsposted 9 hours ago
by sidarya0

1 Comments

sidarya0

9 hours ago

Author here:

As someone who plays pokemon its actually super interesting to also note Jev's second and third options lining up with viable options and stratergies. Not sure if it was trained on multi-turn tasks or anything but super impressive capability.

Obviously just one battle as a demonstration - but in a finite state space this thing is cool.

Also un-aware if there are comparisons between language models at ability in pokemon-battle type settings - let me know if anyone knows of a benchmark here :)