JevBench: Benchmarking decision models (Clef, pplx-decider and 15 more)

3 pointsposted a day ago
by paidev

2 Comments

verdverm

21 hours ago

Thoughts on adding benchmarks like coin flipping, dice rolls, and other well known outcome questions?

Some of the decision models do really poorly on this with the outcome probability and being very confident about the decision.