Show HN: jevals – replacing LLM judges with typed Jev decisions

47 pointsposted 3 days ago
by gbayomi
(github.com)

6 Comments

adityamishra241

2 days ago

Interesting idea. How do you handle cases where the decision depends on context that isn't captured by the typed decision type?

gbayomi

a day ago

It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!

jacksun788

2 days ago

Curious — how's this compare to laya?

gbayomi

a day ago

It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc