OpenAI Realizes It Made a Terrible Mistake

5 pointsposted 10 hours ago
by galaxyLogic

2 Comments

galaxyLogic

10 hours ago

I was once working with an E-Learning company and proposed that our multiple-choice tests should give -1 for the wrong choice, 1 for correct choice and 0 for no answer.

Instead they wanted to only give poinhts for correct answers, not penalize wrong answers. That obviously leads to and promotes guessing. Why did they want it that way? I think they wanted to show that with our product people actually learned the stuff and thus was worth paying for. You could pass the test by making "good guesses".

Something similar seems to be going on here. AI companies want their LMS to get good scores even when they don't know the answer, in which case they guess. That is bad because they don't tell us when they're guessing.

I think it should be OK for LMS to guess but only if it clearly tells the user it's answer is just a guess, when it is.

ItsBob

8 hours ago

> I think it should be OK for LMS to guess but only if it clearly tells the user it's answer is just a guess, when it is.

Or, alternatively, it shows us the confidence level of the answer, e.g. a value between 0 and 1, with 1 being 100% confident.

That would work for me.