As someone who's been fact-checking a lot of the stuff I see on the internet for years, I love the idea.
The overall ratings by the algorithm seem pretty solid to me, and the explanations are very helpful for understanding the thought process behind the scoring.
That being said, some of the points in the "dubious" section seem a little questionable in my opinion. For example:
"'Applicable to US customers only.'
Reason: This is a marketing tactic to get US customers to sign up."
This doesn't seem like a marketing tactic, I think this is just relevant, factual information.
Do you have a strategy for tweaking the algorithm for things like this?
Maybe you could add an option to provide feedback on the points in the breakdown. Either a like/dislike system or just a "report" button with a prompt for optional feedback.
How does Credible handle novel ideas that don't yet have strong consensus but are still valuable?
appreciate the question and I understand the concern. Our algorithm will balance for this by surfacing and allowing discovery of novel ideas even if they don't have the strongest weight yet - as the source of that idea gets validated over time, their own track record will be used as a proxy for the quality of their future ideas.
The idea is that by relying on their eventual track record rather than just an individual claim, we can surface and reward novel ideas that have merit too.
How does the AI define Fact vs Opinion vs dubios in this context?
- Facts are claims that are citing matter-of-fact reporting, public data, sequences of events etc.
- Opinion statements are ones where its clearly the author's perspective or take on what's being reported on - the embedded reasons show the explanations in context.
- Dubious claims usually highlight either exaggeration of some sort, or manipulative argumentation including extreme emotional language, appeals to authority and such