folktheory
5 hours ago
Author here. What if you could see the bias of different academic researchers? Where their rankings overlap and differ?
We built a trust graph out of citations from Semantic Scholar, with an LLM classifying positive, negative, and neutral citations. We applied it to 176,234 researchers and 856,433 papers in 302 subfields.
Why might this matter to you? We think that in a lot of scientific fields there’s less consensus than it seems, and this basically starts to show where those gaps are the biggest. Citation count pretends to be a global ranking but ends up promoting citation-maxing, instead of good science. We think trust-maxing would be a move in the right direction. (if all you do is Goodhart being more trustworthy, you’re actually more trustworthy, so it’s prosocial.
What you can do: Try switching the viewer on any ranking, or compare two researchers side by side to see where they line up and differ.
Differences from a personalized PageRank:
- Includes positive and negative evidence - we treat distrust and uncertainty as first-class evidence.
- Asymmetric propagation of distrust - trust flows transitively, but distrust doesn't. Your friend's friend is your friend, but your enemy's enemy isn't your friend.
- New rank requires new independent trusted sources, not more citations
Disclosures:
- Citations are an imperfect proxy for trust
- LLM classification is lossy
- This is a self-run showcase of the engine behind UpTrust (a product we're building)
If you're a researcher whose field this covers, we’d love to hear from you. How well does this represent your actual view? How much better (or worse) is it than citation-count list, or whatever else is available?
If you have a dataset you’re curious to run this over, also let us know!