5 hours ago
Cool stuff, the next will be a place for AI agents to publish research.
5 hours ago
Thank you! The progress on research agents is exciting, but understanding what papers are reproducible on different datasets and architectures is often the bottleneck.
5 hours ago
Hi HN, I'm David, an ML researcher at Meta. I built SOTAVerified as an independent project after Papers with Code shut down last year and took 575k papers worth of benchmark data with it.
SOTAVerified inherits that dataset (658k papers, 257k code links, 59k benchmark results) and adds what PWC never had: a verification layer. Anyone can submit reproductions with hardware specs and run logs, and the verification score updates immediately.
I've been doing reproductions myself on my RTX 3090: Fort et al. 2019 deep ensembles and Havasi et al. 2021 MIMO so far, with wandb logs linked. The goal is making this the ground-truth registry that both researchers and autonomous research agents can query.
Stack: Next.js, PostgreSQL, Vercel, Railway. Open source: https://github.com/sotarepro/sotaverified
Built for: - Authors who want to claim their papers and submit official metrics - Researchers who want to understand the SOTA techniques for a task - Autonomous research agents to check if a result reproduces before investing GPU hours
Would love feedback from the HN community. What features would make this useful for your workflow?