How do you evaluate a foundation model before you know what it's for?

1 pointsposted 11 hours ago
by galsapir

1 Comments

galsapir

11 hours ago

foundation models in biology still haven't proven they're worth it vs simpler methods (imo). we just published one in Nature, and i feel i spent more time on "how will we know this worked" than on the model itself. the hard part was (mostly) deciding what success even means. open for thoughts