bob778
40 minutes ago
> Preliminary trials with Claude Mythos Preview showed that it would not provide an apples-to-apples comparison with other models because of how we had set up the experiment and how the model was served.
What does this mean? My guess is they couldn’t co-locate Mythos close enough to reduce latency?
(I’m assuming this experiment pre-dates the export controls)
georgemcbay
20 minutes ago
> My guess is they couldn’t co-locate Mythos close enough to reduce latency?
I doubt network latency is the reason. Even when connecting from literally across the world network latency is lost in the noise of overall response latency of even fast models.
The overall response latency of the model very well could have been the difference, though. AFAIK Mythos is structured to do relatively slow "deep thinking".
bannable
a minute ago
Depending on the timeline, it could be that they're not allowed to access Mythos because of something like non-US citizens on the team or the lack of some way for them to meet the constraint DOD has them under.