PyTorch: A Reference Language

52 pointsposted 9 hours ago
by matt_d

4 Comments

insumanth

an hour ago

Anyone not doing distributed training across thousands of GPU, pytorch is the best option. Others have an infra team to work on it.

ansk

7 hours ago

This kind of sounds like just giving up on making the compiler better.

antonvs

5 hours ago

The point is there are limits on how much better the compilers can get, without modifying the language to be closer to the target environments. And there are multiple target environments, so a one-size-fits-all solution is not really possible. You have to compromise somewhere - you can’t have a single high level language that automatically produces maximally efficient code for every target, because the targets are not general-purpose computing environments, they have significant constraints.

If what you’re looking for is a way to avoid having to get into the operational details of the targets, and just provide a high level description that somehow gets figured out, then an LLM-based solution is almost certainly going to be your best bet.

madhu_ghalame

6 hours ago

The idea makes sense because researchers already think in PyTorch. The tooling should adapt to them, not the other way around.