Lots of words and weird analogies to say basically nothing.
What is the status of the project? What can it do? What has it achieved in 5 years?
But no, let's highlight how we follow the "Elon process".
As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.
Using lines of code as a metric for productivity is bad. Using it to show how simple something is, or how a refactor removed x lines of code that doesn’t need to be maintained any more isn’t such a bad thing I’d say.
Yeah this is exactly right, if you can trust the contributors to not code-golf or otherwise Goodhart the LoC metric, then it's a reasonable measure of complexity.
It doesn't work as well when you start mixing languages, or generating code.
TFA includes a time measurement though, and 5 years for 18'935 SLOC doesn't scream quite "how simple something is".
Less LOC also doesn't imply simplicity: just look at the demoscene, which often has the former but not the latter.
From [0]:
"When we can reproduce a common set of papers on 1 NVIDIA GPU 2x faster than PyTorch. We also want the speed to be good on the M1. ETA, Q2 next year."
[0] https://tinygrad.org/#tinybox
>People get hired by contributing to the repo. It’s a very self directed job, with one meeting a week and a goal of making tinygrad better
I find this organizational structure compelling, probably the closest to reaching 100% productivity in a week as you can get.
I wonder what happened to George’s old policy of requiring everyone to move to San Diego?
That's comma.ai's policy since they make hardware and solve physical problems. The tiny corp has been hybrid (remote-first) since day 1 because it primarily writes open source software, and there's a long track record of success with remote for this kind of task.
We have a few whole-team meetups in Hong Kong each year for 2-4 weeks, and there's a San Diego or Hong Kong office that anyone can work from as they choose. We also have a wide array of fancy multi GPU boxes that everyone on the team gets full access to (known external contributors can get some access also).
I think many companies that were quick to embrace remote have walked it back, not everyone is capable of working productively remotely, nor are all types of work amenable to remote.
The risk for Tinygrad is that PyTorch will create a new backend for Inductor, plug in their AMD codegen stuff and walala, PyTorch still king. I mean, they could have easily just taken that route themselves instead of bothering with a new ML framework and AD engine. 99% of the work is just the AMD codegen part of the compiler.
Either way, super cool project and I wish them the best.
The main risk is that an LLM will rewrite itself and programmers will no longer be needed. I worked a bit with tinygrad and it looks quite amusing I managed to run it right away and make fixes in one of the tasks, but I decided not to commit because I was afraid of rejection. For example, the tasks are strange: $500 for two months, optimizing H.265, something that only a small group of people in the world can do.
The SV is a unique place where you can meet Geo and get $5M, maintain a bunch of hardware, build a framework in 20,000 LOC and everything works well.
Very weird to market this as subscribing to "Elon process for software"
I remember when defcon ctf would play Geohot's PlayStation rap video every year on the wall.
I hate it when ‘inspirational’ quotes are attributed to the person with the largest audience and not the people who came up with it, like in this case, the engineers at Lockheed’s Skunk Works.
It's an apocryphal quote.
"A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away."
- Antoine de Saint-Exupéry
> To fund the operation, we have a computer sales division that makes about $2M revenue a year.
What's the margin on that? Do 5 software engineers really subsist on the spread from moving $2M/yr in hardware?
George raised $5.1M in 2023 for Tinygrad
Programming a GPU in 2025 is complex, that might be because it has been made complicated, but regardless, it is not complexity that this project can control.
The fact that it competes with PyTorch in so few lines speaks to the incredibly low incidental complexity imposed by Tinygrad.
What would tinygrad replace if they continue to proceed like this?
Potentially PyTorch and Tensorflow.
I think it has great potential for deployments on edge systems.
It is already used in comma.ai’s openpilot hardware
Is this the guy who talked a big game about all the things he was going to fix at Twitter, then utterly failed when confronted with a real world codebase and gave up having done nothing of use?
He left after realizing nothing was going to change (not like he’s in the leadership).
Also half-joked how the good food went away.
—
George is many things but not a quitter (see comma ai for example).
If someone could pull this, it’s him due to “never give up, never surrender” attitude.
The shit with nvidia just needs to stop
So this is all python? I bet Chris Lattner probably approached them.
Lattner is a smart guy, but I think Mojo might be the wrong direction.
Time will tell.
History has not so far been kind to projects which attempt to supplant cPython, whether they are other Python variants such as PyPy, or other languages such as julia.
Python has a lot of detractors, but (despite some huge missteps with the 2-3 transition) the core team keeps churning out stuff that people want to use.
Mojo is being positioned "as a member of the Python family" but, like Pyrex/Cython, it has special syntax, and even worse, the calling convention is both different than Python, and depends on the type of variable being passed. And the introspection is completely missing.
Honestly, I feel like Julia might as well beat Mojo or sommat to the punch, sooner or later. It has some facilities and supporting infrastructure for a lot of scientific and data-handling tasks surrounding ML, if not for compiling and dispatching kernels (where XLA reins supreme to anything in the CUDA ecosystem!) For example, Bayesian programming like Turing.jl is virtually unmatched in Python. It's been a while since I looked at Lux.jl for XLA integration, but I reckon it could be incredibly useful. As long as LLM's and RLVR training thereof should continue to improve, we may be able to translate loads of exiting Pytorch code eventually.
> tinygrad is following the Elon process for software. Make the requirements less dumb. The best part is no part.
That’s not Elon. See Russian TRIZ
https://en.wikipedia.org/wiki/TRIZ
>"We also have a contract with AMD to get MI350X on MLPerf for Llama 405B training."
Anything to help AMD (and potentially other GPU/NPU/IPU etc. chip makers) catch up with NVidia/CUDA is potentially worth money, potentially worth a lot of money, potentially worth up to Billion$...
Why?
If we have
a) Market worth Billion$
and
b) A competitive race in that Market...
then
c) We have VALUE in anything (product, service, ?, ???) that helps any given participant capture more of that market than their competitors...
(AMD (and the other lesser known GPU/NPU/IPU etc. chip vendors) are currently lagging behind NVidia's CUDA AI market dominance -- so anything that helps the others advance in this area should, generally speaking, be beneficial for all technology users in general, and be potentially profitable (if the correct deals could be struck!) by those that have the skills to do such assisting...)
Anyway, wishing you well in your endeavors, Tinygrad!
Fell bad for geohotz. Such a lovely guy, i hope he strikes it right soon
Seems like he's doing fine, why do you feel bad for him?