Implementation of GPT-2 in pure CMake

79 pointsposted 12 hours ago
by porridgeraisin

15 Comments

voxelghost

5 hours ago

Using python as a build tool, and Cmake as the programming language, now all that is missing is a bit of config in ASM

ahartmetz

10 hours ago

FWIW, this is most likely not even fast enough to laugh at how slow it is - for that, you need to see something happening in less than an hour or so.

It can only use one core at a time, but that is not the worst thing: CMake is interpreted and stringly typed, so every number will have to be converted string -> number and usually back on every use.

voxelghost

10 hours ago

We just need to find a way to extend Moores law into the 4th dimension, and hardware will catch up to this.

ahartmetz

9 hours ago

Btw, I just realized that "cannot use all cores" has become a similar performance handicap to "is interpreted". So many cores now!

avilay

8 hours ago

Why? Just...why?

I get that sometimes we like to build useless things because they are elegant, or beautiful, or "cool" in some sense of the word. But CMake is not cool in any sense of the word! I can barely use it to write build files, and that too I have gladly offloaded to AI. Sometimes when looking at CMake files in a large project, I become nostalgic about Makefiles, but then I think about what the rewrite would entail, and I resign myself to my fate.

voxelghost

8 hours ago

Sometimes these little endevors, are like art, they don't have to mean something, they just has to make you feel something.

Feeling Queasy, is still a feeling, mind you.

OJFord

2 hours ago

An exercise in understanding gpt2, flexing the author's cmake skills, while demonstrating the absurd complexity of what it allows similarly to 'c++ templates are turing complete' or 'doom on my calculator'.

edg5000

5 hours ago

It's surprisingly tiny when you exclude the data

pkilgore

9 hours ago

disgusting (complementary)

paaloeye

9 hours ago

Somebody has more time than sense