GPU Puzzles

166 pointsposted 5 days ago
by cgadski
(github.com)

7 Comments

srush

3 hours ago

I made these a couple of years ago as a teaching exercise for https://minitorch.github.io/. At the time the resources for doing anything on GPUs were pretty sparse and the NVidia docs were quite challenging.

These days there are great resources for going deep on this topic. The CUDA-mode org is particularly great, both their video series and PMPP reading groups.

nextos

an hour ago

Slightly offtopic, but any chance you could update or re-upload code for your https://github.com/harvardnlp/DeepLatentNLP tutorial? I found the NLP latent variable models discussed there really interesting, and notebooks were excellent. However, these seem gone and the only thing left are slides?

Alternatively, any other places that discuss the same topics, including some code? I could only find equivalent discussions with code in Pyro docs and Kevin Murphy's book, volume 2. But these are more sparse as they also cover many other topics.

bytepoet

2 hours ago

Thanks a lot, Sasha, for creating these. I found your LLM training puzzles to be excellent as well.

throwaway314155

19 minutes ago

Either puzzle 4 has a bug in it or I'm losing my mind. (Possible answer to solution below, so don't read if you want to go in fresh)

    # FILL ME IN (roughly 2 lines)
    if local_i < size and local_j < size:
        out[local_i][local_j] = a[local_i][local_j] + 10
Results in a failed assertion:

     AssertionError: Wrong number of indices
But the test cell beneath it will still pass?