Deep Seek v4.1 M5 Max at 17 tokens/s

6 pointsposted 8 hours ago
by Argonautlabs
(github.com)

2 Comments

Argonautlabs

8 hours ago

We stream DeepSeek-V4.1-Flash (518 GB, 4-bit) from NVMe on a 128 GB M5 Max, because it doesn't fit in RAM.

                        prompt processing      steady decode
    upstream, internal       16.23                 10.59
    ours, internal only      28.04  1.73x          14.38  1.36x
    ours, +1 external        36.88  2.27x          16.05  1.52x
    ours, +2 external        43.62  2.69x          17.38  1.64x

  The first fork row is the one that matters: same single internal SSD, no replicas, no enclosures.

nextaccountic

an hour ago

Just so you know, you are getting automatically flagged in HN (your comments are automatically removed). I think it's because you are using LLM to write them. You should not copy paste LLM output directly to comments in this website.

(I'm not talking about using LLM to generate the README.md of your project, that's also distasteful but not nearly as bad)

In either case your repository is interesting, and might be useful for people that own a M5 Max