Apple approves driver that lets Nvidia eGPUs work with Arm Macs

139 pointsposted 3 hours ago
by naves

57 Comments

tensor-fusion

2 minutes ago

As more people carry ARM laptops and keep the GPU somewhere else, I think the interesting UX question becomes whether the GPU can "follow" the local workflow instead of forcing the whole workflow to move to the GPU host. That's the problem we've been looking at with GPUGo / TensorFusion: local-first dev flow, remote GPU access when needed. Curious whether people here mostly want true attached-eGPU semantics, or just the lowest-friction way to access remote compute from a Mac without turning everything into a remote desktop / VM workflow.

dd_xplore

23 minutes ago

Why does Apple need to make the drivers in a walled garden? Atleast they should support major device categories with official drivers.

embedding-shape

8 minutes ago

> Why does Apple need to make the drivers in a walled garden?

Isn't that the whole point of the walled garden, that they approve things? How could they aim and realize a walled garden without making things like that have to pass through them?

mlfreeman

14 minutes ago

I followed the instructions link and read the scripts...although the TinyGPU app is not in source form on GitHub, this looks to me like the GPU is passed into the Linux VM underneath to use the real driver and then somehow passed back out to the Mac (which might be what the TinyGrad team actually got approved).

Or I could have totally misunderstood the role of Docker in this.

bangonkeyboard

an hour ago

I don't know how Apple has evaded regulatory scrutiny for their refusal to sign Nvidia's eGPU drivers since 2018.

MBCook

40 minutes ago

The government doesn’t care? They’re a minority of the market? The vast majority of their computers didn’t have slots to put Nvidia GPUs in, and now none of them do?

hgoel

28 minutes ago

They said eGPU

mulderc

33 minutes ago

Apple doesn’t have a monopoly in any market they are in.

TheDong

15 minutes ago

It depends how you define the market. In the 2001 microsoft case [0], the courts ruled Microsoft had a monopoly over the "Intel-based personal computer market".

Apple has a monopoly over the "M-chip" personal computer market. They have a monopoly over the iOS market with the app store. They have a monopoly over the driver market on macOS.

Like, Microsoft was found guilty of exploiting its monopoly for installing IE by default while still allowing other browser engines. On iOS, apple bundles safari by default and doesn't allow other browser engines.

If we apply the same standard that found MS a monopoly in the past, then Apple is obviously a monopoly, so at the very least I think it's fair to say that reasonable people can disagree about whether Apple is a monopoly or not.

[0]: https://en.wikipedia.org/wiki/United_States_v._Microsoft_Cor....

Underphil

a minute ago

I don't think any of what you're describing are legal "monopolies". I don't have a single Apple product in my life but I'm fairly sure there's nothing I'm prevented from doing because of that.

raw_anon_1111

7 minutes ago

That’s not how monopoly definitions work. That makes about as much sense as saying Nintendo has a monopoly on Nintendo consoles or Ford has a monopoly on Mustangs

GeekyBear

an hour ago

The same way Google evaded regulatory scrutiny for refusing to allow a YouTube client for Windows Phone?

bigyabai

44 minutes ago

Internet Explorer Mobile is a YouTube client. You're describing a client-server disagreement when the user is talking about an entirely client-based conflict.

realusername

32 minutes ago

Google deployed custom code to actively block the clients so it went beyond just a disagreement

the__alchemist

an hour ago

I'm writing scientific software that has components (molecular dynamics) that are much faster on GPU. I'm using CUDA only, as it's the eaisiest to code for. I'd assumed this meant no-go on ARM Macs. Does this news make that false?

wmf

27 minutes ago

This driver doesn't support CUDA.

Keyframe

an hour ago

Such a shame both companies are big on vanity to make great things happen. Imagine where you could run Mac hardware with nvidia on linux. It's all there, and closed walls are what's not allowing it to happen. That's what we as customers lose when we forego control of what we purchase to those that sold us the goods.

deepsun

42 minutes ago

Don't purchase? I don't own any Apple devices, everything works fine.

TheDong

10 minutes ago

Unfortunately, Apple still won't release iMessage for Android or Linux (unlike every other messenger platform, like whatsapp, telegram, wechat, microsoft teams, etc, which are all cross-platform).

Because of that, you need an apple device around to be able to deal with iMessage users.

aljgz

11 minutes ago

I don't understand the logic for downvotes. We vote with our wallets. When I could not update the Ram on my personal Dell machine I asked for a Frame.work in my new job. As my Intel based FW at work had thermal throttling problems, for my next personal purchase I got an AMD one. As Ubuntu had shady practices, I installed Fedora, as Gnome forced UX choices I did not want, I used KDE. As I wanted my machine to be even more stable I use an immutable spin.

The machine I'm using now represents my choices and matches what matters to me, and works closer than perfectly than all my machines in the past

And yes, I have worked with macs, and no, the UX and the entire tyranny in the Apple ecosystem was not something I could live with.

arjie

2 hours ago

Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/

I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?

999900000999

an hour ago

You can buy a cheap GPU enclosure for about 100$ off ali express.

Takes a standard PSU. However, Mac Minis don't have occulink. So you might be a bit limited by whatever USB C can do.

Now if Intel can get there Arc drivers in order we'll see some real budget fun.

https://www.newegg.com/intel-arc-pro-b70-32gb-graphics-card/...

32 GB of VRAM for 1000$. Plus a 500$ Mac Mini.

Fnoord

an hour ago

Those $100 ones don't come with a cage. If you do want a cage, you'll end up with $180 in total, with zero warranty.

Article mentions: "Apple finally approved our driver for both AMD and NVIDIA"

Does not mention Intel (GPUs). Select AMD GPUs work on macOS, but...

Macs (both Intel and ARM) support TB, but eGPU only work on Intel Macs, and basically only with AMD.

Good news is for medium end gaming choices are solid, and CUDA works on AMD these days.

999900000999

29 minutes ago

Fortune favors the bold my friend.

I own one of these, the cage is just a piece of plastic. Anyway, I don't think 80$ is that big of a difference here. I can't really afford a 4k Nvidia GPU. Intel is my only hope.

manmal

2 hours ago

Maybe I’m lacking imagination. But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? The interconnect isn’t powerful enough to change layers on the GPU rapidly, I guess?

zozbot234

an hour ago

> But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute?

It would work just like a discrete GPU when doing CPU+GPU inference: you'd run a few shared layers on the discrete GPU and place the rest in unified memory. You'd want to minimize CPU/GPU transfers even more than usual, since a Thunderbolt connection only gives you equivalent throughput to PCIe 4.0 x4.

manmal

an hour ago

But isn’t the Mac Mini the weak link in that scenario?

zozbot234

an hour ago

It has way more unified memory than your typical dGPU.

arjie

2 hours ago

My Mini is actually the smallest model so it actually has "small but slow VRAM" (haha!) so the reason I want the GPU for are the smaller Gemmas or Qwens. Realistically, I'll probably run on an RTX 6000 Pro but this might be fun for home.

GeekyBear

an hour ago

We've seen many recent projects to stream models direct from SSD to a discrete GPU's limited VRAM on PCs.

How big a bottleneck is Thunderbolt 5 compared to an SSD? Is the 120 Gbps mode only available when linked to a monitor?

manmal

an hour ago

That’s what, 14GB/s? The GPU‘s VRAM can do 100x that.

GeekyBear

an hour ago

A discrete consumer GPU card doesn't have enough fast RAM to run a very large model that hasn't been quanitized to hell.

That's why all the projects streaming models into the GPU from an SSD popped up recently.

lowbloodsugar

an hour ago

“Lying around”. I’ve got an unopened 5090 in a box that I know will suffer the same fate, so I’m sending it back. So privileged to have the money to impulse buy a 5090 and yet no time to actually do anything with it.

vondur

26 minutes ago

If you could get Nvidia driver support on Mac’s I bet Apple would have sold more MacPro’s.

eoskx

2 hours ago

Interesting, but cannot run CUDA or more to the point `nvidia-smi`.

embedding-shape

2 hours ago

Well, to be fair, the whole shebang is from a completely different company, that have their own ML library and such, so that isn't that surprising. Although I agree that some CUDA shim or similar would be a lot more interesting, still getting to the place of running inference and training with your very own library is pretty dope already.

frankc

2 hours ago

My main thought is would this allow me to speed up prompt process for large MoE models? That is the real bottleneck for m3ultra. The tokens per second is pretty good.

embedding-shape

an hour ago

tinygrad does have pretty neat support for sharding things across various devices relatively easy, that'd help. I'm guessing you'd hit the bandwidth ceiling transferring stuff back and forth though instead.

wmf

2 hours ago

Pretty misleading. This driver is only for compute not graphics.

polotics

2 hours ago

As a sizable share of the market is going to want to use this for local LLMs, I do not think this is that misleading.

comboy

2 hours ago

GPUs can do graphics too?

manmal

2 hours ago

Graphics was not what came to mind when I saw the headline.

Fnoord

an hour ago

The term eGPU gives it away, but is inaccurate.

Something like eNPU or eTPU seems more appropriate here.

brcmthrowaway

2 hours ago

What are the limitations of USB4/Thunderbolt compared with a regular PCIe slot?

embedding-shape

an hour ago

Well, for starters, PCIe 5.0 x16 would do something like about 60 GB/s each way, while Thunderbolt 4 does 4 GB/s each way, TB 5 does 8 GB/s each way. If you don't actually hit the bandwidth limits, it obviously matters less. Whether you'd notice a large difference would depends heavily on the type of workload.

justincormack

24 minutes ago

It carries pcie, but only at x4. Thunderbolt 4 is pcie gen 3 and Thunderbolt 5 is pcie gen 4.

bigyabai

2 hours ago

The opportunity cost of Apple refusing to sign Nvidia's OEM AArch64 drivers is probably reaching the trillion-dollar mark, now that Nvidia and ARM have their own server hardware.

chuckadams

2 hours ago

Apple got out of the server game long before they adopted aarch64, so that's a trillion worth of server hardware they never would have sold anyway. And probably not actually a trillion.

bigyabai

2 hours ago

Apple was the only one stopping themselves from getting back in. It's not like the Mac is a trillion-dollar market segment to begin with.

QuantumNomad_

an hour ago

Almost everyone including myself had MacBook Pros at my last place of work.

If Apple was in the high-end server market, I see no reason why the company I was working for would not be running macOS on Apple hardware as servers, instead of the fleet of Linux based servers they had.

varispeed

2 hours ago

USD starts sounding more and more like meaningless tokens. Billion here, trillion there. I still have 100 trillion Zimbabwean dollars somewhere.

altairprime

2 hours ago

Feels like that here in the U.S., too.