maz1b
12 hours ago
Didn't they just recently invest in them? Curious about their strategy, considering NVIDIA already does Nemotron and i think diffusion models as well.
johnnyApplePRNG
12 hours ago
Their strategy is to prevent open models from proliferating, so their massive investments in these AI frauds are not completely unwound.
That's my take, at least.
Nemotron is TERRIBLE, and purposefully so. It must be.
They cannot be THAT BAD at training AI models. I don't believe it.
terribleperson
11 hours ago
I think almost the exact opposite. They want open models. Without credible open models, they only have a few customers, and those customers have leverage against nvidia. With open models, they have tons of customers, and nvidia has all the leverage.
Smaller customers are also less able to develop their own hardware and threaten NVidia's business.
jawiggins
7 hours ago
This is the correct take. NVidia did the same thing with the hyperscalers - when GOOG/MSFT/AMZN were all building their own chip competitors, NVidia began funding the neoclouds to fight down the margin on their competitors.
louiereederson
4 hours ago
Commoditize complements
ekidd
8 hours ago
> Without credible open models, they only have a few customers, and those customers have leverage against nvidia. With open models, they have tons of customers, and nvidia has all the leverage.
I wonder if Nvidia is also trying to cover themselves against a market crash the bankrupts the AI labs but leaves the tech standing? (Similar to the dotcom crash or the big railroad crash back in the day.) If Anthropic and OpenAI struggle, Nvidia can always sell local inference hardware. But local inference hardware often has a lower utilization, so you need more GPUs for the same number of tokens.
PunchyHamster
8 hours ago
I think that's their exact plan. Cover all bases. The bubble can burst but that really only affects the UI model giants, the demand for AI won't drop even if frontier model companies won't find profitability enough to pay the bill
hparadiz
3 hours ago
They are worried because prior to 2022 50% of their revenue was tied to gaming and now that division has actually dropped further cause it's not affordable. Meanwhile there's a dozen competitors working on asics for models. What point is there to run your model on a GPU when it's been made into an asic? None. So eventually when the models hit 99% on all the benchmarks you hit end game and the GPU becomes the worst way to run the model.
SR2Z
9 hours ago
They want LEVERAGE.
They can't ride the hyperscaler gravy train forever; at some point between Google, AMD, and Apple NVIDIA is going to lose its monopoly on serving large customers.
At that point, it would be useful if a few open models existed which were only a couple of months behind the frontier.
But it's very important that the open models never be TOO good, because the AI companies are buying compute on the assumption that their software will add value. If it becomes a commodity business with frontier open models, NVIDIA won't be able to get away with such a crazy markup.
Wowfunhappy
9 hours ago
> If it becomes a commodity business with frontier open models, NVIDIA won't be able to get away with such a crazy markup.
Of course they would. You need hardware to run the model. nVidia sets the floor.
SequoiaHope
11 hours ago
Sure but it also means these companies nvidia buys don’t work with AMD and other competitors anymore. There are myriad motivations and it’s not all one or the other but this is a classic component of Silicon Valley acquisition strategies.
mark_l_watson
11 hours ago
Have you used Nemotron-3.5-lightening? I don’t use it as much as Poolside’s (excellent!!) Laguna XS 2.1 6bit, but the new Nemotron model is good.
I think NVIDIA does want small open models running on-prem to explode as a market! Lots of smaller GPU installations for companies who wisely want on-prem inference.
Of course NVIDIA will also keep making a ton of money selling to hyper scalers, but not forever: Chinese chips are getting better, Google, Microsoft, Amazon, etc. designing their own inference chips.
NVIDIA is handling this brilliantly.
odo1242
10 hours ago
I think this is a classic [Commoditize Your Complement](https://news.ycombinator.com/item?id=17047348) - Nvidia wants open models because their business is hardware and it's complement is AI models, so they want AI models to be commoditized so that hardware is the industry with leverage. OpenAI/Anthropic/etc want closed models so that the AI model development/data has leverage over the hardware providers.
radium3d
11 hours ago
I believe NVIDIA's long term strategy will be to pivot from the data center to the public, and the public will utilize open models on NVIDIA hardware at home. This will come after the RAMpocalypse completes (when the new fabrication plants (China, Tesla/SpaceXAI/Intel) fully ramp up and start selling their RAM for cheap in the next few years). Data centers will be for training mostly.
charcircuit
10 hours ago
Why do you think the public wants to self host models over using a cheaper solution hosted in the cloud?
chii
an hour ago
> Why do you think the public wants to self host models over using a cheaper solution hosted in the cloud?
you aren't truly in control of your compute in the cloud, and if you came to become reliant on ai, the cloud company could simply turn it off for you and you'd be shit outta luck. Unlike a regulated utility like electricity and water, where you cannot be discriminated against based on how you use said utility.
Therefore, having the ability to run local ai is a good thing. Currently, hardware constraints have kept it out of most hobbyist's hand, but i am hoping that will change soon enough (within 2-3 yrs).
The mega corps are looking to regulate ai such that individuals cannot run their private ai themselves, so that they can become more effective a monopoly. That's why it is crucial that ai is not regulated, but companies providing ai.
fc417fc802
9 hours ago
Almost all the non-tech people I know acknowledge concerns about privacy but have no realistic alternative available to them. I think they won't want to self host until and unless it's easy to deploy, there's nothing to administrate, and the hardware cost is in the same ballpark as a new laptop was prior to the apocalypse.
fooker
7 hours ago
Because the public prefers to use ad funded Google search over hosting a tiny web search index you can fit in memory. :)
salawat
4 hours ago
Nigh impossible nowadays to expand now that sites are increasingly blocking indexing by unknown crawlers.
Geezus_42
10 hours ago
Subscriptions are almost never actually cheaper.
charcircuit
9 hours ago
It is cheaper to subscribe to AI for $500 a year for the rest of your life than to buy a machine capable of running a current frontier model with no subagents.
SV_BubbleTime
11 hours ago
There’s a lot about this that would make sense.
But, not really at the current technologies. Kimi and GLM are fucking awesome, but I don’t have 3TB of VRAM to run them, and I don’t expect to even when ram prices drop.
So now you’re back to the scaling issue before talking about power and compute distribution.
jacquesm
6 hours ago
GLM 5.3 runs like a charm in ~240G of VRAM (with half a million tokens context). Depending on what you've got that's between 3 and 10 GPUs. Fewer, larger GPUs will likely run it faster.
SV_BubbleTime
24 minutes ago
Yes you can run a quantized GLM on 240GB of ram.
fc417fc802
9 hours ago
Do you think you'll realistically need 3 TB RAM to run a sufficiently good model 1 to 2 years from now? I certainly don't. Considering what can already be done with 128 GB of relatively slow unified memory, imagine if efficiency improvements continue apace, the memory becomes 128 GB of HBM, the flash device becomes capable of sequential throughput matching today's DDR5, and such a system was affordable as a routine purchase for the average person.
SV_BubbleTime
8 hours ago
The things in LLM “knows” are not infinite per its physical size. “Experts” don’t exist in a world where hardware capacity was a trivial issue.
in two years, yes I expect us to have much better models. Perhaps many times more efficient. But not 100 times.
fc417fc802
an hour ago
Don't discount improvement in the ability of models to search for and ingest external data - either web or purpose built databases. I really don't think 3 TB will be necessary with the right model architecture and training environment.
MangoCoffee
11 hours ago
Nvidia is trying to increase its customer base. Look at the Mag 7, Amazon, Google, Microsoft, and even Meta are all working on their own inference chips. I don't think they can completely ditch Nvidia for LLM training, but they can make their own chips for inference. I believe that's also why OpenAI made its own.
No one wants to pay the Nvidia tax
tayo42
9 hours ago
If you make your own chip don't you need to make your own software stack too. Which is what I thought kept everyone using Nvidia.
overfeed
7 hours ago
> Which is what I thought kept everyone using Nvidia.
Once an organization is big enough to have AI-platform or AI-infra teams, you stop being hung-up on details like CUDA; resources will be invested to make whatever abstractions the AI researchers and engineers rely on (e.g. Torch) will be made to work, and work well.
tayo42
7 hours ago
Yeah but if you have the capability why not write that software for existing hardware alternatives and abstract hardware vendor away for your self? It seems like instead your choosing to do two hard things instead of just one
bruce511
4 hours ago
The big tech companies are cash-rich, can afford to hire (and pay) top people and are looking for things for those people to do.
In other words their motivations and your and my motivations are different. I have limited cash, and my primary goal is expanding my ability to get more cash.
By contrast, the Google's of this world have too much cash. Their primary goal is finding things to depend the cash on (billions at a time) that can return, or at least potentially return, either mountains of more cash, or a competitive advantage.
So yes, for them, hard problems, the harder the better, are desirable.
hparadiz
3 hours ago
If Nvidia was actually smart they'd increased production enough that these companies don't wanna do all this. Instead they're producing the conditions for 50 new competitors to not only rise up but be able to to print money. And the gaming community they are burning in the process will happily jump ship.
InsideOutSanta
11 hours ago
I doubt Nvidia wants to be fully dependent on the success of two highly unprofitable companies that could implode at any moment. It makes much more sense for them to commoditize LLMs so that their target market grows to every mid-sized or larger company.
nl
7 hours ago
The Nemotron project isn't supposed to be SOTA. It's supposed to provide proven training recipes for labs to innovate based on.
The Nemotron models are just a by-product of this strategy.
solarkraft
11 hours ago
Maybe as an LLM it is, but I constantly use their streaming ASR model (called Nemotron Streaming) through Handy and it works wonderfully well.
ReptileMan
12 hours ago
The other way. Nvidia would love open source jevon paradoxed ai - that would run inference on their chips.
kimixa
12 hours ago
But their goal would be to ensure it only runs on their chips, and not any competitors. I can't see how they could do that if the best models truely were "Open".
I can see one of Nvidia's biggest fears is the inference hardware becoming commoditised.
bigyabai
8 hours ago
The existence of inference ASICs has been priced-in to Nvidia's stock since Google started designing TPUs. It's not that new, really.
What Nvidia has the market cornered on is flexible GPU architectures. Nobody else has stepped up to the plate on that, and it's how Nvidia will butter their bread with robotics and future model training efforts.
redanddead
12 hours ago
can someone weigh in on this. what's the actual play here
are they actually suppressing the western open models?
china doesn't give a fuck either way
imo, they see the weakness emerging at the intersection of all the labs, everybody knew there was no moat, so they're gonna control its direction and basically tell the Jev guys what they want them to work on
mattmaroon
12 hours ago
It’s just an illogical conspiracy theory. Open weights models still have to run on someone’s chips. NVIDIA is model-agnostic.
They are just trying to grow the pie because they have nobody else competing for slices.
My guess is they want as many frontier models using their chips as possible. The only threat to their business is companies making their own chips which Google does and the others are working toward. The last thing they want is only 3 frontier labs who are all not buying NVIDIA.
MangoCoffee
11 hours ago
yes, this is my read as well. Nvidia does not care about how many LLM provider is out there as long as they pay the toll (Nvidia tax). for now, you can't beat the Nvidia CUDA/chip for training but for inference. that's where you can gain ground.
Hugging Face - distribution for model that you can run on your local Nvidia Spark
Neoclouds - Nvidia setup a 500 billion investment fund with Wall Street so Nvidia can sell chip and this news about buying a LLM start up. it seem like another customer for Nvidia.
correct me if i'm wrong but i remember i saw an interview with Jensen where he want more company to have their own model and country to have their own LLM model.
Nvidia doesn't make money from the gold rush. Nvidia made money from selling the shovels.
redanddead
9 hours ago
Ah yes sovereign AI
But we’re getting ahead of ourselves
jollyllama
12 hours ago
Watch what they do and not what they say.
Aurornis
7 hours ago
This is one of the sillier conspiracies when you think about the economics.
nVidia sells hardware. They want open models everywhere because it encourages people and companies to buy more of their hardware, which will sit idle most of the time, instead of using data center services which are ruthlessly optimized to maximize token throughput per hardware unit.
Acquiring companies that are good at training models is exactly what you would do if you were bad at training models.
3ddss
5 hours ago
Lol garbage post.
Have you seen who composes most of Nvidia's revenue?
seizethecheese
11 hours ago
I think the idea that they’d purposely spend company time and resources making a bad model is an extraordinary claim, requiring extraordinary evidence. The more likely explanation is that they aren’t willing to distill from their own customers, so they are at a disadvantage.
frozenseven
12 hours ago
Insane conspiracy theory. There's no incentive whatsoever for Nvidia to release weak models. If you bothered to pay any attention, they are aggressively trying to catch up. Whether they succeed, that's of course a separate question.
alexdns
11 hours ago
why would Nvidia try to compete against their biggest customers - openai and anthropic ?
cbsks
10 hours ago
Dog fooding their own product. This isn’t new for Nvidia. For a long time they have been selling GPU cards, and also licensing the chips for other manufacturers to make their own cards. Similar for their automotive products, Shield, and DGX Spark.
bobthepanda
11 hours ago
the power that giveth can also taketh away.
similar to how eventually AWS started making their own chips for data centers, and Apple did that for their hardware, it's not a ridiculous thing to plan for the AI companies to start making their own chips to optimize for their use cases and cut out the middleman for margins.
rapsin4
11 hours ago
So that nvidia gets bargaining power...? Nvidia needs to diversify its customer base
raincom
25 minutes ago
Every ex-Google Deep mind should start a start up, get funding from VCs and Nvidia. Sell the startup back to Nvidia. That's the best play for those who are capable of this kind of AI startup.
mrandish
11 hours ago
Yes, NV took $800M of a $2B round in Reflection.ai.
NV's long-term strategic incentive in funding a semi-open model provider like Reflection.ai is to ensure competitive frontier models which fully leverage the NV proprietary stack (chips, interconnects, servers, CUDA) continue to be widely available and continue to offer performance worth a higher price to the most profitable market segments.
NV's ~75% margins on hardware(!) at >$100B/yr scale are historically unprecedented and still increasing, creating tectonic pressure on NV's largest customers (hyperscalers and frontier labs) to escape the "NV Tax" by gaining access to competitive frontier chips, servers, and/or middleware at lower margins. Why would NV help create semi-open models that threaten their best customers? Because as Jeff Bezos famously said, "Your margin is my opportunity" and that's turning NV's biggest customers into their largest existential threat.
At the moment, NV's moats blocking significant competition are almost unimaginably deep, but on a decadal time-scale, literal trillions of dollars are at stake. That's enough to get people thinking the unthinkable, making NV the biggest target in modern business history. It's to the point that it's almost "Everyone against NVidia" which is forcing all the big companies into playing 3D strategic chess on multiple time horizons at once, simultaneously working with, investing in and hedging against each other. It's a 'co-opetition' (https://en.wikipedia.org/wiki/Coopetition) race where the smaller players are grouping into tactical alliances and uneasy truces while the biggest players are spending billions to 'commoditize their complements' (https://gwern.net/complement) as NV is doing with Reflection.ai. This can create strange bedfellows overnight. I wouldn't be surprised to see some of NV's biggest customers, who compete fiercely against each other, pooling resources with NV's competitors to create a viable alternative to NV. This is the stuff Jensen has nightmares about, waking up in a cold sweat in his black leather pajamas.
NV doesn't need Reflection to be better than the best models or even be profitable. They just need to ensure a viable alternative to frontier lab's proprietary models: A. Remains widely available at low enough cost for all NV's other customers to buy, B. 'Works best on NVidia', and C. Stays close enough in price/perf to prevent any single proprietary model becoming as dominant in models as NV is in hardware. The Chinese semi-open models have been strategically convenient for NV but it'd be foolish to count on the Chinese govt continuing to subsidize them or Chinese models not getting blocked or limited by some governments. If it only costs a few billion, Reflection.ai being "good enough," especially for a semi-open, (near-)free, US-based model, is a cheap strategic hedge against long-term threats to NV's (near-)monopoly, especially when Jensen has trouble finding room to store the mountains of cash NV is piling up. When your margins are ~75%, there's literally no better place to put money except toward extending your dominance.
niltecedu
7 hours ago
I think their strategy right now is just spend as much raw cash as possible into companies
Qiu_Zhanxuan
10 hours ago
It's really hard to know what's keeping you ahead in a field that's constantly innovating.