reilly3000
a day ago
I think this whole premise is ignoring the point that most decently sized companies want to and eventually will be running their own LLM workloads. Currently use cases are limited by scope and imagination, and predominantly focused on cost efficiency. Once a use case becomes a top line revenue driver budgets will become essentially only limited by ROI. There are some inference workloads that are unacceptable to send to the frontier labs for privacy reasons. Most of it will come from firms who want to productionalize their own fine-tunes. In any case, the market for inference is less than 1% of what it will be in 5-10 years. This whole notion that GPUs will be sitting idle en masse is ridiculous. People will just start running GPU databases if it becomes cost efficient.
reticulates
a day ago
but this is just an absolute fantasy, what, exactly, will this compute be doing? Our lives are already deeply entwined with technology and use barely any compute. How could our lives become 100x more dependent on compute?
You can be thrilled by this exciting technology and all the possibilities it brings without thinking it is going to require this huge capital investment in GPUs. You could radically change billions of lives with a few dozen GPUs.
dgellow
21 hours ago
Push everybody to buy hardware they don’t need to constantly run agents for pretty basic tasks, such as processing your daily emails and sending you little summaries notifications. So, the most inefficient software tools ever produced, but that keeps the whole industry alive. That’s pretty much the vision Jensen Huang is selling to ensure NVIDIA continues to grow
michelb
17 hours ago
I hope we can quickly escape these tasks and the focus on coding. It’s all very superficial. There are a ton of serious AI workloads imaginable and I haven’t seen anyone pushing those frontiers, at least not publicly. Nvidia had a shiny keynote about their digital twin of the world for solving large problems but i guess that one died already.
b1gOhbuddy
20 hours ago
Ditch the servers.
Do and sync over client to client.
Keep data local again.
Only use cloud for backups of local client encrypted blobs of vectors:data
If you get rid of a lot of the suspect semantics hallucinated up over decades of software development it's not hard to see the geometry of an electronic snowflake. All the language just obfuscates the elegance. Crude meat suit grunts and clicks.
Streamline it all to management of geometric states and access control and put the semantics on the presentation layer. What if we don't need python and go and ruby anymore? Made sense in a pre-gpu everywhere reality. Could just be high school stats classes to generate sets of values. Let go of the obscure linguistic chants.
The data centers are just to serve surveillance purposes. Obfuscated behind politically correct memes of creating jobs.
Chip away at the monolith and atomize the topology
Reviving1514
a day ago
TIL about GPU databases. I did a bit if research but only found this one: https://github.com/bakks/virginian
What are the advantages of a GPU database?
SahAssar
21 hours ago
Is anyone really running GPU databases in prod at scale? Or are you assuming that the GPU compute from a crash will become so cheap that it makes this niche scale?
user
18 hours ago
mikae1
a day ago
> most decently sized companies want to and eventually will be running their own LLM workloads
And probably some decently sized states too. Not only commercial actors are up to the job.
babymetal
17 hours ago
Is it just me or is this comment extremely difficult to parse? Besides the lack of any links/evidence the assertions are full of jargon "decently sized", "scope and imagination", "productionalize their own fine-tunes", "This whole notion... is ridiculous". It is currently at the top of HN and I hope that is just because it's a Sunday afternoon in many places.
iwontberude
a day ago
[dead]