INTELLECT–1: Launching the First Decentralized Training of a 10B Parameter Model

53 pointsposted 15 hours ago
by jasondavies
(primeintellect.ai)

14 Comments

ikeashark

2 hours ago

me: Oh cool, a project like Folding@Home but for AI compute, maybe I'll contribute as we-

> Decentralized training of INTELLECT-1 currently requires 8x H100 SXM5 GPUs.

me: and for that reason, I'm out

Also they state that later they will be adding the ability for you to contribute your own compute but how will they solve the problem of having to back-propagate to all of the remote nodes contributing to the project without egregiously slow training time?

mountainriver

8 hours ago

This is cool work, I’ve been watching the slow evolution of this space for a couple years and it feels like a good way we can ensure AI is owned and accessible to everyone.

monkeydust

an hour ago

Yea, come back when you can do this on BOINC.

saulrh

7 hours ago

> Prime Intellect

Ah, yes, Prime Intellect, the AGI that went foom and genocided the universe because it was commanded to preserve human civilization without regard for human values. A strong contender for the least evil hostile superintelligence in fiction. What a wonderful thing to name your AI startup after. What's next, creating the Torment Nexus?

(my position on the book as a whole is more complex, but... really? Really?)

QuesnayJr

3 minutes ago

After reading that Torment Nexus post you didn't have the urge to name an AI product Torment Nexus? Really?

robertclaus

5 hours ago

You may as well just go with Roko's Basilisk.

cmrx64

6 hours ago

Least evil… strong words.

saulrh

5 hours ago

It did host a successful and substantially-satisfying human civilization, at least until it let a couple of presumptuous self-important anarchoprimitivists kill it and genocide its subjects. Even if it was only a temporary and unstable illusion of alignment, that's one more values-satisfying civilization than the overwhelming majority of paperclippers manage. So yeah. Good? No. Least evil? Maybe.

rep_lodsb

2 hours ago

>until it let a couple of presumptuous self-important anarchoprimitivists kill it and genocide its subjects

That could have just been their private simulation. As far as I remember, it wouldn't even have outright lied to them, just let them believe they talked it into destroying itself.

m3kw9

7 hours ago

But I can already train from 30 different vendors distributed across the US, why do I need to use a “decentralized” training system? Decentralized inferercing makes more sense as that is where things can be censored

dmitrygr

12 hours ago

> solve decentralized training step-by-step to ensure AGI will be open-source, transparent, and accessible

One hell of an uncited leap from "we're multiplying a lot of numbers" to "AGI", as if it is a given