Usually “open source” for an LLM means you get the weights and the inference code, which I’ve started calling “open inference”. It’s certainly good and useful, but it’s not the actual source of the model.

I find people get into silly arguments about the terminology because they’re focused on whether the “source” is “open” and not on what the “source” is actually the source of.

“Weights available” indicates even the weights aren’t “open” in the usual software meaning of the term, as they typically come with restrictive licenses (more restrictive than copyleft or attribution).

I call them open weights or just freeware like when we got only the EXE’s on Windows.

Aren't academic institutions more likely to claim ownership of anything produced than they are to totally open source something?

Plenty of people publish actual open source software, the definition isn’t the problem, it’s the people who misuse it that are the problem.

> that I don't know what to do with it if it works.

After you get it up and running you can just ask it what to do with it.

You are missing key points here. "reproduce" means produce the same. Not just train similar model.

I can simplify the task, can you convincingly explain how the same model can be produced from this dataset? We can start simple, how you can possibly get the same weights after the first single iteration? I.e. the same as original model got. Pay attention to randomness, data selection, initial model state.

Ok, if you can't do that. Can you explain in believable way how to prove that given model was trained on give dataset? I'm not asking you for actually doing all these things, that could be expensive, only to explain how it can be done.

Strict 'open source' includes not only open weights, open data. It also includes the word "reproducible". It's not "reproduced", only "reproducible". And even this is not the case here.

The interesting part of the product we’re taking about (that is, the equivalent of the executable binary of an ordinary software product) is the weights. The “source” is not sufficient to “recompile” the product (i.e., recreate the weights). Therefore, while the source you got is open, you didn’t get all the source to the thing that was supposedly “open source”.

It’s like if I said I open-sourced the Matrix trilogy and only gave you the DVD image and the source to the DVD decoder.

(Edit: Sorry, I replied to the wrong comment. I’m talking primarily about the typical sort of release we see, not this one which is a lot closer to actually open.)

But wouldn't failure to achieve independent reproduction falsify the open claim?

Similar to you publish the source for Oracle (the database), but nobody can build a binary from it because it needs magic compliers or test suites that aren't open source?

Heck when the browser was open-sourced, there was an explicit test where the source was given to some dude who didn't work for Netscape to verify that he could actually make a working binary. It's a scene in the movie "Code Rush".

the goal posts moved?!

Reproducibility? I mean what's the point of an open technology nobody knows if it works or not.

Cheaper for the cloud company. But that doesn’t always translate to cheaper for the end user. Maybe they cost more to run or maybe there’s fewer of them so they’re more expensive to book?

They're not hiding anything; they've very clearly described what they've done and how they've done it.

They've branded their specific architecture and integration, which allows me to easily refer to it as an example.

I understand that it's easy to be cynical about Apple's approach to product development, but it seems unwarranted in this case.