[delayed]

how many requests per second can the server take?

> We're running Kimi 2.8 on a $107k server

Equipped with what? Is it CPU based inference, a mix...?

Are you developing software? Is most of it used on a coding agent? (Like Claude Code or ChatGPT Codex?) If so, what coding agent do you use? If you're not developing software what do you use it for (roughly)?

Where do I sign up to get 200k/yr to keep one rack running? Sounds like an incredibly chill job

As mentioned, it is a "large enough company" already; they have full time sysadmins running things.

Adding another rack beside the VMWare cluster, managing any storage/networking issues, etc. will be incremental costs; they already have a pager (probably not a pager not anymore just an app on their phone) like rotation schedule etc.

> I don’t trust hosted LLMs for anything that needs to be private

I'd update this to

'I don’t LLMs for anything that needs to be private'

What's to prevent the LLM from sliding a heavily obfuscated binary blob into the application that does nefarious things? If you aren't creating the LLM itself from scratch, I don't feel it can be trusted.

> However, I don’t trust hosted LLMs for anything that needs to be private.

Why not? Do you trust AWS with things that need to be private?

> And hire 2 or 3 dev ops to keep it running

Not a devops but I'd say one full time is already too many.

There will be cloud/SaaS vendors who have lower cost of labor/capital due to automation and financing terms.

Having these models in the open caps the inference margin.

Just like that new jobs created by AI! Localized model maintainer/technician.

You’ll slap some training on existing technologists/infra/sysadmin folks and perhaps have a support contract for the edge cases (hardware troubleshooting and advanced replacement).

(managed an entire data center building with thousands of servers a lifetime ago with ~2-3 other people, it’s only gotten easier over the last two decades imho)

Just let it manage itself, what could go wrong! :)

I think internal use is allowed at any scale in the license?

> 4. The requirements set forth in Sections 2 and 3 do not apply to: (a) internal use of the Software, defined as any use that does not make the Software, its outputs, or its underlying capabilities available to third parties; [...]

As far as I can tell, license fees are only applicable if you have more than 20m USD/year revenue from services provided using the model, or serve more than 100m users.

I'm not sure I understand the case for open-source models being decelerationist, is this it?

Decel:

- Potentially reduces investor appetite for funding big labs.

- More risk of powerful AI getting in bad hands -> more regulation.

Accel:

- More competition so big labs can't rest on laurels.

- More research in open, so all labs can accrete advancements faster.

I feel like open-source = acceleration has a much more clear argument. (and how bad would deceleration be in any case?)

Why? Absolutely correct, especially considering the position of the person they're referring to

Why?

It’s a different topic but it’s correct imo. Open sourcing things is the best way to accelerate development.

Put another way, if you want to slow things down, put it behind a paywall, tag ideas ans “intellectual property” (meaning you’re the only one who can use it) and get the lawyers involved (injecting our slow legal system).

None of the above is a judgement call on whether development should be accelerated.

I know it's a hard ask on this site, but I need you to start parsing content and not tone. It was a helpful bit of context, even if it was a bit vitriolic.

only if you only get your news from main stream business press and Big Lab propaganda channels

There's no chance K3 is a distill of Fable, it came out way too soon after the limited fable release to be feasbile.

If you look at all of the top ML conferences, chinese labs contribute way more to advances in ML than "Open"AI and Anthropic: https://www.reddit.com/r/TheMachineGod/comments/1pi4q7f/pape...

This K3 release just helped every other lab on the planet stay in the race by making it possible for them to build on top of it, placing them at the frontier starting line instead of having to spend billions of their own dollars and risking it all to attempt to catch up.

The open source contributions I linked to above will move the whole field forward and reduce the costs of training and inference for everyone.

Open science compounds on it self, every new advancement pushes the field forwards and opens up new grounds for future improvements.

It is impossible for a single closed lab to consistently stay ahead of the rest of the field, especially in a huge growing research area like machine learning. The only only advantage the big labs have is money, but the naive scaling game is not sustainable long term when you have to pay 10-100x more then the fast followers and we start getting more and more open models or use case specific models that can handle 90% of high volume use cases.

Research is a high variance, low expected value activity, meaning that the few large concentrated labs have to be conservative with their bets and double down on proven things when scaling up. The rest of the field is like a diversified portfolio, with thousands of players making smaller riskier bets that only require a few of them to succeed (like K3 did here, and DeepSeek a year ago)

EDIT: also if you look at most of the work from OpenAI, it's mostly taking existing promising open research work and scaling it up. (except for things like CLIP and etc from Alec Radford)

OpenAI's head of strategic futures publicly stated that you can't explain the quality of the newest Kimi via distillation.

Further, you can just read the papers released alongside most open models. Plenty of hugely influential research results published that drive the frontier forward. It's not like these models are just existing architectures downloaded from Huggingface and trained on frontier lab APIs.

Distilling can be done by fast follower closed models too, so this argument against open models doesn't hold up.

being able to replicate it in the open means that there's nothing special about frontier models.

Frontier models would have to do something extraordinary or unique, or unreplicatable, because clearly there is no moat, and US companies are sitting on huge nvidia valuations and get surprised when competitors beat them.

To be clear, the restriction on "Model as a Service" past $20M yearly revenue is new to K3. K2.x had the attribution requirement for any commercial use with more than $20M monthly revenue.

In what way was the license "janky"? Moonshot isn't a trillion dollar US behemoth. If these terms allow it to release near frontier models with open weights, so startups and anyone with enough hardware can use them, while charging only companies with more than $20 million in revenue or 100 million users, that seems like a reasonable trade off.

These are the right questions. IANAL, but I am of the opinion that model weights are not copyrightable. See my sibling comment: https://news.ycombinator.com/item?id=49073127

How many companies host and serve models via API and have a $20M+ revenue? Going to be pretty straightforward to catch offenders.

Software licenses aren't enforced through litigation as much as they are enforced through the _threat_ of litigation and legal risk. In other words, pretty much every company pays lawyers to minimize legal risk. Those lawyers inevitably look at all the contracts, agreements, and software licenses, and tell the C-suite what to do in order to keep their legal exposure as low as possible. "Don't violate other companies' IP," is pretty low-hanging fruit in those conversations.

It is a very rare (and ballsy, and perhaps incompetent) company that ignores their lawyers' recommendations to adhere to the letter of all of the software licenses they are bound to.

They haven't litigated the last public non-compliance... Despite that one being extremely public. So probably not at all for now.

They have certainly trained in some secret call/response pairs that would uniquely identify Kimi serving.

Open source projects frequently have separate commercial licenses no? This isn't abnormal.

Still more of a tanh than I've seen in years

Which part of that is doing gating? I thought gate generally looks like

    f_gate(x, y) = f(x) * y

    f_gate(x) = 4 * tanh(x / 4) * sigmoid(x)

> which OpenAI seems to allow via API but not Anthropic

Does openai still allow logprobs in their current gen models?

It's not that light on the details. I read the paper and they train several different models in parallel over a few different domains and then they distill from their own models to get the final model.

NVIDIA-Nemotron-3-Ultra-550B-A55B was released in June 4th 2026 and I think it was the largest open US model until thinking machines' Inkling (975B) was released a couple of weeks ago.

the longer i read this comment the wronger it gets

August 2025.

Yeah, besides that, the only big other open source US model that was worth looking at was Inkling (975B params), Jul 15, 2026.

https://thinkingmachines.ai/news/introducing-inkling/

1e21 flops is hilariously wrong. for reference the llama 3 8B model (https://arxiv.org/pdf/2407.21783) used 10 times that many flops. This model is 350x bigger in total params and 12x bigger in active params and was trained on 3x the data.

> Kudos to Moonshot for making these weights available for download. Lets not fool ourselves and claim these are "open source" by any understanding of the concept though, there are usage restrictions (even if you download them) and also training data isn't clearly broken down either, nor it it actually using a FOSS license.

They will probably never release the training data because that represents a large part of their competitive moat. The same is true of US companies (Google, OpenAI, Meta etc) none of which has released the full training data for its open models.

They use private datasets that cost a lot to acquire, synthetic datasets and a lot copyrighted material for which they don't have licensing.

What matters most is that, with the necessary hardware, I can download a near frontier model, run it and modify it however I want. The other concerns you mentioned are just noise. And if my company is generating $20 million in revenue or serving 100 million users, it can probably afford a relatively inexpensive commercial license.