RAM needed for keeping KV cache around may be the more expensive factor.

Not surprising it's a hard cutoff: they almost certainly have two infrastructure configurations for the two max sequence lengths

Fewer nodes dedicated to prefill per instance, and fewer nodes in total since you don't need to support a higher KV cache.

Disaggregated inference also means they can tune the balance of compute dedicated to prefill seperately from decode

I don't think so. This is a separate model, so I assume that if you just use this and switch to the 1 million context model when you reach 256k, your cache will be invalidated, so you'll re-pay the 256k tokens on the 1 million context model pricing.

Edit: I was wrong, thanks to longwave for pointing this out. It's absolutely possible to start out on the 256k model and then switch to the 1 million model when you get close to the context limit without invalidating the cache:

"When switching from k3-256k to k3 (1M), if k3-256k is close to the 256k limit and you don't want compact to lose information, you can switch directly to 1M. The current version switching from 256k to 1M does not affect the cache."

As far as I understand, no. They're suggesting that smaller context windows are typically cheaper (fewer input tokens over time).

Its 256k context window. Quantization is orthogonal. We cant really tell directly so it could be quantized.

It's pretty handy for have very long contexts for long running agents, or else when doing literary analysis to simply be able to load the entire book in.

> Are they really that good,

They did exceed the expecations of pretty much everyone! I've also blogged about using the model, it's a bit on the slow side but pretty good!

> so were totally surprised and overwhelmed by the requests ... or do they just don't have the hardware being in china?

Yes, this is mostly the case: https://x.com/Kimi_Moonshot/status/2078855608565207130

As a user, I much prefer that to service disruptions or severely degraded or secretly quantized performance. However if I didn't have an account, I'd be pretty pissed off about not being able to give them money and become a user.

Since the model is open-weights, you can get from other providers, for example see https://openrouter.ai/moonshotai/kimi-k3#providers

I honestly prefer this as opposed to what Anthropic has done in the past which is to continue accepting users even if they don't have the capacity to serve and behind the scenes tone down everyone's limits.

Not at all a marketing gimmick, the demand is simply that high.

I was able to press "Join waitlist" and then within 48 hours got accepted. The limits aren't very high, no where near the endless subsided+resets given on ChatGPT/Claude. I recommend ChatGPT for good value output!

Others here mentioned "the providers could be quantizing it!" but some of the providers on OpenRouter have partnered with Moonshoot and OpenRouter shows the int when you expand on the provider.

It should say "mxfp4" but providers like Baseten report FP8.

No, the waiting list is true.

Kimi had become that popular. I was a subscriber of Kimi back when latest version was Kimi K2. Later I unsubscribed because I jumped over to GLM subscription (they had amazing deal). Now when I wanted to try out Kimi K3 to find out what the fuzz was all about, I couldn’t subscribe to them.

I remember reading a post from Moonshot team about this, they are doing this because they are almost at peak capacity and want to reserve it to keep the quality for their current customers.

We are actually witnessing an open-weight model catching up at catching mainstream users attention. And instead of behaving like Anthropic, they actually care about their users experience.

It's an extremely popular model hosted by a company affected by hardware export bans. I doubt they'd voluntarily prevent people from subscribing if they didn't absolutely have to to maintain service quality.

> Codex uses 256k masterfully

This.

When working on hard problems (not "vibecode me a script to show an alert box", but e.g. "let's see what this three-level LUT-state-machine obfuscated binary does"), hitting the 1M (!) context window with Claude Code feels like you were talking to Claude Claudewski when his shift just abruptly ends, he packs his things, throws the office keys at Claude Claudeson in-between the front door frame while handovering like "Hi! Nice to see you, good luck." and now here we go again, you are working with someone who just experienced an acute amnesia. It tries everything it already tried, everything it was told in the initial prompt to not do, everything it was told in follow-up prompts not to do. "You were right, this approach does not work and we don't have 20 TB RAM on this machine for full symbolic execution, let me try..."

In Codex, it's so seamless that I sometimes just notice "wait, the context was 20 % remaining, it is 70 % now, wow, when did this happen", while it seamlessly works on the task. Basically never had an issue with context on Codex, be it coding features, cracking hard crack-me ciphers, or researching basically anything.

(For full disclosure, my last experience with Claude was a few weeks ago when I cancelled the subscription, maybe they fully reworked the traumatic "Summarizing" - "Oh, hi! Where are we? Who am I? What we are doing? This is taking too long, let me take a shortcut..." lobotomy they were doing in the meantime.)

256k is enough when the harness uses it properly and the model is not stupid. And also when the tokenizer is not tuned to invoice as many tokens as possible...

Needing large context windows is an illusion.

That's because it has the best compaction of anyone, and it's not even close.

Claude might as well not even do it in my experience.

I'll have to give it a shot some time soon. I just can't imagine working on any of my serious projects doing a new large feature with that little room. By the time it has looked at half the code required to start planning the work, it'd be out of context. Must use sub agents better or something.

I believe Codex harness is pretty sticky though I haven’t tried many others. Does anyone else provide a harness of that quality?

I have so much gratitude to the frontier companies who did all the extremely complicated research and development, model by model. It already feels difficult to remember how much capital it really took. Thank you for getting us to this point.

This exact same comment is the top comment on the reddit thread for this news

https://www.reddit.com/r/kimi/s/BFa1TR9vNg

For me the sweet spot is somewhere under 500k depending on how extensive I want to get. You can build up a sizable effort project in half a million tokens with Claude, with Claude having all the context from ground 0 to wherever you're off at.

"256k ought to be enough for anybody"

At least in the self-hosted LLM inference engines, you have to pre-allocate space for the maximum amount of context you want to allow for each parallel session. By using a lower maximum, you don't have to allocate as much VRAM for each session, allowing more usage for the same amount of hardware. Thus, cheaper.

Uses less quota (i.e., cheaper). For people who like to keep their contexts small, this is a no-brainer.

> k3 (1M) consumes about twice as much quota as k3-256k

Cheaper?

It seems to only be available in Kimi Code, via subscription, no there's no API pricing. The linked page says it consumes about half as much quota as the 1M version though.

Anthropic services have outages on all days ending in y.

Same model, new configuration

I don’t know, but I do think that the days of the US “forcing” Euro countries to do anything, is over.

It isn't legally possible for them to do this at the EU level. The EU parliament would never vote for it.

For pressure at the country level leading to this kind of thing I think it's very unlikely. Here in Sweden it wouldn't just require a vote in the Swedish parliament and before this there'd have to be förarbeten and you can't just brazenly push things through with insane arguments, Swedish social convention goes against it-- and there's just no way to get it through.

It also might not even be legal. "We aren't at war with China and I'm a communist, and the US LLMs are so aligned with values inimical to my political ideology that this is interference with opinion formation" might be an actual legal argument that the ECHR or CJEU might actually have to accept.

Trump burned a lot of bridges in the EU, that one will be a hard sell

Ah come on, HN is not the place for these types of unfounded conspiracy theories that keep popping up on Reddit.

I'd guess because it costs them nothing and it gives you a smoother transition back towards paying for their products.

From my view, as soon as they do that, they send people out the door to use Opencode instead - and once many people have a taste of trying every model via Openrouter, it's eye opening as to the possibilities.

Of course - Anthropic and OpenAI have an advantage in the amount they can subsidize the usage, but I think those days are waning.

Well when a huge part of potential revenue is all in on Bedrock... you need the harness to be able to talk to Bedrock. And Vertex. And all the other places these models are hosted. And allow for proxy because many businesses do not all direct internet access... all valid business reasons.

It honestly should be easier, its a by product of everyone using the same API standard.

Codex and Claude require editing a .json file, but most other harnesses have direct connections via a /provider or /login command.