If you think the idea of buying Fable like model is hard, you should really try some of the latest open models like Qwen 3.8. They’re getting there very fast.

I like this analogy a lot.

It reminds me that coin-op SF2 champion edition was like $6000 new, and you rented time with tokens. Eventually you could play a very good but more limited SNES version of SF2 at home for no tokens. And then PC hardware matured to the point where you can play the exact bits via emulator at better than original fidelity.

It will be very nice when AI gets to the emulator stage.

I'd say Nintendo gave you 10x the fun of neo geo. Yeah it looked cool, but mario and Zelda on the nes played like nothing else. And then we got street fighter 2 on snes which is better than any snk fighter I've played :)

(I'll run for cover now as I'm sure neogeo fans will disagree!)

I'm stealing this analogy. This is good.

But also look at live service games and the shit modern game studios are doing to try to go back to the old world of rent seeking.

Going to the arcade was about a lot more than just the games, though.

>Gaming used to be done in arcades for 25c a play.

in the early days, 2 plays for a quarter was not unusual (but you couldn't buy one)

Arcades mostly failed to adapt to the advances of home consoles to provide experiences you couldn't get at home. That's on the dimwitted game suits of the 1990s who cared little about novel gameplay, networked gameplay, or better NPC AI and physics. Then mobile happened and there was a brief glorious Cambrian explosion of new ideas until the Zynga playbook and DLC. Evidence: lived through it as a small game studio developer trying to do original things. The fun experiment now and then is to break out your 1980s video game consoles and play through their libraries. Most of the games are objectively terrible but gems like Yar's Revenge are still gems today IMO.

It will take a while till prices drop to a level where affording 256GB card (even if it would be shared between few coworkers) will have any sense over buying tokens.

And the AI debt bubble price spike will probably stay there for a while.

Well, if thw ai bubble implodes, youll sudden be exposed to real costs. Thats when a real opinion will be formed.

Uber took a decade but eventually, the real cost of a taxi service went up for many use cases.

Your argument is only solid if we are headed to the singularity.

You've stumbled on exactly why the big providers are lobbying for regulation and big claims that Ai is unsafe. They don't want you running these at home.

Working on a blog post to document it all, but that will take a few more days. The short version: the big rig is probably $25K, ttft is dependent on how much prefill you have but typically a couple of seconds (2K prefill/sec in one setup, about 500 in the other), tgs 100 to 150 or so depending on what it is doing and how well the speculative decoder works with the current context. Electricity is pretty bad, full tilt it's about 2500W or so but I've power limited it to 1500 to keep it civil. It is not cheaper than using a frontier lab (or at least, I don't think it is), but the independence bit is the important part for me and I don't care if that carries a surcharge, even if it is a substantial one.

I mean, I just have 4xV100S GPU and a Macbook Pro w/48GB of RAM and I find it kind of amazing.

My server can run Qwen Flash Next at 500 t/s aggregate (about 200 t/s single user). It's pretty great. I use it for local coding and all my worker agents. It is about as reliable as Sonnet 5 for agentic coding. I make 3D models, keep the home network and server updated, fully built ansible files.

You do NOT need 20K in hardware to run these models reliably. I built my own inference engine from scratch for fun. You can run Strata on 16-32GB cards at very respectable token rates if you have the standard RAM to go with it. I strongly encourage people to go use this model on OpenRouter and try it out for tasks.

I also have a somewhat lobotimized quant of GLM Flash 5.3 working and it's pretty decent too.

Not OP. But I’ve done some calculation here https://localaicost.com/

They didn’t steal them, they copied them, similar to how Anthropic and OpenAI copied the materials they used to train their models to begin with.

No need to send data away. Local open models keep everything on your device.

Smart move to get them bought ahead of time.

You may get the same guarantees on paper, but at this point there's absolutely no reason to trust any AI provider.

At best they get a fine, and it will not make a dent in their cash reserves.

Well... that's a tricky one. I bought a lot of these second hand. I can sell them for more than they originally cost new, so indeed, it is not 'cheap'.

Then again, I run a business and for the business it is absolutely manageable, it is roughly the price of a single job. Privately this would be a different matter but I assume that if you would do this privately you'd simply get a single (but powerful) GPU second hand and offload to RAM. Not as fast but with llama.cpp, 'ninfer', DwarfStar and various other clones of those two you can get some pretty decent performance out of single GPUs.

I've been buying powerful hardware since the 90's, starting with SGI and then ever larger Linux boxes. I make my living with them so it's never been a huge decision to invest some of the proceeds back into the tools. But on an absolute scale, yes, it is a lot of money.

IBM has two faces. Their consultancy arm is the exact opposite of what I would call a good company, even so, they make a good bit of money. The hardware side of the business is still quite impressive, yes, it is too expensive but their power rack boxes are effectively a little mainframe (check the IO stats on those things) and the performance is - I have to admit - impressive. That said I would never buy it because I can get much better value elsewhere. Even so I think they could do this if they really wanted to.

You need to reread what was written. No one is proposing to take away your right to leak proprietary edge to the rapacious frontier labs that need to eat most of the economy in order to survive.

As the OP said really. It is very very freeing to just be able to give a task that in ye olden days you knew was going to be a pain in the arse to do, and it will do it without you. Amazing.

It's like having extra copies of yourself that you can just fork and get stuff done. E.g. the other day I was in a meeting and there were some heavy discussions about a few gnarly issues. I fired off a few prompts in parallel while the AI researches options, approaches, made a prototype etc all while I was present and paying attention still in the face to face meeting. I am so much so much more efficient as an engineer now - getting better feedback, more bonuses etc. It's never been better - I handle thinking about the requirements, the computer science and software engineering big picture etc, and the AI handles the syntax trivia for implementation.

The models are really really good now.

The only downside is that as a result of more stuff getting done, we need to have more meetings to discuss what actually needs to be worked on.

Think bigger. Don't think in features. Think in business workflows.

You are a consumer, so you naturally think like the consumer you are. But consumers want cheap stuff and only big big businesses are able to provide cheap stuff at massive scale. So stop thinking in b2c. For us little businesses the money is in b2b. Businesses pay lots of money to have problems solved. They want to focus on their core business and outsource the rest.

Consumers hate paying subscriptions. Businesses love paying subscriptions. Because then they can make their problems someone else's problem and focus on what their actual business is.

Making an app has always been approximately 1% of the work involved in turning an idea into a business.

I've hit a weird point myself. I'm not overly interested in building software for other people now. What I'm able to do is build all my stupid hobby project ideas for my very specific use cases and not have to worry about making them profitable in order to justify the time spent on them. I just built my own point of sale system because nothing on the market meets my needs, and my needs are extremely niche.

Take your time to set up the tools and just build something small. Create a personal website, or a calendar that helps to track friends' birthdays, or something else of that sort. It's a good way to calm yourself down, because it will give you a good picture of both the capabilities and the limitations of AI.

I've found it very liberating that I can handle some of the tedious parts of a project to AI. In the end, I spend perhaps the same amount of time on a project, but more of it goes to important work and less gets wasted on plumbing.

Ideas have always been worthless though, the alpha is still in making something good + useful

Upvote for what sounds like genuine curiosity!

Small nitpick, but you can use any model you want with Claude Code and Codex - doesn't have to come from Anthropic or OpenAI

People regularly switch

[dead]

My guess is if you want to run AI on prem and do it with IBM gear it would be via their x86 servers + OpenShift AI via IBM Fusion HCI.

"Affordable" and "IBM" don't belong in the same sentence together, but they're happy to sell you their inference-capable hardware if you've got the wallet for it.

> It doesn't make sense that IBM is just watching NVIDIA take its lunch.

If you have ever worked at IBM, it makes total sense. Completely dysfunctional middle management and on up.

I think they will need a far more aggressive management before such a thing could happen. But I can dream about it.

It is not going away any time soon, but when it does it will be sudden and have a massive drop in demand.

It was pure coincidence that the AI-craze neatly followed the crypto-craze (many neo-clouds in the AI space were originally crypto miners).

If they raise their prices you can switch pretty quickly, insurance isn’t worth much.

I have single sentence prompts in one of my legacy codebases that it takes Opus 5.5 over $1000 in API spend to return an answer correctly on. I am not getting by with a $20 plan.

That's my point - do you need the best available to you? What if tomorrow your employer said no more 5.5, or Anthropic IPOs and suddenly investors want to see a profit and that 20USD a month for Opus 5.5 becomes 20USD per-day or even per-hour? We're already seeing corps restrict their employees token usage.

Could your survive on say Opus 4.8? Or GLM 5.3?

I bet you could if you had to.

I think it's the same argument about cars. Why wouldn't you want a Ferrari? Do you not value your time? Why not go as fast as physically possible?! You need the absolute fastest, most-capable, best-handling car for every single journey you take! Or... Back in reality not every journey you take is trying to lap the track as quickly as possible and just any reasonable car will get you where you need to go.

This is my point. Will Anthropic and OpenAI continue to exist selling Ferraris when actually people realise they can get by with a Toyota Corolla 99% of the time? Ferrari exists in the car world so I guess there is an argument for "yes". I am not sure myself - Ferraris are after all rare and exotic cars ... I suspect that even if we are not there now, we are already seeing current SOTA models that are mostly equivalent/better than the average engineer - another generation or two they'll be waaaaay better than the average engineer. How much more car do you need, and how much are you prepared to pay for it?

At least for me, I don't like using Opus or Fable because I don't like the way it talks to me (talking down to me, refusals, moral high-grounding, overly pomposity etc). 3.8 Flash is currently my favourite model due to how it communicates and works with me and the speed is much better suited for rapid iteration. So put me in the comfortable, reliable, use-it-every-day-to-get-shit-done-with-no-fuss Toyota Corolla camp. Its good enough for me and I don't get any benefits in my day to day life by using Opus 5.5 and it's sanctimonious personality.

Yes, but that was not the point I was making.

They are many people and none of them are in some sort of AI death cult like the EA polycule.

They're not suddenly good guys. They're just sane

That’s not a bad little model! Way more impressive than I expected. I am still messing about with it, but I think it’s a keeper for general discussions.

Though if you have the room for it, Muse Glimmer is better in practice on low reasoning strength, and while its token generation is slow, it actually gets to answers pretty quickly.

What happens when your competitors invest in better and more expensive tools or materials in general? They (potentially) make better products, but those products also cost more. It's turtles all the way down.

Cool! That's when you pay the difference in marketing and auto-win your sector! People don't actually buy the best, they buy what they think is.

Ford vs. Ferrari, except that the Ford is actually a few months old Ferrari.

Does Ferrari have any moat if it can't convince the government to ban Ford?

The target audience is likely those who can't or won't use frontier models unless they are on-premise.

You buy the competitor and kill their culture from the inside, ruin them and take a tax loss on the company.

depends on the economics and much more, the best Ai cannot do a lot of things, 90% of human tasks by some reports

we technologists tend to extend what we see and think to other realms where it doesn't make sense

Great term. Plus the related "Politician Said a Thing!" journalism

thank you, I was trying to re-find this the other day

Linux already served as a life-preserver for them also.

the only credible "conversation" is "well he would say that, wouldn't he?"

That sounds like a task that can be automated.

The whole point of this was for internal usage.

> an appropriate workload

Honest question: What would such a workload look like? Why would Salesforce or GitLab not make sense on a mainframe?

Transaction can be a nebulous term; think of credit card transactions, ach and wire transfers, mutual fund buy/sells, bank deposits and withdrawals, etc.

OpenAI most likely pushed Cerebras into a contractual agreement not to do business with Anthropic through 2030. ("Most likely" because the names of the companies Cerebras is contractually obligated not to sell to are redacted in public filings)