Yeah, of course we are paying for the compute on our end. I think we are waiting for our users to stress test our theory, however our secret sauce is optimizing inference so that this is a feasible product on both ends.

Like I alluded to in my initial comment, I've never understood why the inference providers of today charge per token. Whichever marketing department got our industry to be ok with being charged for the output of a REST call, I applaud.

I'm not sure it will all hold up at scale, but I am still waiting for the test to break. It helps to be a curious and optimistic person in this endeavor.

Not currently implementing any rate limits. Using an assortment of open models.

Pricing is largely based on how many users we think we can fit per our baseline modset, and how much the cost of that modset is determines where we can work back from in price. From there, we would scale on a per modset basis.

It may not turn out to be a perfect science, but in the name of shipping something and getting feedback, it's out there

It is very new (launched a couple days ago), so still trying to figure out how to get it out there. I'm a software nerd not a marketing genius unfortunately.

As for how it works, its a pretty standard inference provider, where we offer a chatbot and an API, at unlimited usage for a fixed cost. Since its very new, we're still wondering if there's something we're missing as for why fixed cost AI is not the norm.

Hoping it is similar to the music industry, where you once had to pay $1.99 to download a song, and now you can stream all you want for $12.99. We want to begin that new trend.