Celld: Self-hosted, distributed Durable Objects

104 pointsposted 6 hours ago
by calvinfo

16 Comments

khalidx

an hour ago

Finally!

So happy to see support for running durable objects outside of one provider. Upvoted.

The "durable object" concept has been repeatably demonstrated to be a valuable abstraction.

"Each object is its own SQLite database, addressed by name and replicated to an S3-compatible bucket you own" -- this concept can take you a long way, both in its power and simplicity.

hhthrowaway1230

an hour ago

I like it! Would be great if I could run locally from the start without configuring an s3 for easy playing/prototyping

wmf

36 minutes ago

Maybe Wrangler is a better way to do that.

vmg12

2 hours ago

All we need is a way to run these on spot instances

wmf

an hour ago

I don't see why not. Somebody once wrote a simple Lambda that spawns a new instance when your spot instance dies so there's always at least one running.

jauntywundrkind

4 hours ago

Well that couldn't have been any more timely. Did they know? Were they keeping this in a back pocket for the time?

CloudFlare OS: https://blog.cloudflare.com/cloudflare-os/ https://news.ycombinator.com/item?id=49182996

And yes, celld is isolates! The very lightweight within v8 isolation boundary. Hence the "very low idle cost". The deno team rolled their own new runtime! This one without deno_core! Shout out to this excellent 2022 post: https://deno.com/blog/roll-your-own-javascript-runtime https://news.ycombinator.com/item?id=35819990 https://news.ycombinator.com/item?id=35819990

It's super super exciting having an age where many pieces of software can all run with low profiles. Very timely, again, for the unbelievably order of magnitude (and thats 10 to the not 2 to the, for you fellow computer nerds) ish ram crunch we are in. Next up, at some point, ideally we get some v8-like runtimes where we can share libraries across multiple isolates! Separating the shared code from the shared data, so we have multiple instances of the library, seems much harder. (It feels like wasm has some/much promise here and I eagerly await clearer signals that code-sharing while sandboxing is indeed possible)

reactordev

2 hours ago

Now you’re getting into the realm of old SOA… where data exchanges are just as stochastic as services on the pipe.

wartywhoa23

2 hours ago

The order of magnitude of this is simply mindblowing.

zuzululu

an hour ago

i donno...i have some doubts about this

cloudflare gives you that instant geo-sync across the world that is hard to beat

with Celld do i need to buy bare metal in major continents

wmf

32 minutes ago

With apologies to Obama, if you like Cloudflare you can keep your Cloudflare. It looks like celld is not intended to be an exact clone of Cloudflare but is for someone who wants to run a bunch of workers in a single region.

benatkin

4 minutes ago

If it were intended to be a direct alternative for the biggest use case, I think it would be more like Cloudflare Workers and Durable Objects. However, I think it's an alternative for a lot of uses of those two things. For a lot of uses of them, likely most uses, they are overkill.

It's a 100% open play, despite mentioning three hosted APIs: S3, Cloudflare Workers, and Durable Objects. The thing is, that Deno tried making similar ones. Deno Deploy is analagous to S3, and Deno KV is analagous to Durable Objects. However, now the only dependencies are being able to run a binary and some S3-compatible storage.

FWIW here's an old Deno blog post comparing Durable Objects to other stuff: https://denoland.medium.com/deno-kv-vs-cloudflare-workers-kv...

vmg12

an hour ago

Geo-sync? I'm pretty sure durable objects are tied to the datacenter they are created in.

zuzululu

an hour ago

nothing beats cloudflare's reach they have nodes everywhere

thats hard to beat

jauntywundrkind

44 minutes ago

on the one hand, hosting is hard, you should pay people to host, especially distributed systems.

on the other hand, deno land folks stole from (used open source from) the best. the coordination layer/control-plane is all S3 CAS of dumb json files, which is a fantastic common-mode infra requirement for most orgs anyways, & perhaps a durable control-plane substrate you'd feel comfortable having someone else run (such as aws or others), while you run the data-plane (workers) yourself. and then for the durable objects themselves, they used litestream, which is a pretty top pick, excellent way to get radical distribution (but actual topology not included, some assembly required)! https://hn.algolia.com/?q=litestream

there's no reason this wouldn't run fine on most hosting, you definitely don't need bare metal. the virtues of v8 sandboxing / isolates! no need for vm's at all, no nested vm difficulties if you are trying to host on a shared host! but if you're asking questions like this, i want to again point you back to my top point.

q3k

an hour ago

> addressed by name and replicated to an S3-compatible bucket you own; nodes coordinate through that bucket alone, with no control plane or consensus

... but that S3 _is_ the control plane and consensus layer, no? You're just pushing this down the stack to whoever runs that S3 clone.