Realms matter. I don't really feel that holding terms hostage between hardware and software worlds is worth much energy.

People that know enough to care will know within 1 second what the article is about.

"Soft real-time" is probably the correct term here, but that is actually more confusing for 99% of people.

"Interactive" is not descriptive. "Duplex" is certainly not obvious.

Indeed. This is just about user interfaces driven by remote events.

There are patterns for "real time". Things such as:

- What matters is the slowest case, not the average case.

- What has to be real time, and what can run in background?

- Avoiding priority inversions.

- Is there a stall timer that trips if you miss the control loop timing? What happens when the stall timer trips?

- What influence do caches have on time repeatability? What's the worst case? Can you keep the worst case from being the nothing-in-cache case?

“Interactive” is overly broad and doesn’t imply multi-user sync. “Duplex” tells me about the transport layer, not the desired functionality — some of the patterns in the article are achieved with half-duplex transports.

“Realtime” in this sense is akin to real-time strategy games which has been a named genre since at least 1992 (and is also not real-time in the classical computing sense).

Let us admit that realtime even it its most traditional use is quite foggy.

It basically just referes to a system that reacts in a predictable time. Traditionally that time is short, but that isn't part of the definition.

As a audio DSP programmer my deadline is determined by the samplerate and the size of my buffer, and that is a hard deadline, as any dropouts might be audible. For game people their deadline is more flexible as there the framerate can drop without it being completely unacceptable. How low of an framerate is too low isn't a clear deadline, but the clear deadline isn't part of the definition. You can run any realtime code in an environment where it can't perform.

So what about a webservice that guarantees that every internal state update is communicated to clients via a websocket within an acceptable timeframe? Like the game people they don't have a clearly formalized deadline, but when a websocket update takes a minute that would surely be unacceptable. So the original definition of "a system that reacts in a predictable time" can still hold — just not for the reason that people colloquially think ("I get updates as they happen or, in realtime").

And to be frank, I think when some dev says "realtime updates" we all understand what they meant and that is the point of language right?

I like the terms multiplayer and collaborative

I've been writing elixir for years (i think since 0.14?) and have been writing liveview for years. I'm all in on elixir in general, but I'm not sure I'd use liveview for another big project. Maybe it's just rose colored glasses because the TS/React world certainly has its own issues, but I think TS/react and regular old phoenix is a sweet spot. The composability of live view components still has a number of footguns (duplicate IDs, etc) and I think it has higher coupling than a nicely structured react app. You also have to treat liveviews and components fairly differently. All these design choices are for good reason, but it ends up being annoying in a large app. Also in deeply nested trees with lots of component reuse, static typing from TS really helps with refactors. As the projects grew to be large, I think I'm more productive with phoenix/TS than I am with phoenix/liveview.

I think there's a certain class of small to medium sized application (I'm building several) where liveview can be a good fit, but after writing it professionally and in hobby projects for several years, I'm less convinced that it's a great solution everywhere.

Came to the comments to say this. As I've been learning Elixir + LiveView, I've been consistently surprised at the amount you get for "free"

I have not enjoyed a language and framework like I'm enjoying Elixir and Phoenix! It has become my stack for just about everything.

I think Elixir/Phoenix has to be the best stack currently for building applications with real-time features and that's all thanks to BEAM. Perhaps Go is another decent alternative but I definitely prefer BEAM's concurrency model.

Exactly! I was halfway through the article and thought how LiveView is basically equivalent of the "push ops" pattern described but beautifully abstracted away and comes for free, while you (mostly) write dynamic HTML markup. Magic!

I work on the team that built a Postgres pub/sub connector at Ably (we call it LiveSync).

Our postgres connector also works on LISTEN/NOTIFY and has horizontally scalable consumers that will share the load.

There's two types of LISTEN/NOTIFY you can use. Either have the notify event carry the payload or have the notify event tell you of a change and then query for the data. If you choose the second option you'll get much better resilience under load, as the back pressure is contained in the table rather than dropped on NOTIFY.

If you do go with a poll-on-change design, you'll likely benefit from some performance tuning around denouncing the polls and how big a batch size of records to poll for.

As for the exact collaboration features, when clients are online and interacting, it's fairly easy. Lots of the hard stuff comes from knowing when a connection is dropped or when a client goes away. Detecting this and updating the other clients can be hard.

Another team at Ably worked on that problem, and called it Spaces.

I assume you are using Postgres? If so i believe you will be limited by max_notify_queue_pages.

I would instead try would be to create an event table in the db, then create a publication on that table and have clients subscribe to that publication via logical replication. You would have to increase the number of replication slots though.

One thing to keep in mind is that when you LISTEN on a connection, you remove it from the connection pool until you are done with it. The straightforward approach of using one connection per connected client can quickly consume all of the connections in your connection pool and prevent you from handling additional requests.

What I’ve done for this in the past is to do all subscriptions over a single connection per server, and then have code in the server that receives all of the events and dispatches them internally based on a lookup table.

I think you'll for an app like this you'll eventually want to go with something that was designed for a high throughput (I don't think LISTEN/NOTIFY was). Yes, it's nice because you get it out of the box with Postgres, but you'll find that it's not resilient, you need to be careful with managing DB connections (in your case of hundreds/thousands users it is a significant thing), and it's lackluster compared to a specialized PubSub.

The other commenters have mentioned doing deploys behind a proxy, which is fine, but eventually you're going to have to re-deploy the component that terminates the client connections (i.e. websocket server or SSE, etc).

From the client's perspective, there's not a lot of difference between the server dropping the connection (on redeploy) or the connection being dropped for some other transient reason.

That is to say, with a decent client side handling of connection state you just incrementally rollout your new servers and each server terminates its connections triggering reconnects from the clients.

The hardest part is often maintaining continuity on some stream of events. That is; picking up exactly where you were before the connection dropped. You need some mechanism for the client to report the event it last received, and some way to "rewind" back to that point on the stream.

On BEAM/OTP you can control how state is handled at code updates. Finicky but you can.

In most other contexts you'd externalise state to a data store like Redis or RDBMS, and spawn one, kill one or do blue-green in the nebula behind your load balancer constellation.

I work on an open source project for doing just that: https://plane.dev

It follows the model used by Figma where each active document gets its own process on a cluster of machines, and requests are routed to the document server. The advantage of this over something like pubsub is that you have a single authoritative server that can persist the document, instead of having to coordinate multiple sources of truth.

(We also provide a managed version of Plane at https://jamsocket.com)

In general you do it by doing a failover behind some variation of a reverse proxy.

If you can start new instances quickly and clients can handle short delays you can do it by just stopping the old deployment and starting the new one, booting off of the snapshotted state from the prior deployment.

If you need “instant” you do it by implementing some form of catchup and then fail over.

It is a lot easier to do this if you have a dedicated component that ”does” the failover, rather than having the old and new deployments try to solve it bilaterally. Could just be a script ran by a human, or something like a k8s operator if you do this a lot

I've worked in robotics for a long time. In robotics nowadays you always end up with a distributed system, where each robot has to have a view of the world, it's mission, etc, and also of each other robot, and also the command and control dashboards do too, etc etc.

Always always always follow parent's advice. Pick one canonical owner for the data, and have everyone query it. Build an estimator at each node that can predict what the robot is doing when you don't have timely data (usually just running a shadow copy of the robot's software), but try to never ever do distributed state.

Even something as simple as a map gets arbitrarily complicated when you're sensing multiple locations. Just push everyone's guesses to a central location and periodically batch update and disseminate updates. You'll be much happier.

Durable Objects solves so many problems for realtime & persistence. The biggest problem is they're vendor locked, so we can't use them if we want to keep all of our infrastructure on AWS.

So we built out a library that lets you run Durable Object-like backends on any cloud: https://github.com/rivet-gg/actor-core

Thanks for sharing your experience, and what you have found to work.

Sometimes I feel we (fellow HN readers) get caught into overly complex rabbit holes, so it's good to balance it out with some down-to-earth, practical perspectives.