Nice idea, although it is still slower than COM.

COM can run over the network (DCOM), inside the same computer on its own process (out-proc), inside the client (in-proc), designed for in-proc but running as out-proc (COM host).

So for max performance, with the caveat of possibly damaging the host, in-proc will do it, and be faster than any kind of sockets.

Kind of, as anyone with CORBA, DCOM, RMI, .NET Remoting experience has plenty of war stories regarding distributed computing with the expectations of local calls.

In my mind the abstraction should allow for RPCs and being on the same machine should allow to optimise things a bit, this way you simply build for the general case and lose little to no performance.

Think of the loopback, my programs don't know (or at least shouldn't know) that IPs like 127.0.0.5 are special, but then the kernel knows that messages there are not going to go on any wire and handles that differently.

One does not simply walk into RPC country. Communication modes are architectual decisions, and they flavor everything. There's as much difference between IPC and RPC as there is between popping open a chat window to ask a question, and writing a letter on paper and mailing it. In both cases you can pretend they're equivalent, and it will work after a fashion, but your local communication will be vastly more inefficient and bogged down in minutia, and your remote comms will be plagued with odd and hard-to-diagnose bottlenecks and failures.

Some generalities:

Function call: The developer just calls it. Blocks until completion, errors are due to bad parameters or a resource availability problem. They are handled with exceptions or return-code checks. Tests are also simple function calls. Operationally everything is, to borrow a phrase from aviation regarding non-retractable landing gear, "down and welded".

IPC: Architectually, and as a developer, you start worrying about your function as a resource. Is the IPC recipient running? It's possible it's not; that's probably treated as fatal and your code just returns an error to its caller. You're more likely to have a m:n pairing between caller and callee instances, so requests will go into a queue. Your code may still block, but with a timeout, which will be a fatal error. Or you might treat it as a co-routine, with the extra headaches of deferred errors. You probably won't do retries. Testing has some more headaches, with IPC resource initialization and tear-down. You'll have to test queue failures. Operations is also a bit more involved, with an additional resource that needs to be baby-sat, and co-ordinated with multiple consumers.

RPC: IPC headaches, but now you need to worry about lost messages, and messages processed but the acknowledgements were lost. Temporary failures need to be faced and re-tried. You will need to think in terms of "best effort", and continually make decisions about how that is managed. You'll be dealing with issues such as at-least-once delivery vs. at-most-once. Consistency issues will need to be addressed much more than with IPC, and they will be thornier problems. Resource availability awareness will seep into everything; application-level back-pressure measures _should_ be built-in. Treating RPC as simple blocking calls will be a continual temptation; if you or less-enlightened team members subcumb then you'll have all kinds of flaky issues. Emergent, system-wide behavior will rear its ugly head, and it will involve counter-intuitive interactions (such as bigger buffers reducing throughput). Testing now involves three non-trivial parts--your code, the called code, and the communications mechanisms. Operations gets to play with all kinds of fun toys to deploy, monitor, and balance usage.

This could possibly even be dead simple to accomplish if application-level semantics aren't being communicated by co-opting parts of the communication channel's spec.

I think that this factor might be the ultimate source of my discomfort with standards like REST. Things like using HTTP verbs and status codes, and encoding parameters into the request's URL, mean that there's almost not even an option to choose a communication channel that's lighter-weight than HTTP.

Binder is totally unavailable outside of Android, right? IIRC it's pretty closely coupled to Android's security model and isn't a great fit outside of that ecosystem.

My most vivid gRPC experience is from 10 years or so ago, so things have probably changed. We were heavily Go and micro-services. Switched from, IIRC, protobuf over HTTP, to gRPC "as it was meant to be used." Ran into a weird, flaky bug--after a while we'd start getting transaction timeouts. Most stuff would get through, but errors would build and eventually choke everything.

I finally figured out it was a problem with specific pairs of servers. Server A could talk to C, and D, but would timeout talking to B. The gRPC call just... wouldn't.

One good thing is you do have the source to everything. After much digging through amazingly opaque code, it became clear there was a problem with a feature we didn't even need. If there are multiple sub-channels between servers A and B. gRPC will bundle them into one connection. It also provides protocol-level in-flight flow limits, both for individual sub-channels and the combined A-B bundle. It does it by using "credits". Every time a message is sent from A to B it decrements the available credit limit for the sub-channel, and decrements another limit for the bundle as a whole. When the message is processed by the recipient process then the credit is added back to the sub-channel and bundle limits. Out of credits? Then you'll have to wait.

The problem was that failed transactions were not credited back. Failures included processing time-outs. With time-outs the sub-channel would be terminated, so that wasn't a problem. The issue was with the bundle. The protocol spec was (is?) silent as to who owned the credits for the bundle, and who was responsible for crediting them back in failure cases. The gRPC code for Go, at the time, didn't seem to have been written or maintained by Google's most-experienced team (an intern, maybe?), and this was simply dropped. The result was the bundle got clogged, and A and B couldn't talk. Comm-level backpressure wasn't doing us any good (we needed full app-level), so for several years we'd just patch new Go libraries and disable it.

While you are correct the majority of Google services internally are Stubby-based, it isn't correct to say gRPC is not utilized inside Google. Cloud and external APIs are an obvious use case but also internal stuff that are also used in open source world use gRPC even when deployed internally. TensorFlow etc comes to mind...

So "not used much" at Google scale probably justifies 40 people.

> What makes Google feel it's worthwhile to have 40ish(?) people working on an open source project that they don't actually use much themselves? Honest questions -- I don't know the answer, but I'd like to.

It's at least used for the public Google Cloud APIs. That by itself guarantees a rather large scale, whether they use gRPC in prod or not.

[dead]

fuchsia was pretty deeply impacted by google layoffs iirc

> What are the other solutions that are much faster?

Why ask the question, then

> (besides rolling your own mini format)

box it in?

Do you think I'm saying using an RPC is bad? I'm not. I simply took issue with the way the article was worded.

The thing about engineering is you don't want to do what some blogger says is best. You want to analyze YOUR particular needs and design your system from that. So, with every properly engineered solution there are trade-offs. You want easy? It may run slower. You want faster? You may have to roll your own.

Reverse tunnel SSH was used for remote management. Once we replaced it with MQTT, we realized we could use it for IPC, too. I was not too clear about this in the post above.

Before MQTT we used whatever we had at hand; the touchscreen frontend (that ran Qt, not an embedded browser) talked to the local backend with HTTP. SMS messages were first sent to another local HTTP gateway and then via UNIX shm to a master process that templated the message to the smstools3 spool directory. A customer-facing remote management GUI ran on the device's built-in HTTP server but I no longer remember how it interfaced with the rest of the system. The touchscreen GUI might actually have had its own HTTP server for that, because it also handled Modbus communication to external I/O.

This was back in the day when everyone was building their own IoT platforms and obviously we were required to dogfood our own product, trying to make it do things it was never meant for. The gateway was originally designed to be an easy solution for AV integrators to remotely manage whole campuses from a single GUI. I'm pretty sure no customer ever wrote a line of code on the platform.

There is an art to having forwards and backwards compatible RPC schemas. It is easy, but it is surprisingly difficult to get people to follow easy rules. The rules are as follows:

  1) Never change the type of a field
  2) Never change the semantic meaning of a field
  3) If you need a different type or semantics, add a new field

You can trivially make breaking changes in a JSON blob too. GRPC has well documented ways to make non-breaking changes. If you're working somewhere where breaking schema changes go in with little fanfare and much debugging then I'm not sure JSON will save you.

The only way to know is to dig through CLs? Write a test.

There's also automated tooling to compare protobuff schemas for breaking changes.

- JSON doesn't have any schema checking either.

- You can encode the protocol buffers as JSON if you want a text based format.

I guess you're talking about the relative vs. absolute import paths?

This solves it: https://github.com/cpcloud/protoletariat

Do you need to look at that generated code though? I haven't used gRPC yet (some poor historical decisions mean I can't use it in my production code so I'm not in a hurry - architecture is rethinking those decisions in hopes that we can start using it so ask me in 5 years what I think). My experience with other generated code is that it is not readable but you never read it so who cares - instead you just trust the interface which is easy enough (or is terrible and not fixable)

It wouldn't surprise me if inet sockets were more optimized though and so unix sockets ended up slower anyway just because nobody has bothered to make them good (which is probably why some of your benchmarks show equal performance). Benchmarks are important.

Search around on Google Docs for my 2018 treatise/rant about how the TI Envelope was the least-efficient program anyone had ever deployed at Google.

servo's Ipc-channel doesn't use Unix domain sockets to move data. It uses it to share a memfd file descriptor effectively creating a memory buffer shared between two processes