tptacek
3 months ago
This is something that the classic TCP/IP books (both Comer and Stevens) got wrong, inexplicably, because David Reed was very clear about it. UDP isn't a transport protocol at all. It's an extension interface for building new transport protocols. It is the thing you would use (and weirdly didn't) to ship SCTP, and did use to build QUIC. It does not make sense to compare UDP to TCP; it's like saying TCP is worse at URL handling than HTTP.
Talking about what UDP provides is a little bit like talking about what IP provides. It's almost a category error. Is "UDP" "reliable"? It's as reliable as whatever transport you build on it is!
jeroenhd
3 months ago
I disagree. Raw UDP is about as useful as raw TCP if your application matches the protocol's features and weaknesses. Plenty of protocols work by spamming packets around the network where it doesn't matter if half of them go missing or arrive out of order.
TCP is also barely enough to be a real "protocol". It does a bunch of useful stuff but in the end raw text is pretty useless as a protocol. That's why everything from IRC to HTTP has its own protocol on top.
SCTP is a bit of a weird outlier to be placed at the level of TCP and UDP, but there's no need to go all QUIC and reinvent TCP if you want to use UDP. DNS, NTP, and plenty of other protocols don't need to rebuild reliability and streams to be useful.
tptacek
3 months ago
I mean, OK, but the inventor of UDP disagrees. I agree you don't have to build a real transport at all if you don't want to. But the moment you do so much as a sequence number that lets you detect that you've missed some packets, you've build a transport, and you're not just using UDP.
zamadatix
3 months ago
The inventor of UDP could have intended one thing 45 years ago, but that does not actually mean that's the only way it happened to be. UDP is great for building a transport protocol on top of. Most uses of UDP do treat it as such. That is not evidence UDP must not be able to qualify or be used as a transport protocol itself.
tptacek
3 months ago
I mean, sure, but then by that logic so is IP. UDP is just IP with an extra checksum and a 32-bit multiplexing field instead of an 8-bit one.
zamadatix
3 months ago
All transport protocols are "just" adding things on top of a network protocol so you can multiplex to applications instead of hosts. That's what makes them transport protocols instead of alternate network protocols. TCP is "just" adding a few more things. I kinda wish UDP didn't have the checksum field, it would have been a bit nicer to build other transports with if one could decide to validate their headers with one's desired checksum approach (or not bother, if so desired) rather than have 2 separate checksums for the transport layer role. You could at least blank it for wasted space riding on v4, then v6 mandated it for some reason.
UDP also isn't the only transport protocol used extensible either, it's just far more common because it has fewer assumptions. E.g. TCP has MPTCP and a bolt-on transport extension.
tptacek
3 months ago
OK. But if you put a 32-bit header with a sequence number and an "ACK requested" flag on top of UDP, and arrange to send NACKs when you see skips in the sequence numbers, you're not using UDP; you're using a transport you built on top of UDP.
This isn't just a semantic argument. People getting this conceptually wrong has broken the deployment story for things like SCTP, which for no good reason rides on top of IP directly and thus gets blocked by middleboxes.
zamadatix
3 months ago
The disagreement is in that being the ONLY way UDP can be used is in building a more complex transport layer, not that building a more complex transport layer on top of UDP means it's still just UDP alone. Any transport protocol can be built upon as such, even including transport protocols already built atop UDP. That UDP can be layered on top of does not redefine UDP's ability to be a complete transport layer protocol in itself. UDP cannot be used as a network layer protocol, so it is not one, but UDP provides multiplexing (source+dest based), datagram sizing independent of the network layer, and even header checksumming on top of a network layer protocol - making it a (relatively) minimal transport protocol. The ability to be a transport layer protocol is not defined by the lack of ability to be built on top of to make a more complex transport layer on top of the given protocol.
A classic practical example of "plain Jane UDP transport" might be an traditional SNMP trap (not one of the newer fancy flavors). No ACK, no bidirectional session setup, no message ID/sequence number, no retransmission, no congestion control - just a one shot blast of info to the given destination - split into datagrams and multiplexed to the receiving service via the transport layer over an arbitrary network layer.
A lot of things broke SCTP... but I'd rather not get into a side debate about what those reasons are. Just that it's not because UDP alone is unusable as a transport layer protocol.
tptacek
3 months ago
I honestly think a lot of stuff like SNMP's UDP semantics are based on faulty path dependence from people believing they had basically two options, either TCP's rigid sequential delivery and HOLB, or yolo-mode UDP.
zamadatix
3 months ago
On the contrary, I think it was clearly a very intentional and deliberate decision to not add complexity (it is, after all, "Simple" Network Management Protocol) to the transport layer of SNMP. At the very least, the people involved certainly had alternative transports in consideration rather than a mistaken view of TCP vs UDP alone. Additional transport requirements were just not relevant to the use cases of SNMP. Select excerpts from RFC 1157:
> Consistent with the goal of minimizing complexity of the management agent, the exchange of SNMP messages requires only an unreliable datagram service, and every message is entirely and independently represented by a single transport datagram. While this document specifies the exchange of messages via the UDP protocol [11], the mechanisms of the SNMP are generally suitable for use with a wide variety of transport services.
From this, the authors intentionally kept things within a single datagram across any unreliable datagram service - UDP was just an obvious choice to define for the needs.
> In the text that follows, the term transport address is used. In the case of the UDP, a transport address consists of an IP address along with a UDP port. Other transport services may be used to support the SNMP. In these cases, the definition of a transport address should be made accordingly.
They continued to account for and allow for how other generic transport protocols could be used (at the time, not as many), rather than assume they only had the options of TCP or UDP.
> In cases where an unreliable datagram service is being used, the RequestID also provides a simple means of identifying messages duplicated by the network.
This shows other portions of SNMP did account for which specific features may need to be built on top of a minimal transport protocol and only added those to the specific PDUs which needed it. E.g. for this request based PDUs used by Get/GetNext/GetBulk etc they intentionally added an ID to handle message duplication, just not to every PDU, like traps, because it was unnecessary.
> A limited number of unsolicited messages (traps) guide the timing and focus of the polling. Limiting the number of unsolicited messages is consistent with the goal of simplicity and minimizing the amount of traffic generated by the network management function.
This shows the design of traps was heavily focused around simplicity and minimization of traffic, not based on what TCP or UDP could specifically offer. In fact, you won't find a mention of "checksum" or "hash" anywhere either in the RFC - UDP just had it as extra cruft on top of that generic "unreliable datagram service" they were designing against!
.
SNMPv3 did eventually add TCP as an option for traps a couple decades later, and hardly anyone ever opted to use it since as there really isn't much benefit from other transports for the use case. More have used the TLS option, but even more have just relied on the minimal, purpose defined encryption and HMACs added instead.
Thanks for this discussion by the way, there is nothing more I love working with or talking about than network protocol design and history :).
ekr____
3 months ago
I agree with you about the category error.
In all fairness, though, there are quite a few application protocols which are built directly on top of UDP with no explicit intermediate transport layer. DNS, RTP, and even sometimes SIP come immediately to mind.
tptacek
3 months ago
I'd argue that DNS has an implicit transport protocol by dint of the QIDs. It's a very simple transport! But that's the point, right.
ekr____
3 months ago
I think we agree here: protocols built on top of UDP often build their own ad hoc reliability layer, as opposed to having an explicit one like TCP, QUIC, or SCTP.
juliusdavies
3 months ago
SIP still has a reliability layer built on top: the humans will say “I didn’t catch that, can you repeat yourself?“ on the video call.
jeroenhd
3 months ago
More so than that, SIP itself has a ACK system to resend dropped packets or report failures.
The RTP part will carry the "can you repeat that" part, unless you're sending DM's over SIP INFO messages.
user
3 months ago
atoav
3 months ago
There are legit use-cases for "fire-and-forget"-protocol. E.g. anything that is more about "hey, here is a new value, just like the past 100.000 I sent within the last second" than about "here is a critical command where it would really suck if we didn't know about it not arrive at its target system.
Not caring about whether every single transmission arrives is a totally legitimate technological choice and it can have some tangible benefits when it comes to throughput, latency etc.
As with every piece of technology UDP gets problematic when you are ignorant towards its limitations and misapply it.