Game downloads can be tens of GiB per day, and are only growing. Multiple 4K streams can add up. I wouldn't be surprised if folks are saturating links during prime time hours.

I worked for a major UK ISP for a while, about 1% of customers fully utilise the connectivity they get.

One of the bigger issues for us, was getting enough backhaul capacity from the wholesaler (BT OpenReach) at smaller regional locations. Also we were obligated to pay for multicast traffic on a delivered bandwidth basis per user, even though multicast isn't unicast and so the overall network traffic was only individual on the last mile!

I have absolutely no issue with contention as long as it doesn't substantively affect my experience. Some users get very upset about "ping times" but there's also evidence that the numbers in games aren't actually practically useful to a point (unless they're objectively large). ISTR one game developer fudged the ping numbers to stop people complaining about competitiveness.

In China they have something called PeerCDN which does this and pays people money to use their upload. ISPs disconnect subscribers who are caught using it. To avoid suspicion from up/down ratio as some ISPs use, many PeerCDN subscribers also download lots of dummy data from random internet servers, increasing network load substantially.

As a civilization we really under utilize distributed models for several solutions.

Respectfully, nah. I don’t want to troubleshoot compute nodes in varying states of abuse. I’d rather deal with 10,000 of them crammed into a dc for consistency purposes.

I am personally of two minds about it, because while I might be partial to the example mentioned, I can immediately tell how it would in real life. It would be just like with the Simpsons[1].

"Oh, please. For a nickel a person tax increase... we could build a theater for shadow puppets. Balinese or Thai? Why not both? Then everybody's happy."

In other words, no one will be happy. Let individuals keep making individual decisions.

There's Peer5, which got acquired by Microsoft and got rebranded "ECDN":

https://peer5.com/

https://news.ycombinator.com/item?id=13501581 Launch HN: Peer5 (YC W17) – Serverless CDN

https://www.ycombinator.com/companies/peer5 P2P delivery network that enables high quality video streaming

Also PeerCDN apparently, which got acquired by Yahoo?

https://web.archive.org/web/20150810065820/https://peercdn.c...

If you run a Web search for some combination of "P2P", "Peer", and "WebRTC", combined with "CDN", you can also find a smattering of other (vaguely spammish) company blogs and (small, personal) GitHub libraries that seem to talk about something similar:

https://www.w3.org/wiki/Networks/P2P_CDN

https://github.com/AgustinSRG/webrtc-cdn

https://mediafoundation.medium.com/the-rise-of-peer-assisted...

https://github.com/vardius/peer-cdn

https://www.cachefly.com/news/deciphering-p2p-the-basics-and...

https://castr.com/blog/ecdn-or-p2p-cdn/

https://github.com/Peer-to-Peer-CDN/P2P-CDN

https://blog.blazingcdn.com/en-us/next-generation-content-de...

So there appears to actually have been quite a bit of "concerted effort" in this direction, both open-source and with company funding. Either it's kept fizzling out for some reason, or became standard and is now taken for granted.

IMO the most prominent project in this direction is PeerTube, in that it's using idle upstream bandwidth as a big part of its pitch (It's in the name!):

https://en.wikipedia.org/wiki/WebTorrent

https://en.wikipedia.org/wiki/PeerTube

https://news.ycombinator.com/item?id=38443855#38444796 (V6 dropped WebTorrent but still does "P2P via WebRTC".)

But there are some privacy and security issues with this idea too. What happens if your website uses up all of a customer's data plan by uploading to peers when they leave it open in the background? And do you really want every webpage you visit to be NAT-hole-punching, announcing your presence, and uploading who-knows-what data to random strangers?

A surveillance op could construct a list of everybody who visits a given page by simply refreshing the page every few seconds and logging the peers it connects to. And a vulnerability anywhere in the WebRTC stack, from router to browser, could let you get `pwn`ed by a random peer even if the website you visit is clean.

That's DRM for you. If you pay, you get a worse experience.

That can also be affected by the supported DRM level. Try safari or edge.

[dead]

Many residential fiber deployments are PON with some amount of overbooking on the last mile. If the fiber is 2.5g down/1.25g up and split 4 ways, not everyone can have the full 1g/1g service.

But even if you have point to point fiber, that probably has an upstream connection that's less than the aggregate bandwidth of the end user connections it manages. And so on until you get through your providers network to internet transit/peering. And once you get through to the hosting network too.

I ran hundreds of 1g/10g boxes at a facility with 80g aggregated to the world.

Those boxes would be 40 or so to a rack, with 2x10g to the switch (800gbps aggregate) and probably 4x10g to the upstream switch. Maybe 4x40g to the upstream.

There is no provider network (or even non-trivial corporate network) without over-subscription somewhere. It is completely impractical to achieve. Consider a simple Ethernet network with 4 nodes each with a 10gbps connection:

    A
  B # C
    D

Ok, you say, I'm just worried about ISP networks, not some free-for-all hypothetical Ethernet network where the nodes want to talk to each other. All the subscriber nodes just want to talk to the uplink, not each other:

  ----+ 100 gbps uplink +----
  | DSLAM or similar device |
  ----+-+-+-+-+-+-+-+-+-+----
      | | | | | | | | | |
      A B C D E F G H I J ...

   [                      uplink switch                      ]
       +         +         +         +         +         +
   [ DSLAM ] [ DSLAM ] [ DSLAM ] [ DSLAM ] [ DSLAM ] [ DSLAM ] ...

From there, at a typical big ISP, you'd have connections going toward (directly, or via another intermediate location) two different exchange points. And there, the math, which is already completely implausible, becomes totally bonkers. The reason being that a typical ISP wants to do what's called settlement-free peering to exchange traffic with other providers, rather than buy connectivity from one or more of them.

So, to keep our network "oversubscription free", whatever max uplink (likely in the thousands of terabits per second range at this point, at the very least,) has to be provisioned with every single provider our hypothetical ISP peers with. After all, we aren't doing any statistical traffic analysis. We're just making sure that there is no point of contention between any of our customers and any provider we peer with, anywhere in our network.

Even if you take peering out of the equation, all you've done is move the problem around: a provider could buy, say, 10,000 tbps of bandwidth from another provider, I guess, but then that provider which is peering with hundreds of other ISPs has the same problem. Otherwise, its just a shell game where provider A says "we don't oversubscribe" knowing full well their upstream provider does.

This is all completely insane. Nobody does it. What good operators do, instead, is proactively monitor their network and add capacity as needed, preferably before anything gets saturated.

Quite popular in the UK is the concept of

A) Splitter cabinets which you get a fibre back to and then get put on one of several splitters. These cabs are completely passive and come in 48 or 96 (maximum) ports (usually fed by 12f back to the exchange). You can then patch a user onto a different splitter or even a dedicated fibre back to the exchange for P2P ethernet.

B) Split at 8:1 near to the end user and then split again at the exchange, sometimes upto 8:1 again, so 1:64 on XGS-PON. The advantage of this is as a PON gets busy it's a 10 second job to change it to 2 X 4:1 splits in the exchange and so on all the way down to having upto 8 subscribers on a port. If you have a business park you can make a central decision to not further split at the exchange so they get 8:1, and you don't have to change your physical plant at all, so the field engineers build to one standard regardless of if it's domestic or business.

> Some providers have the splitters placed close to the customers

In the PON networks I've noticed, they tend to have a splitter per (serviced) pole, usually with about 4 ports. What they do in their cabinets, I don't know. If it's all undergrounded, I can't see that either. I'd guess they can swap in an 8 port splitter on a pole if needed as well.

And my observed pole density seems to naturally be increased in areas of denser housing. When I lived on a postage stamp lot, we had poles every two houses or so; my parents live in suburbia with bigger lots, but poles were still about every other house. Now that I'm semi-rural, poles seem to be farther apart, but also not all poles host service connections.

4096 would be the theoretical maximum number of subscribers. Most providers will average about half of a 32 split PON actually having subscribers on. So using this case the maths now looks like 16Mb per customer in a failure scenario, 32Mb per customer in a normal scenario. The OLT can also take 100GB unlink cards and its only the cheap folks who are buying these 4x10G cards (which are about half the price).

I think it is only inside the household (and maybe some additional restrictions, not sure), so not very useful, but IIRC Steam actually has some ability for users to provide downloads to each-other, peer-to-peer style. Applying that at a neighborhood level could be really nice.

There is always an oversubscription ratio at pretty much every hop in the network, at least in a commercial service provider network. That doesn't mean packet loss is occurring all the time, for the most part providers monitor and plan their networks so that doesn't happen, at least during steady-state non-failure.

Utilization patterns can also vary quite a bit based on geography and the type of users being served.

US cable providers do keep track of users who consume their last mile bandwidth all the time. It's typically against their terms of use, so if you are nailing up 300Gbps 24/7, you will get a letter from them and may get your connection turned off.