martinald
4 months ago
This really is a function of two things:
1) (Mainly) the huge increase in upstream capacity of residential broadband connections with FTTH. It's not uncommon for homes to have 2gbit/sec up now and certainly 1gbit/sec is fairly commonplace, which is an enormous amount of bandwidth compared to many interconnects. 10, 40 and 100gbit/sec are the most common and a handful of users can totally saturate these.
2) Many more powerful IoT devices that can handle this level of attack outbound. A $1 SoC can easily handle this these days.
3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
bsder
4 months ago
> If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
Suddenly, everybody (ISPs, carriers, end users) can blackhole a compromised IP and/or IP range without affecting non-compromised endpoints.
And DDoS goes poof. And, as a bonus, we get the end to end nature of the internet back again.
lgeek
4 months ago
From having worked on DDoS mitigation, there's pretty much no difference between CGNAT and IPv6. Block or rate limit an IPv4 address and you might block some legitimate traffic if it's a NAT address. Block a single IPv6 address... And you might discover that the user controls an entire /64 or whatever prefix. So if you're in a situation where you can't filter out attack trafic by stateless signature (which is pretty bad already), you'll probably err on the side of blocking larger prefixes anyway, which potentially affect other users, the same as with CGNAT.
Insofar as it makes a difference for DDoS mitigation, the scarcity of IPv4 is more of a feature than a bug.
zamadatix
4 months ago
(Having also worked on DDoS mitigation services) That "entire /64" is already hell of a lot more granular than a single CG-NAT range serving everyone on an ISP though. Most often in these types of attacks it's a single subnet of a single home connection. You'll need to block more total prefixes, sure, but only because you actually know you're only blocking actively attacking source subnets, not entire ISPs. You'll probably still want something signature based for the detection of what to blackhole though, but it does scale farther in a combo on the same amount of DDoS mitigation hardware.
spongebobstoes
4 months ago
you can heuristically block ipv6 prefixes on a big enough attack by blocking a prefix once a probabilistic % of nodes under it are themselves blocked, I think it should work fairly well, as long as attacking traffic has a signature.
consider simple counters "ips with non-malicious traffic" and "ips with malicious traffic" to probabilistically identify the cost/benefit of blocking a prefix.
you do need to be able to support huge block lists, but there isn't the same issue as cgnat where many non-malicious users are definitely getting blocked.
swinglock
4 months ago
You should block the whole /64, at least. It's often a single host. It's often but not always a single host, that's standardized.
vladvasiliu
4 months ago
Usually a /64 is a "local network", so in the case of consumer ISPs that's all the devices belonging to a given client, not a single device.
Some ISPs provide multiple /64s, but in the default configuration the router only announces the first /64 to the local network.
TZubiri
4 months ago
Presumably a compromised device can request arbitrarily new ipv6 from the dhcp so the entire block would be compromised. It would be interesting to see if standard dhcp could limit auto leasing to guard reputation of the network
vladvasiliu
4 months ago
Generally, IPv6 does autoconfiguration (never seen a home router with DHCPv6), so no need to ask for anything. Even for ipv4, I've never seen a home router enforce DHCP (even though it would force the public ip).
But the point stands, you can't selectively punish a single device, you have to cut off the whole block, which may include well-behaved devices.
swinglock
4 months ago
In mobile networks it's usually a single device.
bsder
4 months ago
This DDoS is claimed to be the result of <300,000 compromised routers.
That would be really easy to block if we were on IPv6. And it would be pretty easy to propagate upstream. And you could probabilistically unblock in an automated way and see if a node was still compromised. etc.
josteink
4 months ago
> That would be really easy to block -- if we were on IPv6.
Make that: If the service being attacked was on IPv6-only, and the attacker had no way to fall back to IPv4.
As long as we are dual-stack and IPv6 is optional, no attacker is going to be stupid enough to select the stack which has the highest probability of being defeated. Don't be naive.
div72
4 months ago
It'd be far more acceptable to block the CG-NAT IPv4 addresses if you knew that the other non-compromised hosts could utilize their own IPv6 addresses to connect to your service.
TZubiri
4 months ago
Better to rely on ip blocks than on NAT to bundle blocks.
ralnivar
4 months ago
I am a bit split this topic. There is some privacy concerns with using ipv6. https://www.rfc-editor.org/rfc/rfc7721.html#page-6
Some time ago I decided for our site to not roll out ipv6 due to these concerns. (a couple of million visitors per month) We have meta ads reps constantly encourage us to enable it which also do not sit right with me.
Although I belive fingerprinting is sofisticated enough to work without using ip's so the impact of using ipv6 might not be a meaningful difference.
miyuru
4 months ago
its hilarious that you have privacy concerns while at the same time using meta ads.
Xss3
4 months ago
I am guessing they're trying to limit the privacy harm to normal channels that the slightly savvy can understand rather than completely eliminate it.
GoblinSlayer
4 months ago
Reportedly this is often incorrectly implemented, where /64 prefix is still a stable static address.
nine_k
4 months ago
Is there any money an ISP would make, or save, by sinking money and effort on switching to IPv6? If there's none, why would they act? If there is some, where?
For instance, mobile phone operators, which had to turn ISPs a decade or two ago, had a natural incentive to switch to IPv6, especially as they grew. Would old ISPs make enough from selling some of their IPv4 pools?
rendaw
4 months ago
Presumably they'd lose money when a DDoS originating from their network causes all their ips to get blocked.
beeflet
4 months ago
less expensive IP space, more efficient hardware, and lower complexity if you can eliminate NAT.
ROBLOX_MOMENTS
4 months ago
They already lease them out. TELUS in Canada traditional old ISP rents large portion of their space to a mostly used for Chinese GFW VPN server provider in LA „Psychz“
TZubiri
4 months ago
The ISPs have to submit plans on how to use their IPs for the public,especially for IPv4, Arnic shouldn't approve this kind of stuff. Unless they lied in their ip block application, in which case they should be revoked their block.
Braxton1980
4 months ago
I filled out one of these for Cogent to get a /24. I was being honest but all I had to put was services that requires their own IP. I even listed a few but no where near the 253.
They also never responded back and were like "what about NAT" or "what about host based routing".
TZubiri
4 months ago
Not sure what you filled out, but blocks are handed usually not to end users, but to providers that will sublease the ips to their client. So if you are asking for a block for a couple of your HTTP servers, that's a no. If you rent HTTP servers to, say, local small businesses, then that's a yes.
user
4 months ago
josteink
4 months ago
> How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
That depends on the service you are DDosing actually having an IPv6 presence. And lots of sites really don't.
It doesn't help if you have IPv6 if you need to fallback to IPv4 anyway. And if bot-net authors knows they can hide behind CGNAT, why would they IPv6 enable their bot-load when all sites and services are guaranteed to be reachable bia IPv4 for the next 3 decades?
(Disclaimer: This comment posted on IPv6)
rectang
4 months ago
Is it advantageous to be someone who supports IPv6 on a day like today?
createaccount99
4 months ago
Isn't it enough that the target of the DDOS only accepts ipv6?
toast0
4 months ago
> 3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
Null routing is usually applied to the targets of the attack, not the sources. If one of your IPs is getting attacked, you null route it, so upstream routers drop traffic instead of sending it to you.
martinald
4 months ago
Sorry, late here. You are right. I mean filter the IP in question.
user
4 months ago
idiotsecant
4 months ago
Haha that last part is pretty wild. rather than worrying about systemic problems in the entire internet let's just make mandates crippling devices that China, where all these devices are made, will defffinitely 100% listen to. Sure, seems reasonable. Systems that rely on the goodwill of the entire world to function are generally pretty robust, after all.
saagarjha
4 months ago
If they don’t then the devices are not sold in the United States. It’s quite simple.
dylan604
4 months ago
Great to know that smuggling hardware into the US has been completely stopped.
morsch
4 months ago
If the analysis above is accurate, a few smuggled devices would not be an issue, as long as the zillions of devices sold at Walmart are compliant.
idiotsecant
4 months ago
Congratulations on the creation of a thriving new black market in which the main beneficiary is organized crime! What could go wrong?
saagarjha
4 months ago
Do you take issue with the concept of laws or are you just being annoying?
idiotsecant
4 months ago
I'm sorry that you find thinking about second order dynamics annoying, but that's what you have to do if you actually want effective laws. Just making laws doesn't magically fix problems. In many cases it just makes much more exciting problems.
saagarjha
4 months ago
I'm annoyed because you didn't actually come up with an interesting response. Yes, when you make laws people can break them. But you need to explain why there is an incentive to break them, and whether it will happen to the extent that it will actually be a problem to enforce. Personally, I don't see people scrambling to get DDoS attack vectors in their house by any means necessary.
gjsman-1000
4 months ago
> I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from. They'll point to whether your Mac really needs more than 100mbps.
The government is far more likely to figure it out along EU lines: Signed firmware, occasional reboots, no default passwords, mandatory security updates for a long-term period, all other applicable "common sense" security measures. Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
ShowalkKama
4 months ago
>What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from.
any source for this claim? Outside of very specific scenarios which differ significantly for the current botnet market (like manjaro sending too many requests to the aur or an android application embedding an url to a wikipedia image) I cannot remember one occourence of such a bug being versatile enough to create a new whole cybercrime market segment.
>They'll point to whether your Mac really needs more than 100mbps.
it does, because sometimes my computer bursts up to 1gbps for a sustained amount of time, unlike the average iot device that has a predictable communication pattern.
>Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
gjsman-1000
4 months ago
Historically, it was called Windows XP and Vista about 15 years ago (Blaster, Sasser, MyDoom, Stuxnet, Conficker?). Microsoft clamped down, hard, across the board, but everyone outside of Big Tech is still catching up.
Despite Microsoft's efforts, 911 S5 was roughly 19 million Windows PCs in 2024, in news that went mostly under the radar. It spread almost entirely through dangerous "free VPN" apps that people installed all over the place. (Why is sideloading under attack so much lately? 19 million people thought it would make them more secure, and instead it turned their home internet into criminal gateways with police visits. I strongly suspect this incident, and how it spread among well-meaning security-minded people, was the invisible turning point in Big Tech against software freedom lately.)
https://www.fbi.gov/investigate/cyber/how-to-identify-and-re...
> if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
Which is more important, and a growing threat? Dump all her photos once; or install a disguised app that pretends to be a boring stock app nobody uses, that provides ongoing access for years, with everything in real-time up to the minute? Increasingly it's the latter. She'll never suspect the "Samsung Battery Optimizer" or even realize it came from an APK. No amount of sandboxing and permissions can detect an app with a deliberately false identity.
pjc50
4 months ago
> Signed firmware and the sideloading ID requirements
Ending the last corner of actually free market in software is quite a cost for something that wouldn't prevent DDoS.
> sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from
Is that actually true? What evidence do we have, vs. vulnerabilities in the OEM software (the more common case)?
high_na_euv
4 months ago
> A $1 SoC can easily handle this these days.
Could you elaborate?
pjc50
4 months ago
I think there's some exaggeration as few $1 SoC parts come with 10G Ethernet, and >1G to the home is not common, but pretty much any home router can saturate its own uplink - it would be useless if it couldn't!
sekh60
4 months ago
Not always the case. Generating traffic can be more computationally intense than routing the traffic. I've done speed tests on a few routers local to it and the results have been less than stellar compared to getting expected results with it just routing traffic (consumer routers). Granted these tests were a few years ago and things have progressed, but how often are people upgrading their routers?
kees99
4 months ago
Correct.
Also, most 1Gbit/s and faster routers have hardware-accelerated packet forwarding, aka "flow offloading", aka "hardware NAT", where forwarded packets mostly don't touch software at all.
Some routers even have internal "CPU" port of packet core with significantly slower line rate than that of external ports'. So traffic that terminates/originates at the router is necessarily quite a bit slower, regardless of possibly extra-beefy processor, and efficient software. Not really a problem since that traffic would normally be limited to UI, software updates, ARP/NDP/DHCP, and occasional first packet of a forwarded network connection.
martinald
4 months ago
A Allwinner H616 is Quad-Core ARM and can definitely saturate gigabit ethernet with packet generation.
devwastaken
4 months ago
1gb upload is extraordinarily rare.
saagarjha
4 months ago
It’s not; most places that give you gigabit fiber will give you a symmetric connection.
Xss3
4 months ago
Define most places? I know i dont get one (uk) and neither does my german friend or texan friend.
I've only ever seen one despite having used 4 different ISPs for gigabit, and that one was special. It was in an apartment i rented in a converted office tower, line was done via a b2b provider then included in the rent.
saagarjha
4 months ago
I'm in the US and most fiber providers (I checked a handful: AT&T, Sonic, Google Fiber, Frontier) all provide symmetric connections.
zokier
4 months ago
Aren't most residential fiber deployments PONs which generally do not offer symmetric bandwidth? E.g. 10G-PON has 10G down / 2.5G up.
Hikikomori
4 months ago
Depends on country, its not common here.
typpilol
4 months ago
Yup. Spectrum is Michigan will give you up to 2gbps down but not anything more than 200mbps up
devwastaken
4 months ago
Nope. less than a percent of a percent. symmetric plans are extra cost and offered primarily to business.
almost all homes have no ability to exceed gigabit. infact almost all new homes dont even have data wiring. people just want their netflix to work on wifi.
saagarjha
4 months ago
I didn't pay anything extra for symmetric.
vitaflo
4 months ago
Most places do not have fiber.
Dylan16807
4 months ago
We know. The problem is that the above comment said "extraordinarily rare" which is a very different and incorrect threshold.
dylan604
4 months ago
But for those that do...symmetric is the norm. The number of fiber connections is only going up.
devwastaken
4 months ago
symmetric is not the norm. the infra costs are not worth it. symmetric is primarily a business offering.
ls612
4 months ago
This is probably technically true but very misleading. Fiber penetration in the US has been consistently rising for over a decade now and it is not at all uncommon to have either Google Fiber, Fios, or a local fiber provider available to you in a big city. I bet within the next decade most places will have gigabit fiber available.
Xss3
4 months ago
There are probably more English speakers using the Internet in India than there are in the USA...Let alone the hundreds of millions elsewhere.
You cant just assume everyone is talking about your country online.
shagmin
4 months ago
Does it really matter? The grandparent comment states the bandwidth is becoming even more readily available in the US, while the article itself says the bots were largely hosted by US ISPs, and that's obviously enough bandwidth to already cause global disruptions. But that's just the source of the attack, and who is on the receiving end is another.
I get being too US-centric, but I think it's interesting if the US has the right combination of hosting tons of infected devices and having the bandwidth to use them on a much larger scale compared to other countries and possible implications.
Cody-99
4 months ago
You can assume the county when it is in the title.
>DDoS Botnet Aisuru Blankets US ISPs in Record DDoS
_carbyau_
4 months ago
The US is a big place. But the world is bigger. The internet works across the whole world.
There's a long way to go before fibre is commonplace across the world.
nick32661123
4 months ago
Seems more likely that residential modems will be required to use ISP-provided equipment that has government mandated chips, firmware, etc to filter outbound traffic for DDoS prevention.