One of the most popular API frameworks is called “FastAPI” but in any meaningful benchmark it’s one of the slowest commercially used pieces of web software the industry has adopted.

There is no hope for large segments of the industry. We actively choose to make bad decisions all the time, and then people rationalize them.

> Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware.

At that time, it was quite common to start a new thread or even fork a new process per request/connection. Apache was prone to this:

> https://www.digitalocean.com/community/tutorials/apache-vs-n...

"mpm_prefork: This processing module spawns processes with a single thread each to handle requests. Each child can handle a single connection at a time. [...]

mpm_worker: This module spawns processes that can each manage multiple threads. Each of these threads can handle a single connection. [...]"

(this website also mentions the mpm_event MPM).

So, if you have ten thousands of processes or threads on some web server, this can easily overstrain the resources of the web server. Additionally keep in mind that at that time servers were a lot less beefy than today.

2003: http://www.geology.smu.edu/~dpa-www/attention_span. 22 Requests / second was a big deal.

Perhaps the best benchmark for the maximum potential traffic of the community at that time comes from a Slashdot post by Rob Malda (CmdrTaco) on September 14, 2001 (https://news.slashdot.org/story/01/09/13/154222/handling-the...), detailing the site's traffic following the 9/11 attacks. This load on Slashdot itself represents the full attention of the userbase and was higher than the site normally experienced:

Normal Load: 18–20 dynamic page views per second.

Peak Load: 60–70 page views per second.

Daily Volume: 3 million page views (up from a daily average of 1.4 million).

To handle this higher load, Slashdot itself had to disable dynamic content and serve static HTML to survive this load. Their database could not handle the query volume.

A typical "Slashdotting" pushed 5 to 10 Mbps of traffic. Since many sites were hosted on T1 lines (1.5 Mbps), the pipe was instantly clogged, resulting in 100% packet loss for legitimate users.

I remember when Slashdot broke because they used an Unsigned Mediumint counter for a primary key and finally overflowed it with 16 million comments over the life of the site on Nov 6, 2006.

It was a different time back then when we still measured our installed RAM in megabytes.

> Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware.

You're grossly overstating the capacity of a server to meet peak load demands, specially when hit with unexpected load spikes. The slashdot effect was a phenomenon observed two decades ago, in a time when the bulk of the web was served as Apache running CGI scripts. The rate of requests any random server could process was around ~300requests per second, and some of the links being hugged to death were literally served from a box under some dude's desk.

At the time, a standard LAMP stack setup wasn't prepared for the https://en.wikipedia.org/wiki/C10k_problem. Quite often you had no DB connection pool and every request for dynamic data opened/closed a DB connection. Having a fixed number of connections available also quickly hit the limits unless it was a dynamically configured pool. Which also filled up eventually.

If you were lucky you could reduce load by caching the URLs taking up the most resources and generate a webpage copy in the filesystem, then add some URL rewrite logic to Apache to skip going through your application logic and bypass the DB.

Then you discovered there is a limit of open file descriptors in Linux. After updating this and also shutting down all log files you ran out of things to change pretty quick and started pricing a beefier server. For next time.

Many blogs are completely dynamic and grab content out of a database on every request. The static site generation style fell out of early fashion when wordpress took over from moveable type, and didn't really return much until jekyll. Even today I think most blogs you see are mostly dynamic with the platforms using caching to avoid hitting the database every time.

Most people don't do performance tuning of their blogs because the average traffic is miniscule. This means that any configuration issues rear their heads when load happens. For example, having your max connections to the web server be far more than the database can support. Perhaps the machine could handle the load if properly configured, but no one did so because there was no need.

/. effect is basically a ddos, same principles apply. basically sockets need to be created and torn down faster than incoming requests and they just aren't; they get queued up for a while and/or kill the host if it isn't configured correctly and keeps trying serving traffic which it can't due to resource exhaustion. whether it's a 'modern stack' requiring hundreds of MB of ram per request or a C CGI program matters only quantitatively.

In the days of Apache 1.3 in the early 2000s for example every client session had a memory footprint (and in the case of php processes could be in the 1-2 megabytes easy). So there you are on a dedicated piece of hardware with limited RAM and a 10,000 connections hit you at the same time. And you have no caching strategy because you're an idiot child trying your best. Boom - Slashdotted

- slower, often bare metal hardware

- lack of scalability (you can’t dynamically spin up a bare metal server if you don’t already have one in your rack)

- slower memory models (CGI, and forking meant it took longer for the OS kernel to allocate system resources)

- improvements to TCP and fork performances in the Linux kernel over the years. Such as changes to how TCP negotiation happens

- improvements to HTTP transit such as improved compression performance, HTTP/3, etc.

- CDNs weren’t common place, and were super expensive.

- caching was in its infancy. Tools like varnish (I think it was called) were complicated and setting up memcached, varnish, etc all required additional hardware, that added to hosting costs.

- HTTP servers weren’t as mature. Plus weren’t included in the language runtime (eg you’d run httpd that would fork an instance of PHP rather than run the HTTP server as part of the Python, et al)

- serving static content of NFSs mounted SAN was slow (the POSIX overhead, plus NFS protocol, plus network roundtrip, plus disk access times)

- mechanical drives were slow

And so on and so forth.

The biggest difference I think was a combination of affordable or free CDNs, reducing pressure on the backend, and the easy of scaling (VM, Docker, etc) meaning you don’t need to find a happy middle ground between costs and high availability.

In my experience, it's usually the database that gives out first. It's often a shared database, or one running on overprovisioned hardware.

The kinds of sites that go down when receiving unexpected traffic are usually built in such a way where they're making multiple DB requests to render each page. Or they have a dozen poorly configured perf-expensive WordPress plugins running. Or likely all of the above

Oh I remember those days. In our case, it was because we had so much website data that we couldn't keep it all in cache, so when the site became busy enough, performance fell off a cliff as we became disk I/O bound. We had maxed out our main server with 512MB of RAM.

We couldn't afford the new Sun Ultra 2's that would have given us a spacious 2GB of RAM, so we ended up coding a front-end to shard requests across the 2 web servers and used a Sparcstation 5 to run that front-end.

Eventually we rearchitected the entire site to move the static HTML websites into an Oracle database. (by then we had upgraded to a Sun Ultra Enterprise 3000, which could handle up to 6GB of memory, more than we could ever use)

That's what's possible today. It wasn't back then (see: https://en.wikipedia.org/wiki/C10k_problem)

The other reason is, not every web server, web application, database, etc is optimized for large number of connections. Sometimes they take up too much CPU and memory. Sometimes each page request triggers a connection to a database, so the database connection limit is hit. Sometimes the queries are inefficient and slow the database to a crawl. Sometimes the sheer number of packets causes so many interrupts that it sucks CPU away from the application/database. And sometimes the network connection of the webserver was just slow.

Most people hit by the /. effect were not just a simple web server serving static content. Often it was a php website with a mysql database, on a server less powerful than a 2015 smartphone.

You forget that the vast majority of the web in the 2005 era was blogs hosted on wordpress by either service providers or shared hosting providers, neither of which dedicated significant attention to performance. Many of them were hitting quota limits, but many still were truly just going down because of the oversubscribed hardware they were running on. Even an independent server that was properly set up though could hit issues. I had a few of my posts get slashdotted at that time, and the spike was real. Apache hosting wordpress couldn't quite keep up with the spike - I had to use varnish to set up a reverse proxy cache to properly handle the influx.

there used to be a lot more shared hosting in the world when Slashdot ruled the geek news roost

it'd be fine if it's one site on a dedicated machine, but these shared webhosts would routinely cram 500-1000 hosting accounts onto something like a single-core, HT-less 2.4GHz Pentium 4 with two gigs of RAM, running mpm_prefork on Apache 2.0, with single-node MySQL 5.x (maybe even 4.x!) on the same box... which was not terribly efficient as others observed

you carry about 20x the compute power of that machine in your pocket, even a Raspberry Pi 4 could triple the workload

The webserver was fast but PHP in the days before fast CGI didn't take all that much load to bring the server to its knees. And we didn't have cloud hosting at the time either so dynamic scaling behind a network load balancer wasn't a thing (and running multiple servers behind a reverse proxy was uncommon and expensive).

At the time Slashdot was a thing, web servers could serve dozens, maybe a hundred requests per second.

Near to the end of (real) Slashdot's life, project 10k was happening. People were rewriting even the Linux process management to make servers perform close to their theoretic capacities.

Hell, if the connection timeouts are long enough and you're waiting for some pooled DB connection, you could run out of ports even if your throughput was high enough.

There are actually a lot of little things to tune if you can't just auto-scale out of a bottleneck.

Hit a webserver with many thousands or tens of thousands of requests per second, and whatever the weakest link is will break: memory usage, CPU usage, database, bandwidth, billing, third-party service quotas...

> Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware.

Back then it was Wordpress/PHP with no caching.

But: you're right.

Once upon a time named Maxim made a web page that pointed out that a Pentium III should be able to handle 10K simultaneous requests, then said Apache was shit, and that he had made his own webserver, and because he was Russian he filled the webpage with animated hammers and sickles.

This was how NGINX was created.

yeah, it's around, but a ghost of its former glory.

>Makes me wonder what percentage of today's HN readers didn't live through the Slashdot era.

Please don't out me as a ghost parent

> Makes me wonder what percentage of today's HN readers didn't live through the Slashdot era.

I definitely missed out :(

What was it like?

Not me. I'm 86,000-something.

You must be new here

The name changed when Slashdot lost significant mindshare among tech nerds.

Back in the early 2000s there weren't that many sites that could direct that sort of traffic to a website.

that's the redditified babytalk name of the Slashdot effect

Jesus Christ you guys need a life